2024-09-13T09:01:36.301 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T09:01:36.317 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T09:01:36.398 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020 branch: reef description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901020' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi no_nested_subset: false openstack: - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd memory target: 1610612736 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: extra_packages: - rbd-nbd flavor: default sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - - client.0 seed: 248 sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 sleep_before_teardown: 0 subset: 116/128 suite: rbd suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 targets: smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE7VOPSW1CnNGqTTu6lwBvPpvTowORkm2kLGFvlCNxgVPMrzKfuIs1e7QPHN6Z4K2zQ2jJ17k6OOo4Xg6jyXCZo= smithi129.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE5s50LibChURJmLORumpBijimYXHR7QrZnPNtIhguPk2st/3rLA5Xuu7HaXHx1bwCQh6SzoLo9CPx79HW0ZPzQ= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLvWUU9S+ygGjt1DtQ4PPL453Ug0aVHB9+X+kQSXGTQI9jkAyTz3uh/I1zj1oD+eE48vgby6faEb99XcOJMEpBA= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 holebdy: 512 nbd: true ops: 6000 punch_holes: true readbdy: 512 truncbdy: 512 writebdy: 512 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_22:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T09:01:36.398 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa; will attempt to use it 2024-09-13T09:01:36.399 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa/tasks 2024-09-13T09:01:36.399 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T09:01:36.401 INFO:teuthology.task.internal:Checking packages... 2024-09-13T09:01:36.421 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '48a7de7e6e1701a88fd571c7e34908ac4b8ed412' 2024-09-13T09:01:36.422 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T09:01:36.422 INFO:teuthology.packaging:ref: None 2024-09-13T09:01:36.422 INFO:teuthology.packaging:tag: None 2024-09-13T09:01:36.422 INFO:teuthology.packaging:branch: reef 2024-09-13T09:01:36.422 INFO:teuthology.packaging:sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:01:36.422 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-13T09:01:36.677 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-843.g2c01458a 2024-09-13T09:01:36.679 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T09:01:36.685 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T09:01:36.685 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T09:01:36.694 INFO:teuthology.task.internal:Saving configuration 2024-09-13T09:01:36.706 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T09:01:36.712 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T09:01:36.737 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 08:55:52.860145', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE7VOPSW1CnNGqTTu6lwBvPpvTowORkm2kLGFvlCNxgVPMrzKfuIs1e7QPHN6Z4K2zQ2jJ17k6OOo4Xg6jyXCZo='} 2024-09-13T09:01:36.759 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 08:55:52.862509', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE5s50LibChURJmLORumpBijimYXHR7QrZnPNtIhguPk2st/3rLA5Xuu7HaXHx1bwCQh6SzoLo9CPx79HW0ZPzQ='} 2024-09-13T09:01:36.783 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 08:55:52.861539', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLvWUU9S+ygGjt1DtQ4PPL453Ug0aVHB9+X+kQSXGTQI9jkAyTz3uh/I1zj1oD+eE48vgby6faEb99XcOJMEpBA='} 2024-09-13T09:01:36.783 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T09:01:36.789 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-09-13T09:01:36.789 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2024-09-13T09:01:36.790 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['client.0'] 2024-09-13T09:01:36.790 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T09:01:36.877 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcbde97f640>, signals=[15]) 2024-09-13T09:01:36.877 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T09:01:36.883 INFO:teuthology.task.internal:Opening connections... 2024-09-13T09:01:36.883 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T09:01:36.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:01:36.962 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T09:01:36.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:01:37.035 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T09:01:37.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:01:37.109 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T09:01:37.115 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-09-13T09:01:37.134 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-09-13T09:01:37.134 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-09-13T09:01:37.192 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-09-13T09:01:37.192 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9" 2024-09-13T09:01:37.192 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-09-13T09:01:37.192 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-09-13T09:01:37.192 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9" 2024-09-13T09:01:37.192 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9" 2024-09-13T09:01:37.192 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T09:01:37.192 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-09-13T09:01:37.192 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon" 2024-09-13T09:01:37.193 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T09:01:37.193 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-09-13T09:01:37.193 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T09:01:37.193 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T09:01:37.193 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T09:01:37.194 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-09-13T09:01:37.218 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-09-13T09:01:37.236 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-09-13T09:01:37.236 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-09-13T09:01:37.297 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Stream" 2024-09-13T09:01:37.297 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="9" 2024-09-13T09:01:37.297 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2024-09-13T09:01:37.298 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2024-09-13T09:01:37.298 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="9" 2024-09-13T09:01:37.298 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el9" 2024-09-13T09:01:37.298 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T09:01:37.298 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2024-09-13T09:01:37.298 INFO:teuthology.orchestra.run.smithi129.stdout:LOGO="fedora-logo-icon" 2024-09-13T09:01:37.298 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T09:01:37.298 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://centos.org/" 2024-09-13T09:01:37.298 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T09:01:37.298 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T09:01:37.298 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T09:01:37.299 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-09-13T09:01:37.318 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-09-13T09:01:37.337 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-09-13T09:01:37.337 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-09-13T09:01:37.395 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2024-09-13T09:01:37.395 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2024-09-13T09:01:37.395 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2024-09-13T09:01:37.395 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2024-09-13T09:01:37.395 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2024-09-13T09:01:37.395 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2024-09-13T09:01:37.396 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T09:01:37.396 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-09-13T09:01:37.396 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2024-09-13T09:01:37.396 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T09:01:37.396 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2024-09-13T09:01:37.396 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T09:01:37.396 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T09:01:37.396 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T09:01:37.396 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-09-13T09:01:37.418 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T09:01:37.429 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T09:01:37.434 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T09:01:37.435 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2024-09-13T09:01:37.437 DEBUG:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2024-09-13T09:01:37.440 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-09-13T09:01:37.456 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T09:01:37.462 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T09:01:37.462 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2024-09-13T09:01:37.495 DEBUG:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2024-09-13T09:01:37.497 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-09-13T09:01:37.611 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T09:01:37.685 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T09:01:37.698 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T09:01:37.698 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T09:01:37.699 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T09:01:37.699 DEBUG:teuthology.orchestra.run.smithi129:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T09:01:37.700 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T09:01:37.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:01:37.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:01:37.718 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-09-13T09:01:37.718 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-09-13T09:01:37.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:01:37.719 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-09-13T09:01:37.775 INFO:teuthology.orchestra.run.smithi129.stdout:5.14.0-503.el9.x86_64 2024-09-13T09:01:37.776 INFO:teuthology.task.kernel:Running kernel on smithi129: 5.14.0-503.el9.x86_64 2024-09-13T09:01:37.776 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum install -y kernel 2024-09-13T09:01:37.778 INFO:teuthology.orchestra.run.smithi044.stdout:5.14.0-503.el9.x86_64 2024-09-13T09:01:37.778 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-503.el9.x86_64 2024-09-13T09:01:37.778 INFO:teuthology.task.kernel:Running kernel on smithi044: 5.14.0-503.el9.x86_64 2024-09-13T09:01:37.778 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum install -y kernel 2024-09-13T09:01:37.778 INFO:teuthology.task.kernel:Running kernel on smithi157: 5.14.0-503.el9.x86_64 2024-09-13T09:01:37.778 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum install -y kernel 2024-09-13T09:01:41.881 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 8.2 MB 00:02 2024-09-13T09:01:42.356 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.2 MB 00:02 2024-09-13T09:01:43.691 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-13T09:01:44.040 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 36 MB/s | 20 MB 00:00 2024-09-13T09:01:46.361 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 1.2 MB/s | 8.2 MB 00:06 2024-09-13T09:01:48.466 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-13T09:01:49.287 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-09-13T09:01:50.050 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 19 kB 00:00 2024-09-13T09:01:50.083 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.5 MB 00:02 2024-09-13T09:01:50.837 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-13T09:01:51.634 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 19 kB 00:00 2024-09-13T09:01:52.388 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-13T09:01:53.913 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 6.5 MB 00:00 2024-09-13T09:01:55.513 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 19 kB 00:00 2024-09-13T09:01:56.260 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-13T09:01:56.284 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-09-13T09:01:58.011 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-09-13T09:01:58.274 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T09:01:58.275 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T09:01:58.364 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:01:58.377 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:01:58.378 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-09-13T09:01:58.378 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:01:58.378 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:01:58.378 INFO:teuthology.orchestra.run.smithi157.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T09:01:58.378 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-09-13T09:01:58.379 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T09:01:58.379 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T09:01:58.379 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T09:01:58.379 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:01:58.379 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:01:58.380 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:01:58.380 INFO:teuthology.orchestra.run.smithi157.stdout:Install 4 Packages 2024-09-13T09:01:58.380 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:01:58.380 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 79 M 2024-09-13T09:01:58.380 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 126 M 2024-09-13T09:01:58.380 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:01:58.745 INFO:teuthology.orchestra.run.smithi157.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 256 kB/s | 23 kB 00:00 2024-09-13T09:02:00.084 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T09:02:00.084 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T09:02:00.182 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:02:00.192 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:02:00.192 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-09-13T09:02:00.192 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:02:00.192 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:02:00.192 INFO:teuthology.orchestra.run.smithi044.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T09:02:00.192 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-09-13T09:02:00.192 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T09:02:00.192 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T09:02:00.192 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T09:02:00.192 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:02:00.192 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:02:00.193 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:02:00.193 INFO:teuthology.orchestra.run.smithi044.stdout:Install 4 Packages 2024-09-13T09:02:00.193 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:02:00.193 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 79 M 2024-09-13T09:02:00.194 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 126 M 2024-09-13T09:02:00.194 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:02:00.537 INFO:teuthology.orchestra.run.smithi044.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 263 kB/s | 23 kB 00:00 2024-09-13T09:02:00.904 INFO:teuthology.orchestra.run.smithi044.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-13T09:02:01.045 INFO:teuthology.orchestra.run.smithi157.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 6.6 MB/s | 16 MB 00:02 2024-09-13T09:02:01.304 INFO:teuthology.orchestra.run.smithi044.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 40 MB/s | 35 MB 00:00 2024-09-13T09:02:01.671 INFO:teuthology.orchestra.run.smithi044.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-13T09:02:01.672 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:02:01.672 INFO:teuthology.orchestra.run.smithi044.stdout:Total 53 MB/s | 79 MB 00:01 2024-09-13T09:02:01.690 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-13T09:02:02.382 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:02:02.805 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:02:02.805 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:02:03.161 INFO:teuthology.orchestra.run.smithi157.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.5 MB/s | 29 MB 00:04 2024-09-13T09:02:03.638 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T09:02:03.638 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T09:02:03.738 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:02:03.746 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:02:03.747 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-13T09:02:03.747 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:02:03.747 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:02:03.747 INFO:teuthology.orchestra.run.smithi129.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T09:02:03.747 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-13T09:02:03.747 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T09:02:03.747 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T09:02:03.747 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T09:02:03.747 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:02:03.747 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:02:03.747 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:02:03.747 INFO:teuthology.orchestra.run.smithi129.stdout:Install 4 Packages 2024-09-13T09:02:03.747 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:02:03.748 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 79 M 2024-09-13T09:02:03.748 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 126 M 2024-09-13T09:02:03.748 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:02:03.811 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:02:03.811 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:02:03.991 INFO:teuthology.orchestra.run.smithi129.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 166 kB/s | 23 kB 00:00 2024-09-13T09:02:04.961 INFO:teuthology.orchestra.run.smithi157.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 5.5 MB/s | 35 MB 00:06 2024-09-13T09:02:04.961 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:02:04.961 INFO:teuthology.orchestra.run.smithi157.stdout:Total 12 MB/s | 79 MB 00:06 2024-09-13T09:02:05.108 INFO:teuthology.orchestra.run.smithi129.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-09-13T09:02:05.647 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:02:05.683 INFO:teuthology.orchestra.run.smithi129.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 17 MB/s | 29 MB 00:01 2024-09-13T09:02:06.000 INFO:teuthology.orchestra.run.smithi129.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 16 MB/s | 35 MB 00:02 2024-09-13T09:02:06.001 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:02:06.001 INFO:teuthology.orchestra.run.smithi129.stdout:Total 35 MB/s | 79 MB 00:02 2024-09-13T09:02:06.039 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:02:06.039 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:02:06.367 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:02:06.682 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:02:06.989 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:02:06.990 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:02:07.022 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T09:02:07.038 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:02:07.038 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:02:07.105 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T09:02:07.999 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:02:07.999 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:02:09.430 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:02:09.902 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T09:02:10.035 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T09:02:10.153 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T09:02:10.245 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T09:02:10.443 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:02:11.148 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T09:02:11.225 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T09:02:13.058 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T09:02:13.207 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T09:02:14.044 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T09:02:14.176 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T09:02:15.154 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T09:02:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:02:18.027 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T09:02:18.116 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:02:18.988 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T09:02:19.079 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:02:20.344 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:02:22.981 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:02:23.929 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:02:59.312 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:02:59.354 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:02:59.713 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:02:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T09:02:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T09:02:59.714 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T09:03:00.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:03:00.576 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:03:00.576 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:03:00.576 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T09:03:00.576 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T09:03:00.576 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T09:03:00.576 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T09:03:00.576 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:03:00.576 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:03:00.969 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:03:00.980 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T09:03:01.009 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:03:01.081 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T09:03:01.081 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T09:03:01.082 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T09:03:01.082 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T09:03:01.082 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T09:03:01.082 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T09:03:01.082 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi044.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T09:03:01.082 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T09:03:01.141 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T09:03:01.141 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T09:03:01.141 DEBUG:teuthology.orchestra.run.smithi044:> sudo rpm -qi grub2-tools 2024-09-13T09:03:01.177 INFO:teuthology.orchestra.run.smithi044.stdout:Name : grub2-tools 2024-09-13T09:03:01.177 INFO:teuthology.orchestra.run.smithi044.stdout:Epoch : 1 2024-09-13T09:03:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:Version : 2.06 2024-09-13T09:03:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:Release : 82.el9 2024-09-13T09:03:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:Architecture: x86_64 2024-09-13T09:03:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T09:03:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:Group : Unspecified 2024-09-13T09:03:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:Size : 8274670 2024-09-13T09:03:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:License : GPLv3+ 2024-09-13T09:03:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T09:03:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T09:03:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T09:03:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T09:03:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:Packager : builder@centos.org 2024-09-13T09:03:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:Vendor : CentOS 2024-09-13T09:03:01.179 INFO:teuthology.orchestra.run.smithi044.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T09:03:01.179 INFO:teuthology.orchestra.run.smithi044.stdout:Summary : Support tools for GRUB. 2024-09-13T09:03:01.179 INFO:teuthology.orchestra.run.smithi044.stdout:Description : 2024-09-13T09:03:01.179 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:03:01.179 INFO:teuthology.orchestra.run.smithi044.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T09:03:01.179 INFO:teuthology.orchestra.run.smithi044.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T09:03:01.179 INFO:teuthology.orchestra.run.smithi044.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T09:03:01.179 INFO:teuthology.orchestra.run.smithi044.stdout:hardware devices. 2024-09-13T09:03:01.179 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:03:01.179 INFO:teuthology.orchestra.run.smithi044.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T09:03:01.180 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T09:03:01.180 INFO:teuthology.task.kernel:Updating grub on smithi044 to boot 5.14.0-505.el9.x86_64 2024-09-13T09:03:01.180 DEBUG:teuthology.orchestra.run.smithi044:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T09:03:01.332 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:03:01.332 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T09:03:01.333 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T09:03:01.333 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T09:03:01.866 INFO:teuthology.orchestra.run.smithi044.stderr:Generating grub configuration file ... 2024-09-13T09:03:01.937 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:03:01.976 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:03:02.145 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:03:02.145 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:03:02.145 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:03:02.145 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T09:03:02.145 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T09:03:02.145 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T09:03:02.146 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T09:03:02.146 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:03:02.146 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:03:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:03:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T09:03:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T09:03:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T09:03:02.498 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T09:03:02.599 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T09:03:02.599 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T09:03:02.600 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T09:03:02.600 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T09:03:02.600 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T09:03:02.600 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-13T09:03:02.600 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi157.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T09:03:02.600 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T09:03:02.659 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T09:03:02.659 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T09:03:02.659 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -qi grub2-tools 2024-09-13T09:03:02.699 INFO:teuthology.orchestra.run.smithi157.stdout:Name : grub2-tools 2024-09-13T09:03:02.699 INFO:teuthology.orchestra.run.smithi157.stdout:Epoch : 1 2024-09-13T09:03:02.699 INFO:teuthology.orchestra.run.smithi157.stdout:Version : 2.06 2024-09-13T09:03:02.699 INFO:teuthology.orchestra.run.smithi157.stdout:Release : 82.el9 2024-09-13T09:03:02.699 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: x86_64 2024-09-13T09:03:02.699 INFO:teuthology.orchestra.run.smithi157.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T09:03:02.699 INFO:teuthology.orchestra.run.smithi157.stdout:Group : Unspecified 2024-09-13T09:03:02.699 INFO:teuthology.orchestra.run.smithi157.stdout:Size : 8274670 2024-09-13T09:03:02.699 INFO:teuthology.orchestra.run.smithi157.stdout:License : GPLv3+ 2024-09-13T09:03:02.699 INFO:teuthology.orchestra.run.smithi157.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T09:03:02.699 INFO:teuthology.orchestra.run.smithi157.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T09:03:02.699 INFO:teuthology.orchestra.run.smithi157.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T09:03:02.700 INFO:teuthology.orchestra.run.smithi157.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T09:03:02.700 INFO:teuthology.orchestra.run.smithi157.stdout:Packager : builder@centos.org 2024-09-13T09:03:02.700 INFO:teuthology.orchestra.run.smithi157.stdout:Vendor : CentOS 2024-09-13T09:03:02.700 INFO:teuthology.orchestra.run.smithi157.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T09:03:02.700 INFO:teuthology.orchestra.run.smithi157.stdout:Summary : Support tools for GRUB. 2024-09-13T09:03:02.700 INFO:teuthology.orchestra.run.smithi157.stdout:Description : 2024-09-13T09:03:02.700 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:03:02.700 INFO:teuthology.orchestra.run.smithi157.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T09:03:02.700 INFO:teuthology.orchestra.run.smithi157.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T09:03:02.700 INFO:teuthology.orchestra.run.smithi157.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T09:03:02.700 INFO:teuthology.orchestra.run.smithi157.stdout:hardware devices. 2024-09-13T09:03:02.700 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:03:02.700 INFO:teuthology.orchestra.run.smithi157.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T09:03:02.701 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T09:03:02.701 INFO:teuthology.task.kernel:Updating grub on smithi157 to boot 5.14.0-505.el9.x86_64 2024-09-13T09:03:02.701 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T09:03:03.183 INFO:teuthology.orchestra.run.smithi044.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T09:03:03.204 INFO:teuthology.orchestra.run.smithi044.stderr:done 2024-09-13T09:03:03.206 DEBUG:teuthology.orchestra.run.smithi044:> mktemp 2024-09-13T09:03:03.214 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T09:03:03.214 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:03:03.215 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:03:03.215 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T09:03:03.215 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T09:03:03.215 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T09:03:03.215 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T09:03:03.215 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:03:03.215 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:03:03.221 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/tmp.6n4e4IU2SB 2024-09-13T09:03:03.221 DEBUG:teuthology.orchestra.run.smithi044:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6n4e4IU2SB 2024-09-13T09:03:03.286 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0666 /tmp/tmp.6n4e4IU2SB 2024-09-13T09:03:03.384 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2024-09-13T09:03:03.423 DEBUG:teuthology.orchestra.remote:smithi044:/tmp/tmp.6n4e4IU2SB is 6KB 2024-09-13T09:03:03.472 DEBUG:teuthology.orchestra.run.smithi044:> rm -fr /tmp/tmp.6n4e4IU2SB 2024-09-13T09:03:03.487 DEBUG:teuthology.orchestra.run.smithi044:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T09:03:03.539 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T09:03:03.549 INFO:teuthology.orchestra.run.smithi044.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T09:03:03.549 INFO:teuthology.orchestra.run.smithi044.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T09:03:03.549 INFO:teuthology.orchestra.run.smithi044.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T09:03:03.550 INFO:teuthology.orchestra.run.smithi044.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T09:03:03.551 DEBUG:teuthology.orchestra.run.smithi044:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T09:03:03.667 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T09:03:03.668 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T09:03:03.668 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T09:03:03.668 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T09:03:03.668 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T09:03:03.668 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T09:03:03.668 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi129.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T09:03:03.668 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T09:03:03.707 DEBUG:teuthology.orchestra.run.smithi044:> sudo shutdown -r now 2024-09-13T09:03:03.727 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T09:03:03.727 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T09:03:03.728 DEBUG:teuthology.orchestra.run.smithi129:> sudo rpm -qi grub2-tools 2024-09-13T09:03:03.767 INFO:teuthology.orchestra.run.smithi129.stdout:Name : grub2-tools 2024-09-13T09:03:03.768 INFO:teuthology.orchestra.run.smithi129.stdout:Epoch : 1 2024-09-13T09:03:03.768 INFO:teuthology.orchestra.run.smithi129.stdout:Version : 2.06 2024-09-13T09:03:03.768 INFO:teuthology.orchestra.run.smithi129.stdout:Release : 82.el9 2024-09-13T09:03:03.768 INFO:teuthology.orchestra.run.smithi129.stdout:Architecture: x86_64 2024-09-13T09:03:03.768 INFO:teuthology.orchestra.run.smithi129.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T09:03:03.768 INFO:teuthology.orchestra.run.smithi129.stdout:Group : Unspecified 2024-09-13T09:03:03.768 INFO:teuthology.orchestra.run.smithi129.stdout:Size : 8274670 2024-09-13T09:03:03.768 INFO:teuthology.orchestra.run.smithi129.stdout:License : GPLv3+ 2024-09-13T09:03:03.768 INFO:teuthology.orchestra.run.smithi129.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T09:03:03.768 INFO:teuthology.orchestra.run.smithi129.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T09:03:03.768 INFO:teuthology.orchestra.run.smithi129.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T09:03:03.769 INFO:teuthology.orchestra.run.smithi129.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T09:03:03.769 INFO:teuthology.orchestra.run.smithi129.stdout:Packager : builder@centos.org 2024-09-13T09:03:03.769 INFO:teuthology.orchestra.run.smithi129.stdout:Vendor : CentOS 2024-09-13T09:03:03.769 INFO:teuthology.orchestra.run.smithi129.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T09:03:03.769 INFO:teuthology.orchestra.run.smithi129.stdout:Summary : Support tools for GRUB. 2024-09-13T09:03:03.769 INFO:teuthology.orchestra.run.smithi129.stdout:Description : 2024-09-13T09:03:03.769 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:03:03.769 INFO:teuthology.orchestra.run.smithi129.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T09:03:03.769 INFO:teuthology.orchestra.run.smithi129.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T09:03:03.769 INFO:teuthology.orchestra.run.smithi129.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T09:03:03.769 INFO:teuthology.orchestra.run.smithi129.stdout:hardware devices. 2024-09-13T09:03:03.769 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:03:03.770 INFO:teuthology.orchestra.run.smithi129.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T09:03:03.770 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T09:03:03.770 INFO:teuthology.task.kernel:Updating grub on smithi129 to boot 5.14.0-505.el9.x86_64 2024-09-13T09:03:03.771 DEBUG:teuthology.orchestra.run.smithi129:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T09:03:04.419 INFO:teuthology.orchestra.run.smithi129.stderr:Generating grub configuration file ... 2024-09-13T09:03:04.722 INFO:teuthology.orchestra.run.smithi157.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T09:03:04.753 INFO:teuthology.orchestra.run.smithi157.stderr:done 2024-09-13T09:03:04.755 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-09-13T09:03:04.770 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.HgNak39gs4 2024-09-13T09:03:04.770 DEBUG:teuthology.orchestra.run.smithi157:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HgNak39gs4 2024-09-13T09:03:04.837 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.HgNak39gs4 2024-09-13T09:03:04.976 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.HgNak39gs4 is 6KB 2024-09-13T09:03:05.024 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.HgNak39gs4 2024-09-13T09:03:05.039 DEBUG:teuthology.orchestra.run.smithi157:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T09:03:05.108 INFO:teuthology.orchestra.run.smithi157.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T09:03:05.109 INFO:teuthology.orchestra.run.smithi157.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T09:03:05.109 INFO:teuthology.orchestra.run.smithi157.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T09:03:05.109 INFO:teuthology.orchestra.run.smithi157.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T09:03:05.110 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T09:03:05.275 DEBUG:teuthology.orchestra.run.smithi157:> sudo shutdown -r now 2024-09-13T09:03:05.742 INFO:teuthology.orchestra.run.smithi129.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T09:03:05.768 INFO:teuthology.orchestra.run.smithi129.stderr:done 2024-09-13T09:03:05.770 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-09-13T09:03:05.786 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.H4vDuNpKF1 2024-09-13T09:03:05.786 DEBUG:teuthology.orchestra.run.smithi129:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.H4vDuNpKF1 2024-09-13T09:03:05.855 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.H4vDuNpKF1 2024-09-13T09:03:05.998 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.H4vDuNpKF1 is 6KB 2024-09-13T09:03:06.046 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.H4vDuNpKF1 2024-09-13T09:03:06.061 DEBUG:teuthology.orchestra.run.smithi129:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T09:03:06.124 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T09:03:06.124 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T09:03:06.124 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T09:03:06.124 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T09:03:06.125 DEBUG:teuthology.orchestra.run.smithi129:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T09:03:06.279 DEBUG:teuthology.orchestra.run.smithi129:> sudo shutdown -r now 2024-09-13T09:03:33.733 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T09:03:33.734 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T09:03:33.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:03:35.279 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-13T09:03:35.280 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-13T09:03:35.280 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:03:36.282 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T09:03:36.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-13T09:03:36.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:03:52.129 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-09-13T09:03:53.665 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-13T09:03:54.685 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-13T09:04:01.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T09:04:01.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:04:02.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-13T09:04:02.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:04:03.687 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-13T09:04:03.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:04:05.725 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-13T09:04:06.749 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-13T09:04:17.735 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-13T09:04:17.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:04:18.751 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-13T09:04:18.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:04:20.797 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-13T09:04:35.812 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-13T09:04:35.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:04:37.181 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-13T09:04:52.193 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-13T09:04:52.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:04:52.501 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-09-13T09:04:52.920 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-13T09:04:52.920 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-13T09:04:52.920 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-09-13T09:04:52.934 INFO:teuthology.orchestra.run.smithi129.stdout:5.14.0-505.el9.x86_64 2024-09-13T09:04:52.934 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T09:04:52.934 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T09:04:52.935 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T09:04:53.935 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T09:04:53.936 DEBUG:teuthology.orchestra.run.smithi129:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T09:04:54.106 INFO:teuthology.orchestra.run.smithi129.stdout:ttyS1 2024-09-13T09:04:54.127 DEBUG:teuthology.parallel:result is None 2024-09-13T09:04:54.333 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-13T09:05:01.141 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T09:05:12.334 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-13T09:05:12.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:05:12.622 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-09-13T09:05:13.004 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-13T09:05:13.005 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-13T09:05:13.005 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-09-13T09:05:13.021 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-505.el9.x86_64 2024-09-13T09:05:13.021 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T09:05:13.021 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T09:05:13.022 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T09:05:13.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T09:05:13.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:05:13.413 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-09-13T09:05:13.745 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T09:05:13.746 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-13T09:05:13.746 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-09-13T09:05:13.762 INFO:teuthology.orchestra.run.smithi044.stdout:5.14.0-505.el9.x86_64 2024-09-13T09:05:13.762 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T09:05:13.762 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T09:05:13.762 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T09:05:14.023 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-09-13T09:05:14.023 DEBUG:teuthology.orchestra.run.smithi157:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T09:05:14.098 INFO:teuthology.orchestra.run.smithi157.stdout:ttyS1 2024-09-13T09:05:14.123 DEBUG:teuthology.parallel:result is None 2024-09-13T09:05:14.763 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T09:05:14.764 DEBUG:teuthology.orchestra.run.smithi044:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T09:05:14.836 INFO:teuthology.orchestra.run.smithi044.stdout:ttyS1 2024-09-13T09:05:14.865 DEBUG:teuthology.parallel:result is None 2024-09-13T09:05:14.865 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T09:05:14.874 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T09:05:14.874 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T09:05:14.877 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T09:05:14.879 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T09:05:14.898 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T09:05:14.922 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T09:05:14.932 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T09:05:14.933 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T09:05:14.936 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T09:05:14.939 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T09:05:15.006 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T09:05:15.012 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T09:05:15.013 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T09:05:15.015 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T09:05:15.018 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T09:05:15.048 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:05:15.049 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:05:15.053 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:05:15.060 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:05:15.061 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:05:15.065 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:05:15.067 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T09:05:15.073 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T09:05:15.073 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T09:05:15.105 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T09:05:15.108 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T09:05:15.135 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T09:05:15.145 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T09:05:15.145 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T09:05:15.175 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T09:05:15.178 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T09:05:15.195 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T09:05:15.271 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T09:05:15.328 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T09:05:15.406 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T09:05:15.458 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:05:15.458 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T09:05:15.526 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T09:05:15.565 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T09:05:15.621 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T09:05:15.702 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T09:05:15.750 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:05:15.750 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T09:05:15.817 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T09:05:15.860 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T09:05:15.915 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T09:05:15.993 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T09:05:16.044 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T09:05:16.044 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T09:05:16.111 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2024-09-13T09:05:16.114 DEBUG:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2024-09-13T09:05:16.117 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-09-13T09:05:16.163 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T09:05:16.165 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T09:05:16.199 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T09:05:16.646 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T09:05:16.654 INFO:teuthology.task.internal:Starting timer... 2024-09-13T09:05:16.654 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T09:05:16.665 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T09:05:16.674 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-13T09:05:16.674 DEBUG:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2024-09-13T09:05:16.746 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: 2024-09-13T09:05:16.748 DEBUG:teuthology.orchestra.run.smithi129:> sudo service auditd rotate 2024-09-13T09:05:16.807 INFO:teuthology.orchestra.run.smithi129.stdout:Rotating logs: 2024-09-13T09:05:16.808 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-09-13T09:05:16.876 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-09-13T09:05:16.878 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T09:05:16.878 DEBUG:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2024-09-13T09:05:16.906 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2024-09-13T09:05:16.906 DEBUG:teuthology.orchestra.run.smithi129:> /usr/sbin/getenforce 2024-09-13T09:05:16.940 INFO:teuthology.orchestra.run.smithi129.stdout:Permissive 2024-09-13T09:05:16.940 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-09-13T09:05:16.964 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-09-13T09:05:16.964 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi044.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi129.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2024-09-13T09:05:16.965 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T09:05:16.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:05:16.995 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T09:05:17.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:05:17.024 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T09:05:17.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:05:17.051 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T09:05:17.051 DEBUG:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2024-09-13T09:05:17.087 DEBUG:teuthology.orchestra.run.smithi129:> sudo /usr/sbin/setenforce permissive 2024-09-13T09:05:17.118 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-09-13T09:05:17.151 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T09:05:17.160 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-13T09:05:17.164 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-13T09:05:17.272 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T09:05:17.282 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T09:05:17.283 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi044.front.sepia.ceph.com,smithi129.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T09:09:02.243 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi044.front.sepia.ceph.com'), Remote(name='ubuntu@smithi129.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-09-13T09:09:02.245 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T09:09:02.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:09:02.327 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-09-13T09:09:02.409 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-09-13T09:09:02.409 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-13T09:09:02.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:09:02.484 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-09-13T09:09:02.564 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-13T09:09:02.565 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-13T09:09:02.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:09:02.639 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-09-13T09:09:02.716 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-13T09:09:02.717 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T09:09:02.727 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T09:09:02.728 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T09:09:02.728 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:09:02.731 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T09:09:02.731 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:09:02.734 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T09:09:02.734 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:09:02.762 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T09:09:02.764 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T09:09:02.777 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T09:09:02.781 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T09:09:02.786 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T09:09:02.800 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T09:09:02.802 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: ntpd: command not found 2024-09-13T09:09:02.809 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2024-09-13T09:09:02.813 INFO:teuthology.orchestra.run.smithi129.stdout:506 Cannot talk to daemon 2024-09-13T09:09:02.821 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2024-09-13T09:09:02.824 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-09-13T09:09:02.827 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T09:09:02.835 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-09-13T09:09:02.837 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T09:09:02.841 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T09:09:02.850 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T09:09:02.853 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T09:09:02.864 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T09:09:02.907 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-09-13T09:09:02.910 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T09:09:02.910 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-09-13T09:09:02.910 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T09:09:02.911 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T09:09:02.911 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T09:09:02.911 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T09:09:02.919 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-09-13T09:09:02.921 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-09-13T09:09:02.923 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T09:09:02.923 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-09-13T09:09:02.923 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T09:09:02.923 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T09:09:02.923 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T09:09:02.923 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T09:09:02.925 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T09:09:02.925 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-09-13T09:09:02.925 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T09:09:02.925 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T09:09:02.925 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T09:09:02.925 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T09:09:02.926 INFO:teuthology.run_tasks:Running task install... 2024-09-13T09:09:02.933 DEBUG:teuthology.task.install:project ceph 2024-09-13T09:09:02.933 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}} 2024-09-13T09:09:02.933 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'} 2024-09-13T09:09:02.933 INFO:teuthology.task.install:Using flavor: default 2024-09-13T09:09:02.939 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T09:09:02.939 INFO:teuthology.task.install:extra packages: ['rbd-nbd'] 2024-09-13T09:09:02.940 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412', 'tag': None, 'wait_for_package': False} 2024-09-13T09:09:02.940 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:09:02.943 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412', 'tag': None, 'wait_for_package': False} 2024-09-13T09:09:02.943 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:09:02.944 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412', 'tag': None, 'wait_for_package': False} 2024-09-13T09:09:02.944 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:09:03.096 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/ 2024-09-13T09:09:03.096 INFO:teuthology.task.install.rpm:Package version is 18.2.4-820.g48a7de7e 2024-09-13T09:09:03.100 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/ 2024-09-13T09:09:03.100 INFO:teuthology.task.install.rpm:Package version is 18.2.4-820.g48a7de7e 2024-09-13T09:09:03.103 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/ 2024-09-13T09:09:03.103 INFO:teuthology.task.install.rpm:Package version is 18.2.4-820.g48a7de7e 2024-09-13T09:09:03.207 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T09:09:03.207 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:09:03.208 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T09:09:03.220 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T09:09:03.220 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T09:09:03.220 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T09:09:03.224 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T09:09:03.224 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:09:03.224 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T09:09:03.237 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-13T09:09:03.237 DEBUG:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T09:09:03.247 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-13T09:09:03.247 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T09:09:03.251 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-13T09:09:03.251 DEBUG:teuthology.orchestra.run.smithi129:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T09:09:03.305 DEBUG:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T09:09:03.314 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T09:09:03.318 DEBUG:teuthology.orchestra.run.smithi129:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T09:09:03.384 DEBUG:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T09:09:03.391 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T09:09:03.392 DEBUG:teuthology.orchestra.run.smithi129:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T09:09:03.450 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2024-09-13T09:09:03.452 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum clean all 2024-09-13T09:09:03.455 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2024-09-13T09:09:03.456 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2024-09-13T09:09:03.457 INFO:teuthology.orchestra.run.smithi129.stdout:check_obsoletes = 1 2024-09-13T09:09:03.458 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum clean all 2024-09-13T09:09:03.685 INFO:teuthology.orchestra.run.smithi044.stdout:45 files removed 2024-09-13T09:09:03.691 INFO:teuthology.orchestra.run.smithi129.stdout:45 files removed 2024-09-13T09:09:03.695 INFO:teuthology.orchestra.run.smithi157.stdout:45 files removed 2024-09-13T09:09:03.713 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw 2024-09-13T09:09:03.718 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-radosgw 2024-09-13T09:09:03.724 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw 2024-09-13T09:09:04.284 INFO:teuthology.orchestra.run.smithi129.stdout:ceph packages for x86_64 274 kB/s | 77 kB 00:00 2024-09-13T09:09:04.285 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 280 kB/s | 77 kB 00:00 2024-09-13T09:09:04.300 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 267 kB/s | 77 kB 00:00 2024-09-13T09:09:04.568 INFO:teuthology.orchestra.run.smithi129.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-13T09:09:04.569 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-13T09:09:04.592 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 46 kB/s | 11 kB 00:00 2024-09-13T09:09:04.844 INFO:teuthology.orchestra.run.smithi129.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-13T09:09:04.844 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-13T09:09:04.876 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 7.6 kB/s | 1.9 kB 00:00 2024-09-13T09:09:05.393 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-13T09:09:05.586 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-13T09:09:05.827 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.2 MB 00:00 2024-09-13T09:09:07.502 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-13T09:09:07.621 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-13T09:09:07.628 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-13T09:09:13.172 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-13T09:09:13.678 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.5 MB 00:02 2024-09-13T09:09:14.672 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 19 kB 00:00 2024-09-13T09:09:14.729 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.5 MB 00:03 2024-09-13T09:09:15.312 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 19 kB 00:00 2024-09-13T09:09:15.451 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-13T09:09:16.175 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-13T09:09:16.538 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-13T09:09:17.326 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-13T09:09:20.840 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-13T09:09:21.881 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-13T09:09:22.723 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-13T09:09:23.241 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:09:23.270 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:09:23.270 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-13T09:09:23.270 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:09:23.271 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:09:23.271 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 7.7 M 2024-09-13T09:09:23.271 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrading: 2024-09-13T09:09:23.271 INFO:teuthology.orchestra.run.smithi129.stdout: librados2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.3 M 2024-09-13T09:09:23.271 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.0 M 2024-09-13T09:09:23.271 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-13T09:09:23.271 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T09:09:23.271 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 5.1 M 2024-09-13T09:09:23.271 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 18 M 2024-09-13T09:09:23.271 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 25 k 2024-09-13T09:09:23.271 INFO:teuthology.orchestra.run.smithi129.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T09:09:23.271 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T09:09:23.271 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T09:09:23.272 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T09:09:23.272 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 711 k 2024-09-13T09:09:23.272 INFO:teuthology.orchestra.run.smithi129.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T09:09:23.272 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T09:09:23.272 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T09:09:23.272 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 478 k 2024-09-13T09:09:23.272 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T09:09:23.272 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 4.5 M 2024-09-13T09:09:23.272 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T09:09:23.272 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T09:09:23.272 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T09:09:23.272 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 45 k 2024-09-13T09:09:23.273 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 129 k 2024-09-13T09:09:23.273 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 162 k 2024-09-13T09:09:23.273 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T09:09:23.273 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 321 k 2024-09-13T09:09:23.273 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 299 k 2024-09-13T09:09:23.273 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 99 k 2024-09-13T09:09:23.273 INFO:teuthology.orchestra.run.smithi129.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T09:09:23.273 INFO:teuthology.orchestra.run.smithi129.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T09:09:23.273 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:09:23.273 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:09:23.273 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:09:23.273 INFO:teuthology.orchestra.run.smithi129.stdout:Install 28 Packages 2024-09-13T09:09:23.274 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrade 2 Packages 2024-09-13T09:09:23.274 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:09:23.274 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 53 M 2024-09-13T09:09:23.274 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:09:23.950 INFO:teuthology.orchestra.run.smithi129.stdout:(1/30): ceph-base-18.2.4-820.g48a7de7e.el9.x86_ 9.7 MB/s | 5.1 MB 00:00 2024-09-13T09:09:24.008 INFO:teuthology.orchestra.run.smithi129.stdout:(2/30): ceph-selinux-18.2.4-820.g48a7de7e.el9.x 433 kB/s | 25 kB 00:00 2024-09-13T09:09:24.100 INFO:teuthology.orchestra.run.smithi129.stdout:(3/30): ceph-radosgw-18.2.4-820.g48a7de7e.el9.x 11 MB/s | 7.7 MB 00:00 2024-09-13T09:09:24.134 INFO:teuthology.orchestra.run.smithi129.stdout:(4/30): libcephfs2-18.2.4-820.g48a7de7e.el9.x86 5.6 MB/s | 711 kB 00:00 2024-09-13T09:09:24.159 INFO:teuthology.orchestra.run.smithi129.stdout:(5/30): libradosstriper1-18.2.4-820.g48a7de7e.e 8.0 MB/s | 478 kB 00:00 2024-09-13T09:09:24.201 INFO:teuthology.orchestra.run.smithi129.stdout:(6/30): python3-ceph-argparse-18.2.4-820.g48a7d 1.1 MB/s | 45 kB 00:00 2024-09-13T09:09:24.276 INFO:teuthology.orchestra.run.smithi129.stdout:(7/30): python3-ceph-common-18.2.4-820.g48a7de7 1.7 MB/s | 129 kB 00:00 2024-09-13T09:09:24.360 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:09:24.368 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:09:24.368 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-09-13T09:09:24.368 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:09:24.368 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:09:24.368 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 7.7 M 2024-09-13T09:09:24.368 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2024-09-13T09:09:24.369 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.3 M 2024-09-13T09:09:24.369 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.0 M 2024-09-13T09:09:24.369 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-09-13T09:09:24.369 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T09:09:24.369 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 5.1 M 2024-09-13T09:09:24.369 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 18 M 2024-09-13T09:09:24.369 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 25 k 2024-09-13T09:09:24.369 INFO:teuthology.orchestra.run.smithi044.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T09:09:24.369 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T09:09:24.369 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T09:09:24.369 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T09:09:24.369 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 711 k 2024-09-13T09:09:24.369 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T09:09:24.370 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T09:09:24.370 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T09:09:24.370 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 478 k 2024-09-13T09:09:24.370 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T09:09:24.370 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 4.5 M 2024-09-13T09:09:24.370 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T09:09:24.370 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T09:09:24.370 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T09:09:24.370 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 45 k 2024-09-13T09:09:24.370 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 129 k 2024-09-13T09:09:24.370 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 162 k 2024-09-13T09:09:24.370 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T09:09:24.370 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 321 k 2024-09-13T09:09:24.371 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 299 k 2024-09-13T09:09:24.371 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 99 k 2024-09-13T09:09:24.371 INFO:teuthology.orchestra.run.smithi044.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T09:09:24.371 INFO:teuthology.orchestra.run.smithi044.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T09:09:24.371 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:09:24.371 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:09:24.371 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:09:24.371 INFO:teuthology.orchestra.run.smithi044.stdout:Install 28 Packages 2024-09-13T09:09:24.371 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 2 Packages 2024-09-13T09:09:24.371 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:09:24.372 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 53 M 2024-09-13T09:09:24.372 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:09:24.385 INFO:teuthology.orchestra.run.smithi129.stdout:(8/30): librgw2-18.2.4-820.g48a7de7e.el9.x86_64 18 MB/s | 4.5 MB 00:00 2024-09-13T09:09:24.410 INFO:teuthology.orchestra.run.smithi129.stdout:(9/30): python3-cephfs-18.2.4-820.g48a7de7e.el9 1.2 MB/s | 162 kB 00:00 2024-09-13T09:09:24.585 INFO:teuthology.orchestra.run.smithi129.stdout:(10/30): ceph-common-18.2.4-820.g48a7de7e.el9.x 16 MB/s | 18 MB 00:01 2024-09-13T09:09:24.610 INFO:teuthology.orchestra.run.smithi129.stdout:(11/30): python3-rados-18.2.4-820.g48a7de7e.el9 1.4 MB/s | 321 kB 00:00 2024-09-13T09:09:24.636 INFO:teuthology.orchestra.run.smithi129.stdout:(12/30): python3-rbd-18.2.4-820.g48a7de7e.el9.x 1.3 MB/s | 299 kB 00:00 2024-09-13T09:09:24.661 INFO:teuthology.orchestra.run.smithi129.stdout:(13/30): python3-rgw-18.2.4-820.g48a7de7e.el9.x 1.3 MB/s | 99 kB 00:00 2024-09-13T09:09:24.763 INFO:teuthology.orchestra.run.smithi129.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-13T09:09:24.789 INFO:teuthology.orchestra.run.smithi129.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 355 kB/s | 45 kB 00:00 2024-09-13T09:09:24.806 INFO:teuthology.orchestra.run.smithi129.stdout:(16/30): boost-program-options-1.75.0-8.el9.x86 548 kB/s | 107 kB 00:00 2024-09-13T09:09:24.848 INFO:teuthology.orchestra.run.smithi129.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-13T09:09:24.881 INFO:teuthology.orchestra.run.smithi129.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 561 kB/s | 42 kB 00:00 2024-09-13T09:09:24.907 INFO:teuthology.orchestra.run.smithi129.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-13T09:09:24.949 INFO:teuthology.orchestra.run.smithi129.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 1.1 MB/s | 111 kB 00:00 2024-09-13T09:09:24.974 INFO:teuthology.orchestra.run.smithi129.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-13T09:09:24.999 INFO:teuthology.orchestra.run.smithi129.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-13T09:09:25.024 INFO:teuthology.orchestra.run.smithi129.stdout:(23/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.1 MB/s | 308 kB 00:00 2024-09-13T09:09:25.049 INFO:teuthology.orchestra.run.smithi129.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-13T09:09:25.083 INFO:teuthology.orchestra.run.smithi129.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-13T09:09:25.108 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:09:25.117 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:09:25.117 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-09-13T09:09:25.118 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:09:25.118 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:09:25.118 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 7.7 M 2024-09-13T09:09:25.118 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-09-13T09:09:25.118 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.3 M 2024-09-13T09:09:25.118 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.0 M 2024-09-13T09:09:25.118 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-09-13T09:09:25.118 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T09:09:25.118 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 5.1 M 2024-09-13T09:09:25.118 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 18 M 2024-09-13T09:09:25.118 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 25 k 2024-09-13T09:09:25.118 INFO:teuthology.orchestra.run.smithi157.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T09:09:25.118 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T09:09:25.118 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T09:09:25.119 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T09:09:25.119 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 711 k 2024-09-13T09:09:25.119 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T09:09:25.119 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T09:09:25.119 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T09:09:25.119 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 478 k 2024-09-13T09:09:25.119 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T09:09:25.119 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 4.5 M 2024-09-13T09:09:25.119 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T09:09:25.119 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T09:09:25.119 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T09:09:25.119 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 45 k 2024-09-13T09:09:25.119 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 129 k 2024-09-13T09:09:25.119 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 162 k 2024-09-13T09:09:25.119 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T09:09:25.120 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 321 k 2024-09-13T09:09:25.120 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 299 k 2024-09-13T09:09:25.120 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 99 k 2024-09-13T09:09:25.120 INFO:teuthology.orchestra.run.smithi157.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T09:09:25.120 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T09:09:25.120 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:09:25.120 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:09:25.120 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:09:25.120 INFO:teuthology.orchestra.run.smithi157.stdout:Install 28 Packages 2024-09-13T09:09:25.120 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 2 Packages 2024-09-13T09:09:25.120 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:09:25.121 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 53 M 2024-09-13T09:09:25.121 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:09:25.125 INFO:teuthology.orchestra.run.smithi129.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-13T09:09:25.184 INFO:teuthology.orchestra.run.smithi129.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-13T09:09:25.199 INFO:teuthology.orchestra.run.smithi044.stdout:(1/30): ceph-base-18.2.4-820.g48a7de7e.el9.x86_ 9.7 MB/s | 5.1 MB 00:00 2024-09-13T09:09:25.268 INFO:teuthology.orchestra.run.smithi129.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-13T09:09:25.274 INFO:teuthology.orchestra.run.smithi044.stdout:(2/30): ceph-selinux-18.2.4-820.g48a7de7e.el9.x 335 kB/s | 25 kB 00:00 2024-09-13T09:09:25.374 INFO:teuthology.orchestra.run.smithi044.stdout:(3/30): ceph-radosgw-18.2.4-820.g48a7de7e.el9.x 11 MB/s | 7.7 MB 00:00 2024-09-13T09:09:25.408 INFO:teuthology.orchestra.run.smithi044.stdout:(4/30): libcephfs2-18.2.4-820.g48a7de7e.el9.x86 5.2 MB/s | 711 kB 00:00 2024-09-13T09:09:25.418 INFO:teuthology.orchestra.run.smithi129.stdout:(29/30): librados2-18.2.4-820.g48a7de7e.el9.x86 11 MB/s | 3.3 MB 00:00 2024-09-13T09:09:25.458 INFO:teuthology.orchestra.run.smithi044.stdout:(5/30): libradosstriper1-18.2.4-820.g48a7de7e.e 5.6 MB/s | 478 kB 00:00 2024-09-13T09:09:25.476 INFO:teuthology.orchestra.run.smithi129.stdout:(30/30): librbd1-18.2.4-820.g48a7de7e.el9.x86_6 10 MB/s | 3.0 MB 00:00 2024-09-13T09:09:25.480 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:09:25.481 INFO:teuthology.orchestra.run.smithi129.stdout:Total 24 MB/s | 53 MB 00:02 2024-09-13T09:09:25.533 INFO:teuthology.orchestra.run.smithi044.stdout:(6/30): python3-ceph-argparse-18.2.4-820.g48a7d 601 kB/s | 45 kB 00:00 2024-09-13T09:09:25.548 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:09:25.582 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:09:25.582 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:09:25.617 INFO:teuthology.orchestra.run.smithi044.stdout:(7/30): python3-ceph-common-18.2.4-820.g48a7de7 1.5 MB/s | 129 kB 00:00 2024-09-13T09:09:25.817 INFO:teuthology.orchestra.run.smithi044.stdout:(8/30): ceph-common-18.2.4-820.g48a7de7e.el9.x8 16 MB/s | 18 MB 00:01 2024-09-13T09:09:25.842 INFO:teuthology.orchestra.run.smithi044.stdout:(9/30): python3-cephfs-18.2.4-820.g48a7de7e.el9 718 kB/s | 162 kB 00:00 2024-09-13T09:09:25.909 INFO:teuthology.orchestra.run.smithi044.stdout:(10/30): librgw2-18.2.4-820.g48a7de7e.el9.x86_6 8.9 MB/s | 4.5 MB 00:00 2024-09-13T09:09:25.943 INFO:teuthology.orchestra.run.smithi044.stdout:(11/30): python3-rados-18.2.4-820.g48a7de7e.el9 2.5 MB/s | 321 kB 00:00 2024-09-13T09:09:25.968 INFO:teuthology.orchestra.run.smithi044.stdout:(12/30): python3-rbd-18.2.4-820.g48a7de7e.el9.x 2.3 MB/s | 299 kB 00:00 2024-09-13T09:09:25.993 INFO:teuthology.orchestra.run.smithi044.stdout:(13/30): python3-rgw-18.2.4-820.g48a7de7e.el9.x 1.2 MB/s | 99 kB 00:00 2024-09-13T09:09:26.191 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:09:26.192 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:09:26.333 INFO:teuthology.orchestra.run.smithi157.stdout:(1/30): ceph-base-18.2.4-820.g48a7de7e.el9.x86_ 4.8 MB/s | 5.1 MB 00:01 2024-09-13T09:09:26.383 INFO:teuthology.orchestra.run.smithi157.stdout:(2/30): ceph-selinux-18.2.4-820.g48a7de7e.el9.x 506 kB/s | 25 kB 00:00 2024-09-13T09:09:26.475 INFO:teuthology.orchestra.run.smithi157.stdout:(3/30): libcephfs2-18.2.4-820.g48a7de7e.el9.x86 7.6 MB/s | 711 kB 00:00 2024-09-13T09:09:26.569 INFO:teuthology.orchestra.run.smithi044.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 79 kB/s | 45 kB 00:00 2024-09-13T09:09:26.592 INFO:teuthology.orchestra.run.smithi157.stdout:(4/30): ceph-radosgw-18.2.4-820.g48a7de7e.el9.x 5.9 MB/s | 7.7 MB 00:01 2024-09-13T09:09:26.617 INFO:teuthology.orchestra.run.smithi157.stdout:(5/30): libradosstriper1-18.2.4-820.g48a7de7e.e 3.3 MB/s | 478 kB 00:00 2024-09-13T09:09:26.627 INFO:teuthology.orchestra.run.smithi044.stdout:(15/30): boost-program-options-1.75.0-8.el9.x86 156 kB/s | 107 kB 00:00 2024-09-13T09:09:26.667 INFO:teuthology.orchestra.run.smithi157.stdout:(6/30): python3-ceph-argparse-18.2.4-820.g48a7d 899 kB/s | 45 kB 00:00 2024-09-13T09:09:26.711 INFO:teuthology.orchestra.run.smithi044.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 223 kB/s | 160 kB 00:00 2024-09-13T09:09:26.725 INFO:teuthology.orchestra.run.smithi157.stdout:(7/30): python3-ceph-common-18.2.4-820.g48a7de7 2.2 MB/s | 129 kB 00:00 2024-09-13T09:09:26.784 INFO:teuthology.orchestra.run.smithi157.stdout:(8/30): python3-cephfs-18.2.4-820.g48a7de7e.el9 2.7 MB/s | 162 kB 00:00 2024-09-13T09:09:26.834 INFO:teuthology.orchestra.run.smithi157.stdout:(9/30): python3-rados-18.2.4-820.g48a7de7e.el9. 6.3 MB/s | 321 kB 00:00 2024-09-13T09:09:26.935 INFO:teuthology.orchestra.run.smithi157.stdout:(10/30): librgw2-18.2.4-820.g48a7de7e.el9.x86_6 13 MB/s | 4.5 MB 00:00 2024-09-13T09:09:26.960 INFO:teuthology.orchestra.run.smithi157.stdout:(11/30): python3-rbd-18.2.4-820.g48a7de7e.el9.x 2.3 MB/s | 299 kB 00:00 2024-09-13T09:09:26.985 INFO:teuthology.orchestra.run.smithi157.stdout:(12/30): python3-rgw-18.2.4-820.g48a7de7e.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-13T09:09:27.177 INFO:teuthology.orchestra.run.smithi157.stdout:(13/30): ceph-common-18.2.4-820.g48a7de7e.el9.x 9.6 MB/s | 18 MB 00:01 2024-09-13T09:09:27.287 INFO:teuthology.orchestra.run.smithi044.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 73 kB/s | 42 kB 00:00 2024-09-13T09:09:27.328 INFO:teuthology.orchestra.run.smithi044.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 2.6 MB/s | 111 kB 00:00 2024-09-13T09:09:27.362 INFO:teuthology.orchestra.run.smithi044.stdout:(19/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 9.0 MB/s | 308 kB 00:00 2024-09-13T09:09:27.412 INFO:teuthology.orchestra.run.smithi044.stdout:(20/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 373 kB/s | 292 kB 00:00 2024-09-13T09:09:27.454 INFO:teuthology.orchestra.run.smithi044.stdout:(21/30): librdkafka-1.6.1-102.el9.x86_64.rpm 748 kB/s | 662 kB 00:00 2024-09-13T09:09:27.488 INFO:teuthology.orchestra.run.smithi044.stdout:(22/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-09-13T09:09:27.530 INFO:teuthology.orchestra.run.smithi044.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 665 kB/s | 49 kB 00:00 2024-09-13T09:09:27.552 INFO:teuthology.orchestra.run.smithi157.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 180 kB/s | 107 kB 00:00 2024-09-13T09:09:27.555 INFO:teuthology.orchestra.run.smithi044.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-13T09:09:27.567 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:09:27.578 INFO:teuthology.orchestra.run.smithi157.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 113 kB/s | 45 kB 00:00 2024-09-13T09:09:27.588 INFO:teuthology.orchestra.run.smithi044.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-13T09:09:27.636 INFO:teuthology.orchestra.run.smithi157.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 246 kB/s | 160 kB 00:00 2024-09-13T09:09:27.639 INFO:teuthology.orchestra.run.smithi044.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-13T09:09:27.677 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-13T09:09:27.689 INFO:teuthology.orchestra.run.smithi044.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-13T09:09:27.707 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-13T09:09:27.753 INFO:teuthology.orchestra.run.smithi157.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 362 kB/s | 42 kB 00:00 2024-09-13T09:09:27.756 INFO:teuthology.orchestra.run.smithi044.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-13T09:09:27.764 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-13T09:09:27.795 INFO:teuthology.orchestra.run.smithi157.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 2.6 MB/s | 111 kB 00:00 2024-09-13T09:09:27.829 INFO:teuthology.orchestra.run.smithi157.stdout:(19/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 9.1 MB/s | 308 kB 00:00 2024-09-13T09:09:27.870 INFO:teuthology.orchestra.run.smithi157.stdout:(20/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.0 MB/s | 292 kB 00:00 2024-09-13T09:09:27.912 INFO:teuthology.orchestra.run.smithi157.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 623 kB/s | 26 kB 00:00 2024-09-13T09:09:27.938 INFO:teuthology.orchestra.run.smithi157.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-13T09:09:27.940 INFO:teuthology.orchestra.run.smithi044.stdout:(29/30): librados2-18.2.4-820.g48a7de7e.el9.x86 11 MB/s | 3.3 MB 00:00 2024-09-13T09:09:27.954 INFO:teuthology.orchestra.run.smithi157.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 4.0 MB/s | 67 kB 00:00 2024-09-13T09:09:27.990 INFO:teuthology.orchestra.run.smithi044.stdout:(30/30): librbd1-18.2.4-820.g48a7de7e.el9.x86_6 10 MB/s | 3.0 MB 00:00 2024-09-13T09:09:27.996 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:09:27.996 INFO:teuthology.orchestra.run.smithi044.stdout:Total 15 MB/s | 53 MB 00:03 2024-09-13T09:09:28.030 INFO:teuthology.orchestra.run.smithi157.stdout:(24/30): libarrow-9.0.0-11.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-09-13T09:09:28.064 INFO:teuthology.orchestra.run.smithi157.stdout:(25/30): librdkafka-1.6.1-102.el9.x86_64.rpm 1.3 MB/s | 662 kB 00:00 2024-09-13T09:09:28.076 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:09:28.097 INFO:teuthology.orchestra.run.smithi157.stdout:(26/30): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-13T09:09:28.117 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:09:28.117 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:09:28.133 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-13T09:09:28.139 INFO:teuthology.orchestra.run.smithi157.stdout:(27/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 839 kB 00:00 2024-09-13T09:09:28.180 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T09:09:28.189 INFO:teuthology.orchestra.run.smithi157.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-13T09:09:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T09:09:28.356 INFO:teuthology.orchestra.run.smithi157.stdout:(29/30): librbd1-18.2.4-820.g48a7de7e.el9.x86_6 14 MB/s | 3.0 MB 00:00 2024-09-13T09:09:28.365 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 6/32 2024-09-13T09:09:28.391 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T09:09:28.453 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T09:09:28.464 INFO:teuthology.orchestra.run.smithi157.stdout:(30/30): librados2-18.2.4-820.g48a7de7e.el9.x86 9.1 MB/s | 3.3 MB 00:00 2024-09-13T09:09:28.470 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:09:28.470 INFO:teuthology.orchestra.run.smithi157.stdout:Total 16 MB/s | 53 MB 00:03 2024-09-13T09:09:28.490 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-13T09:09:28.533 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x 9/32 2024-09-13T09:09:28.542 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:09:28.580 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:09:28.580 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:09:28.601 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 10/32 2024-09-13T09:09:28.626 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T09:09:28.684 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T09:09:28.761 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:09:28.761 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:09:28.788 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-13T09:09:28.817 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-13T09:09:28.858 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-13T09:09:28.907 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-13T09:09:28.934 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-13T09:09:28.970 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-13T09:09:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-13T09:09:29.195 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:09:29.196 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:09:29.303 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-13T09:09:29.349 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 20/32 2024-09-13T09:09:29.446 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 20/32 2024-09-13T09:09:29.471 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 21/32 2024-09-13T09:09:29.506 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-13T09:09:29.915 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-13T09:09:29.944 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 24/32 2024-09-13T09:09:29.998 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 24/32 2024-09-13T09:09:30.066 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 25/32 2024-09-13T09:09:30.096 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86 26/32 2024-09-13T09:09:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:09:30.249 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-13T09:09:30.278 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-13T09:09:30.338 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-13T09:09:30.554 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:09:30.633 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-13T09:09:30.661 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-13T09:09:30.712 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-13T09:09:30.726 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-13T09:09:30.785 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T09:09:30.895 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T09:09:30.981 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 6/32 2024-09-13T09:09:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T09:09:31.066 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T09:09:31.078 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-13T09:09:31.095 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-13T09:09:31.127 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T09:09:31.139 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x 9/32 2024-09-13T09:09:31.207 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 10/32 2024-09-13T09:09:31.219 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T09:09:31.230 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T09:09:31.297 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T09:09:31.313 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 6/32 2024-09-13T09:09:31.345 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T09:09:31.391 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T09:09:31.406 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-13T09:09:31.428 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-13T09:09:31.438 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-13T09:09:31.471 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x 9/32 2024-09-13T09:09:31.479 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-13T09:09:31.511 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-13T09:09:31.538 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 10/32 2024-09-13T09:09:31.539 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-13T09:09:31.564 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T09:09:31.575 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-13T09:09:31.608 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-13T09:09:31.622 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T09:09:31.728 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-13T09:09:31.780 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-13T09:09:31.820 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-13T09:09:31.861 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-13T09:09:31.888 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-13T09:09:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-13T09:09:31.924 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-13T09:09:31.961 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 20/32 2024-09-13T09:09:31.963 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-13T09:09:32.087 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 20/32 2024-09-13T09:09:32.117 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 21/32 2024-09-13T09:09:32.153 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-13T09:09:32.218 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T09:09:32.252 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T09:09:32.266 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-13T09:09:32.294 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 20/32 2024-09-13T09:09:32.388 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 20/32 2024-09-13T09:09:32.426 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 21/32 2024-09-13T09:09:32.469 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-13T09:09:32.581 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-13T09:09:32.606 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 24/32 2024-09-13T09:09:32.678 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 24/32 2024-09-13T09:09:32.757 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 25/32 2024-09-13T09:09:32.783 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86 26/32 2024-09-13T09:09:32.786 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T09:09:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 28/32 2024-09-13T09:09:32.875 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-13T09:09:32.906 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 24/32 2024-09-13T09:09:32.927 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 28/32 2024-09-13T09:09:32.927 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T09:09:32.927 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T09:09:32.927 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:09:32.956 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T09:09:32.959 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 24/32 2024-09-13T09:09:33.028 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 25/32 2024-09-13T09:09:33.057 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86 26/32 2024-09-13T09:09:34.980 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T09:09:35.013 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T09:09:35.198 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T09:09:35.254 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T09:09:35.573 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T09:09:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 28/32 2024-09-13T09:09:35.703 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 28/32 2024-09-13T09:09:35.704 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T09:09:35.704 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T09:09:35.704 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:09:35.733 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T09:09:35.808 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T09:09:35.840 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 28/32 2024-09-13T09:09:35.944 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 28/32 2024-09-13T09:09:35.944 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T09:09:35.945 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T09:09:35.945 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:09:35.975 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T09:09:42.919 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /sys 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /proc 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /mnt 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /var/tmp 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /home 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /root 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /tmp 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:09:43.679 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 30/32 2024-09-13T09:09:43.705 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 30/32 2024-09-13T09:09:43.705 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:09:43.705 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T09:09:43.705 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:09:43.705 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:09:43.705 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:09:43.707 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T09:09:43.790 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T09:09:43.791 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T09:09:44.885 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T09:09:44.885 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 1/32 2024-09-13T09:09:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2/32 2024-09-13T09:09:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 3/32 2024-09-13T09:09:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 4/32 2024-09-13T09:09:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T09:09:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 6/32 2024-09-13T09:09:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T09:09:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x 8/32 2024-09-13T09:09:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86 9/32 2024-09-13T09:09:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 10/32 2024-09-13T09:09:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T09:09:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 12/32 2024-09-13T09:09:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 13/32 2024-09-13T09:09:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-13T09:09:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-13T09:09:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-13T09:09:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-13T09:09:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-13T09:09:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-13T09:09:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-13T09:09:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-13T09:09:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-13T09:09:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-13T09:09:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-13T09:09:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-13T09:09:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-13T09:09:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-13T09:09:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-13T09:09:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T09:09:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-13T09:09:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 31/32 2024-09-13T09:09:45.727 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T09:09:45.727 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2024-09-13T09:09:45.727 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2024-09-13T09:09:45.727 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2024-09-13T09:09:45.727 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2024-09-13T09:09:45.728 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2024-09-13T09:09:45.728 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /root 2024-09-13T09:09:45.728 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2024-09-13T09:09:45.728 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:09:45.800 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T09:09:45.800 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:09:45.801 INFO:teuthology.orchestra.run.smithi129.stdout:Upgraded: 2024-09-13T09:09:45.801 INFO:teuthology.orchestra.run.smithi129.stdout: librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:45.801 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:45.801 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:09:45.801 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T09:09:45.801 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:45.801 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:45.801 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:45.801 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:45.801 INFO:teuthology.orchestra.run.smithi129.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T09:09:45.801 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T09:09:45.802 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T09:09:45.802 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T09:09:45.802 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:45.802 INFO:teuthology.orchestra.run.smithi129.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T09:09:45.802 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T09:09:45.802 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T09:09:45.802 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:45.802 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T09:09:45.802 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:45.802 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T09:09:45.802 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T09:09:45.802 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T09:09:45.802 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:45.802 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:45.802 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:45.803 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T09:09:45.803 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:45.803 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:45.803 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:45.803 INFO:teuthology.orchestra.run.smithi129.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T09:09:45.803 INFO:teuthology.orchestra.run.smithi129.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T09:09:45.803 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:09:45.803 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:09:46.139 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T09:09:46.139 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2024-09-13T09:09:46.139 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2024-09-13T09:09:46.139 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2024-09-13T09:09:46.139 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2024-09-13T09:09:46.139 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2024-09-13T09:09:46.140 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /root 2024-09-13T09:09:46.140 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2024-09-13T09:09:46.140 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:09:46.180 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-test 2024-09-13T09:09:46.514 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 30/32 2024-09-13T09:09:46.540 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 30/32 2024-09-13T09:09:46.540 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:09:46.540 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T09:09:46.540 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:09:46.540 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:09:46.540 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:09:46.542 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T09:09:46.626 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T09:09:46.626 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T09:09:46.751 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:26 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:09:46.892 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:09:46.893 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:09:46.893 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-13T09:09:46.893 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:09:46.893 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:09:46.893 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 40 M 2024-09-13T09:09:46.893 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-13T09:09:46.893 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 166 k 2024-09-13T09:09:46.894 INFO:teuthology.orchestra.run.smithi129.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T09:09:46.894 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T09:09:46.894 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:09:46.894 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:09:46.894 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:09:46.894 INFO:teuthology.orchestra.run.smithi129.stdout:Install 4 Packages 2024-09-13T09:09:46.894 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:09:46.895 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 40 M 2024-09-13T09:09:46.895 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 164 M 2024-09-13T09:09:46.895 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:09:46.916 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 30/32 2024-09-13T09:09:46.943 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 30/32 2024-09-13T09:09:46.944 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:09:46.944 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T09:09:46.944 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:09:46.944 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:09:46.944 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:09:46.945 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T09:09:47.036 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T09:09:47.036 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T09:09:47.373 INFO:teuthology.orchestra.run.smithi129.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-13T09:09:47.399 INFO:teuthology.orchestra.run.smithi129.stdout:(2/4): libcephsqlite-18.2.4-820.g48a7de7e.el9.x 733 kB/s | 166 kB 00:00 2024-09-13T09:09:47.423 INFO:teuthology.orchestra.run.smithi129.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-13T09:09:47.701 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T09:09:47.701 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 1/32 2024-09-13T09:09:47.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2/32 2024-09-13T09:09:47.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 3/32 2024-09-13T09:09:47.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 4/32 2024-09-13T09:09:47.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T09:09:47.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 6/32 2024-09-13T09:09:47.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T09:09:47.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x 8/32 2024-09-13T09:09:47.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86 9/32 2024-09-13T09:09:47.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 10/32 2024-09-13T09:09:47.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T09:09:47.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 12/32 2024-09-13T09:09:47.702 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 13/32 2024-09-13T09:09:47.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-13T09:09:47.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-13T09:09:47.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-13T09:09:47.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-13T09:09:47.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-13T09:09:47.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-13T09:09:47.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-13T09:09:47.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-13T09:09:47.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-13T09:09:47.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-13T09:09:47.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-13T09:09:47.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-13T09:09:47.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-13T09:09:47.703 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-13T09:09:47.704 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-13T09:09:47.704 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T09:09:47.704 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-13T09:09:47.704 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 31/32 2024-09-13T09:09:48.200 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T09:09:48.200 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 1/32 2024-09-13T09:09:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2/32 2024-09-13T09:09:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 3/32 2024-09-13T09:09:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 4/32 2024-09-13T09:09:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T09:09:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 6/32 2024-09-13T09:09:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T09:09:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x 8/32 2024-09-13T09:09:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86 9/32 2024-09-13T09:09:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 10/32 2024-09-13T09:09:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T09:09:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 12/32 2024-09-13T09:09:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 13/32 2024-09-13T09:09:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-13T09:09:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-13T09:09:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-13T09:09:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-13T09:09:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-13T09:09:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-13T09:09:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-13T09:09:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-13T09:09:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-13T09:09:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-13T09:09:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-13T09:09:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-13T09:09:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-13T09:09:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-13T09:09:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-13T09:09:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T09:09:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-13T09:09:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 31/32 2024-09-13T09:09:48.515 INFO:teuthology.orchestra.run.smithi129.stdout:(4/4): ceph-test-18.2.4-820.g48a7de7e.el9.x86_6 30 MB/s | 40 MB 00:01 2024-09-13T09:09:48.518 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:09:48.518 INFO:teuthology.orchestra.run.smithi129.stdout:Total 25 MB/s | 40 MB 00:01 2024-09-13T09:09:48.546 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:09:48.566 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:09:48.566 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:09:48.837 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T09:09:48.838 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:09:48.838 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-09-13T09:09:48.838 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:48.838 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:48.838 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:09:48.838 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T09:09:48.838 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:48.838 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:48.838 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:48.838 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:48.838 INFO:teuthology.orchestra.run.smithi157.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T09:09:48.838 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T09:09:48.839 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T09:09:48.839 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T09:09:48.839 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:48.839 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T09:09:48.839 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T09:09:48.839 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T09:09:48.839 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:48.839 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T09:09:48.839 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:48.839 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T09:09:48.839 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T09:09:48.839 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T09:09:48.839 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:48.840 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:48.840 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:48.840 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T09:09:48.840 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:48.840 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:48.840 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:48.840 INFO:teuthology.orchestra.run.smithi157.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T09:09:48.840 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T09:09:48.840 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:09:48.840 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:09:48.887 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:09:48.887 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:09:49.083 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi044.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi044.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi044.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:09:49.182 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-test 2024-09-13T09:09:49.394 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:09:49.414 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-test 2024-09-13T09:09:49.468 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-13T09:09:49.516 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-13T09:09:49.542 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 3/4 2024-09-13T09:09:49.753 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:27 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:09:49.890 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:09:49.891 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:09:49.891 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-09-13T09:09:49.891 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:09:49.891 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:09:49.891 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 40 M 2024-09-13T09:09:49.891 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-09-13T09:09:49.892 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 166 k 2024-09-13T09:09:49.892 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T09:09:49.892 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T09:09:49.892 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:09:49.892 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:09:49.892 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:09:49.892 INFO:teuthology.orchestra.run.smithi157.stdout:Install 4 Packages 2024-09-13T09:09:49.892 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:09:49.893 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 40 M 2024-09-13T09:09:49.893 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 164 M 2024-09-13T09:09:49.893 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:09:50.002 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:29 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:09:50.145 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:09:50.145 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:09:50.145 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-09-13T09:09:50.145 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:09:50.145 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:09:50.146 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 40 M 2024-09-13T09:09:50.146 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-09-13T09:09:50.146 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 166 k 2024-09-13T09:09:50.146 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T09:09:50.146 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T09:09:50.146 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:09:50.146 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:09:50.146 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:09:50.146 INFO:teuthology.orchestra.run.smithi044.stdout:Install 4 Packages 2024-09-13T09:09:50.146 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:09:50.147 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 40 M 2024-09-13T09:09:50.147 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 164 M 2024-09-13T09:09:50.147 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:09:50.256 INFO:teuthology.orchestra.run.smithi157.stdout:(1/4): libcephsqlite-18.2.4-820.g48a7de7e.el9.x 712 kB/s | 166 kB 00:00 2024-09-13T09:09:50.391 INFO:teuthology.orchestra.run.smithi044.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-09-13T09:09:50.433 INFO:teuthology.orchestra.run.smithi044.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-13T09:09:50.633 INFO:teuthology.orchestra.run.smithi044.stdout:(3/4): libcephsqlite-18.2.4-820.g48a7de7e.el9.x 458 kB/s | 166 kB 00:00 2024-09-13T09:09:50.865 INFO:teuthology.orchestra.run.smithi157.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 361 kB/s | 304 kB 00:00 2024-09-13T09:09:50.932 INFO:teuthology.orchestra.run.smithi157.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 95 kB/s | 64 kB 00:00 2024-09-13T09:09:51.316 INFO:teuthology.orchestra.run.smithi157.stdout:(4/4): ceph-test-18.2.4-820.g48a7de7e.el9.x86_6 31 MB/s | 40 MB 00:01 2024-09-13T09:09:51.318 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:09:51.319 INFO:teuthology.orchestra.run.smithi157.stdout:Total 28 MB/s | 40 MB 00:01 2024-09-13T09:09:51.345 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:09:51.366 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:09:51.366 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:09:51.702 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:09:51.703 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:09:52.184 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:09:52.259 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-13T09:09:52.301 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-13T09:09:52.326 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 3/4 2024-09-13T09:09:52.525 INFO:teuthology.orchestra.run.smithi044.stdout:(4/4): ceph-test-18.2.4-820.g48a7de7e.el9.x86_6 18 MB/s | 40 MB 00:02 2024-09-13T09:09:52.527 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:09:52.528 INFO:teuthology.orchestra.run.smithi044.stdout:Total 17 MB/s | 40 MB 00:02 2024-09-13T09:09:52.556 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:09:52.578 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:09:52.578 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:09:52.924 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:09:52.924 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:09:53.215 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 3/4 2024-09-13T09:09:53.290 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 4/4 2024-09-13T09:09:53.424 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:09:53.504 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-13T09:09:53.550 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-13T09:09:53.577 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 3/4 2024-09-13T09:09:55.267 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 4/4 2024-09-13T09:09:55.267 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 1/4 2024-09-13T09:09:55.267 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 2/4 2024-09-13T09:09:55.267 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-13T09:09:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-13T09:09:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:09:55.606 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:09:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:55.606 INFO:teuthology.orchestra.run.smithi129.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T09:09:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T09:09:55.607 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:09:55.607 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:09:55.910 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph 2024-09-13T09:09:56.024 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 3/4 2024-09-13T09:09:56.098 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 4/4 2024-09-13T09:09:56.475 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:36 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:09:56.619 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:09:56.622 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:09:56.622 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-13T09:09:56.622 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:09:56.622 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:09:56.622 INFO:teuthology.orchestra.run.smithi129.stdout: ceph x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 6.5 k 2024-09-13T09:09:56.622 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-13T09:09:56.622 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 2.1 M 2024-09-13T09:09:56.622 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 1.5 M 2024-09-13T09:09:56.622 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 246 k 2024-09-13T09:09:56.622 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 4.7 M 2024-09-13T09:09:56.623 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 17 M 2024-09-13T09:09:56.623 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T09:09:56.623 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T09:09:56.623 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T09:09:56.623 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T09:09:56.623 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T09:09:56.623 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T09:09:56.623 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T09:09:56.623 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T09:09:56.623 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T09:09:56.623 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T09:09:56.623 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T09:09:56.624 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T09:09:56.624 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T09:09:56.624 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T09:09:56.624 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T09:09:56.624 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T09:09:56.624 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T09:09:56.624 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T09:09:56.624 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T09:09:56.624 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T09:09:56.624 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T09:09:56.624 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T09:09:56.624 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T09:09:56.625 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T09:09:56.625 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T09:09:56.625 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T09:09:56.625 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T09:09:56.625 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:09:56.625 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:09:56.625 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:09:56.625 INFO:teuthology.orchestra.run.smithi129.stdout:Install 33 Packages 2024-09-13T09:09:56.625 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:09:56.626 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 29 M 2024-09-13T09:09:56.626 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 104 M 2024-09-13T09:09:56.626 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:09:57.268 INFO:teuthology.orchestra.run.smithi129.stdout:(1/33): ceph-18.2.4-820.g48a7de7e.el9.x86_64.rp 50 kB/s | 6.5 kB 00:00 2024-09-13T09:09:57.451 INFO:teuthology.orchestra.run.smithi129.stdout:(2/33): ceph-mgr-18.2.4-820.g48a7de7e.el9.x86_6 4.6 MB/s | 1.5 MB 00:00 2024-09-13T09:09:57.490 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 3/4 2024-09-13T09:09:57.510 INFO:teuthology.orchestra.run.smithi129.stdout:(3/33): ceph-mds-18.2.4-820.g48a7de7e.el9.x86_6 5.6 MB/s | 2.1 MB 00:00 2024-09-13T09:09:57.570 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 4/4 2024-09-13T09:09:57.611 INFO:teuthology.orchestra.run.smithi129.stdout:(4/33): ceph-mgr-modules-core-18.2.4-820.g48a7d 2.4 MB/s | 246 kB 00:00 2024-09-13T09:09:57.744 INFO:teuthology.orchestra.run.smithi129.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-09-13T09:09:57.836 INFO:teuthology.orchestra.run.smithi129.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 13 MB/s | 1.2 MB 00:00 2024-09-13T09:09:57.895 INFO:teuthology.orchestra.run.smithi129.stdout:(7/33): ceph-mon-18.2.4-820.g48a7de7e.el9.x86_6 7.5 MB/s | 4.7 MB 00:00 2024-09-13T09:09:57.945 INFO:teuthology.orchestra.run.smithi129.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 982 kB/s | 106 kB 00:00 2024-09-13T09:09:58.064 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 4/4 2024-09-13T09:09:58.064 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 1/4 2024-09-13T09:09:58.064 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 2/4 2024-09-13T09:09:58.064 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-13T09:09:58.112 INFO:teuthology.orchestra.run.smithi129.stdout:(9/33): ceph-osd-18.2.4-820.g48a7de7e.el9.x86_6 25 MB/s | 17 MB 00:00 2024-09-13T09:09:58.129 INFO:teuthology.orchestra.run.smithi129.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 680 kB/s | 125 kB 00:00 2024-09-13T09:09:58.171 INFO:teuthology.orchestra.run.smithi129.stdout:(11/33): python3-pycparser-2.20-6.el9.noarch.rp 490 kB/s | 135 kB 00:00 2024-09-13T09:09:58.238 INFO:teuthology.orchestra.run.smithi129.stdout:(12/33): python3-urllib3-1.26.5-6.el9.noarch.rp 1.7 MB/s | 215 kB 00:00 2024-09-13T09:09:58.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-13T09:09:58.383 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:09:58.383 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:09:58.384 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:58.384 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:58.384 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T09:09:58.384 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T09:09:58.384 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:09:58.384 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:09:58.707 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph 2024-09-13T09:09:58.738 INFO:teuthology.orchestra.run.smithi129.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 61 kB/s | 35 kB 00:00 2024-09-13T09:09:58.847 INFO:teuthology.orchestra.run.smithi129.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 84 kB/s | 51 kB 00:00 2024-09-13T09:09:58.872 INFO:teuthology.orchestra.run.smithi129.stdout:(15/33): python3-mako-1.1.4-6.el9.noarch.rpm 232 kB/s | 172 kB 00:00 2024-09-13T09:09:58.914 INFO:teuthology.orchestra.run.smithi129.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 652 kB/s | 43 kB 00:00 2024-09-13T09:09:58.948 INFO:teuthology.orchestra.run.smithi129.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-13T09:09:58.973 INFO:teuthology.orchestra.run.smithi129.stdout:(18/33): python3-cherrypy-18.6.1-2.el9.noarch.r 6.2 MB/s | 358 kB 00:00 2024-09-13T09:09:58.990 INFO:teuthology.orchestra.run.smithi129.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 650 kB/s | 11 kB 00:00 2024-09-13T09:09:59.015 INFO:teuthology.orchestra.run.smithi129.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 424 kB/s | 18 kB 00:00 2024-09-13T09:09:59.032 INFO:teuthology.orchestra.run.smithi129.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 673 kB/s | 23 kB 00:00 2024-09-13T09:09:59.049 INFO:teuthology.orchestra.run.smithi129.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 582 kB/s | 19 kB 00:00 2024-09-13T09:09:59.074 INFO:teuthology.orchestra.run.smithi129.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 473 kB/s | 20 kB 00:00 2024-09-13T09:09:59.099 INFO:teuthology.orchestra.run.smithi129.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 921 kB/s | 46 kB 00:00 2024-09-13T09:09:59.125 INFO:teuthology.orchestra.run.smithi129.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.6 MB/s | 79 kB 00:00 2024-09-13T09:09:59.150 INFO:teuthology.orchestra.run.smithi129.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 657 kB/s | 16 kB 00:00 2024-09-13T09:09:59.175 INFO:teuthology.orchestra.run.smithi129.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-13T09:09:59.201 INFO:teuthology.orchestra.run.smithi129.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-13T09:09:59.226 INFO:teuthology.orchestra.run.smithi129.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 712 kB/s | 36 kB 00:00 2024-09-13T09:09:59.251 INFO:teuthology.orchestra.run.smithi129.stdout:(30/33): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-13T09:09:59.274 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:37 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:09:59.277 INFO:teuthology.orchestra.run.smithi129.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 802 kB/s | 20 kB 00:00 2024-09-13T09:09:59.302 INFO:teuthology.orchestra.run.smithi129.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.5 MB/s | 427 kB 00:00 2024-09-13T09:09:59.352 INFO:teuthology.orchestra.run.smithi129.stdout:(33/33): python3-toml-0.10.2-6.el9.noarch.rpm 68 kB/s | 42 kB 00:00 2024-09-13T09:09:59.357 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:09:59.357 INFO:teuthology.orchestra.run.smithi129.stdout:Total 11 MB/s | 29 MB 00:02 2024-09-13T09:09:59.410 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:09:59.412 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:09:59.412 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-09-13T09:09:59.413 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:09:59.413 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:09:59.413 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 6.5 k 2024-09-13T09:09:59.413 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-09-13T09:09:59.413 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 2.1 M 2024-09-13T09:09:59.413 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 1.5 M 2024-09-13T09:09:59.413 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 246 k 2024-09-13T09:09:59.413 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 4.7 M 2024-09-13T09:09:59.413 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 17 M 2024-09-13T09:09:59.413 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T09:09:59.413 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T09:09:59.413 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T09:09:59.414 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T09:09:59.414 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T09:09:59.414 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T09:09:59.414 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T09:09:59.414 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T09:09:59.414 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T09:09:59.414 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T09:09:59.414 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T09:09:59.414 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T09:09:59.414 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T09:09:59.414 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T09:09:59.414 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T09:09:59.414 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T09:09:59.415 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T09:09:59.415 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T09:09:59.415 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T09:09:59.415 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T09:09:59.415 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T09:09:59.415 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T09:09:59.415 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T09:09:59.415 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T09:09:59.415 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T09:09:59.415 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T09:09:59.415 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T09:09:59.415 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:09:59.416 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:09:59.416 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:09:59.416 INFO:teuthology.orchestra.run.smithi157.stdout:Install 33 Packages 2024-09-13T09:09:59.416 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:09:59.416 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 29 M 2024-09-13T09:09:59.416 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 104 M 2024-09-13T09:09:59.417 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:09:59.479 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:09:59.516 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:09:59.516 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:09:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 4/4 2024-09-13T09:09:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 1/4 2024-09-13T09:09:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 2/4 2024-09-13T09:09:59.551 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-13T09:09:59.849 INFO:teuthology.orchestra.run.smithi157.stdout:(1/33): ceph-18.2.4-820.g48a7de7e.el9.x86_64.rp 50 kB/s | 6.5 kB 00:00 2024-09-13T09:09:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-13T09:09:59.885 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:09:59.886 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:09:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:09:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T09:09:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T09:09:59.886 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:09:59.886 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:09:59.925 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:09:59.925 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:09:59.999 INFO:teuthology.orchestra.run.smithi157.stdout:(2/33): ceph-mgr-18.2.4-820.g48a7de7e.el9.x86_6 5.2 MB/s | 1.5 MB 00:00 2024-09-13T09:10:00.066 INFO:teuthology.orchestra.run.smithi157.stdout:(3/33): ceph-mds-18.2.4-820.g48a7de7e.el9.x86_6 6.0 MB/s | 2.1 MB 00:00 2024-09-13T09:10:00.166 INFO:teuthology.orchestra.run.smithi157.stdout:(4/33): ceph-mgr-modules-core-18.2.4-820.g48a7d 2.4 MB/s | 246 kB 00:00 2024-09-13T09:10:00.186 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph 2024-09-13T09:10:00.225 INFO:teuthology.orchestra.run.smithi157.stdout:(5/33): ceph-mon-18.2.4-820.g48a7de7e.el9.x86_6 12 MB/s | 4.7 MB 00:00 2024-09-13T09:10:00.400 INFO:teuthology.orchestra.run.smithi157.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-13T09:10:00.458 INFO:teuthology.orchestra.run.smithi157.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-13T09:10:00.617 INFO:teuthology.orchestra.run.smithi157.stdout:(8/33): ceph-osd-18.2.4-820.g48a7de7e.el9.x86_6 27 MB/s | 17 MB 00:00 2024-09-13T09:10:00.642 INFO:teuthology.orchestra.run.smithi157.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 737 kB/s | 135 kB 00:00 2024-09-13T09:10:00.684 INFO:teuthology.orchestra.run.smithi157.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 2.7 MB/s | 1.2 MB 00:00 2024-09-13T09:10:00.710 INFO:teuthology.orchestra.run.smithi157.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-09-13T09:10:00.767 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:39 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:10:00.793 INFO:teuthology.orchestra.run.smithi157.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 417 kB/s | 35 kB 00:00 2024-09-13T09:10:00.818 INFO:teuthology.orchestra.run.smithi157.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-13T09:10:00.844 INFO:teuthology.orchestra.run.smithi157.stdout:(14/33): python3-requests-2.25.1-8.el9.noarch.r 552 kB/s | 125 kB 00:00 2024-09-13T09:10:00.869 INFO:teuthology.orchestra.run.smithi157.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-13T09:10:00.894 INFO:teuthology.orchestra.run.smithi157.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 552 kB/s | 42 kB 00:00 2024-09-13T09:10:00.913 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:10:00.916 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:00.916 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-09-13T09:10:00.916 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:00.916 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:10:00.916 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 6.5 k 2024-09-13T09:10:00.916 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-09-13T09:10:00.916 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 2.1 M 2024-09-13T09:10:00.916 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 1.5 M 2024-09-13T09:10:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 246 k 2024-09-13T09:10:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 4.7 M 2024-09-13T09:10:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 17 M 2024-09-13T09:10:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T09:10:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T09:10:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T09:10:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T09:10:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T09:10:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T09:10:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T09:10:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T09:10:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T09:10:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T09:10:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T09:10:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T09:10:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T09:10:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T09:10:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T09:10:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T09:10:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T09:10:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T09:10:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T09:10:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T09:10:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T09:10:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T09:10:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T09:10:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T09:10:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T09:10:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T09:10:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T09:10:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:00.919 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:10:00.919 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:00.919 INFO:teuthology.orchestra.run.smithi044.stdout:Install 33 Packages 2024-09-13T09:10:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:00.920 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 29 M 2024-09-13T09:10:00.920 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 104 M 2024-09-13T09:10:00.920 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:10:00.944 INFO:teuthology.orchestra.run.smithi157.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 432 kB/s | 43 kB 00:00 2024-09-13T09:10:00.970 INFO:teuthology.orchestra.run.smithi157.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 460 kB/s | 11 kB 00:00 2024-09-13T09:10:00.995 INFO:teuthology.orchestra.run.smithi157.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 709 kB/s | 18 kB 00:00 2024-09-13T09:10:01.020 INFO:teuthology.orchestra.run.smithi157.stdout:(20/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-13T09:10:01.045 INFO:teuthology.orchestra.run.smithi157.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-13T09:10:01.071 INFO:teuthology.orchestra.run.smithi157.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-09-13T09:10:01.096 INFO:teuthology.orchestra.run.smithi157.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-13T09:10:01.121 INFO:teuthology.orchestra.run.smithi157.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 921 kB/s | 46 kB 00:00 2024-09-13T09:10:01.146 INFO:teuthology.orchestra.run.smithi157.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-13T09:10:01.166 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:10:01.172 INFO:teuthology.orchestra.run.smithi157.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-13T09:10:01.197 INFO:teuthology.orchestra.run.smithi157.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-13T09:10:01.222 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-13T09:10:01.231 INFO:teuthology.orchestra.run.smithi157.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-13T09:10:01.254 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-13T09:10:01.256 INFO:teuthology.orchestra.run.smithi157.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 607 kB/s | 36 kB 00:00 2024-09-13T09:10:01.281 INFO:teuthology.orchestra.run.smithi157.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-13T09:10:01.287 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-13T09:10:01.306 INFO:teuthology.orchestra.run.smithi157.stdout:(31/33): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-13T09:10:01.332 INFO:teuthology.orchestra.run.smithi157.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 397 kB/s | 20 kB 00:00 2024-09-13T09:10:01.357 INFO:teuthology.orchestra.run.smithi157.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-13T09:10:01.362 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:01.362 INFO:teuthology.orchestra.run.smithi157.stdout:Total 15 MB/s | 29 MB 00:01 2024-09-13T09:10:01.379 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-13T09:10:01.478 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-13T09:10:01.481 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:10:01.519 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-13T09:10:01.519 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:10:01.520 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:10:01.553 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-13T09:10:01.584 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-13T09:10:01.584 INFO:teuthology.orchestra.run.smithi044.stdout:(1/33): ceph-18.2.4-820.g48a7de7e.el9.x86_64.rp 23 kB/s | 6.5 kB 00:00 2024-09-13T09:10:01.618 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-13T09:10:01.651 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-13T09:10:01.692 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-13T09:10:01.735 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-13T09:10:01.770 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-13T09:10:01.808 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-13T09:10:01.891 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-13T09:10:01.941 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:10:01.941 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:10:02.040 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-13T09:10:02.067 INFO:teuthology.orchestra.run.smithi044.stdout:(2/33): ceph-mgr-18.2.4-820.g48a7de7e.el9.x86_6 1.9 MB/s | 1.5 MB 00:00 2024-09-13T09:10:02.127 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-13T09:10:02.159 INFO:teuthology.orchestra.run.smithi044.stdout:(3/33): ceph-mds-18.2.4-820.g48a7de7e.el9.x86_6 2.5 MB/s | 2.1 MB 00:00 2024-09-13T09:10:02.180 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-13T09:10:02.218 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-13T09:10:02.268 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-13T09:10:02.268 INFO:teuthology.orchestra.run.smithi044.stdout:(4/33): ceph-mgr-modules-core-18.2.4-820.g48a7d 2.2 MB/s | 246 kB 00:00 2024-09-13T09:10:02.323 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-13T09:10:02.518 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-13T09:10:02.565 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-13T09:10:02.643 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-13T09:10:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-13T09:10:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-13T09:10:02.893 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-13T09:10:03.027 INFO:teuthology.orchestra.run.smithi044.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 334 kB/s | 253 kB 00:00 2024-09-13T09:10:03.064 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.n 28/33 2024-09-13T09:10:03.088 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 29/33 2024-09-13T09:10:03.114 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 29/33 2024-09-13T09:10:03.114 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:03.115 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T09:10:03.115 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T09:10:03.115 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T09:10:03.115 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:03.189 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:10:03.260 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-13T09:10:03.292 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-13T09:10:03.333 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-13T09:10:03.425 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-13T09:10:03.530 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-13T09:10:03.586 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-13T09:10:03.629 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-13T09:10:03.659 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-13T09:10:03.694 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-13T09:10:03.722 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-13T09:10:03.763 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-13T09:10:03.798 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-13T09:10:03.832 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-13T09:10:03.873 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-13T09:10:03.961 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-13T09:10:04.107 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-13T09:10:04.126 INFO:teuthology.orchestra.run.smithi044.stdout:(6/33): ceph-osd-18.2.4-820.g48a7de7e.el9.x86_6 8.0 MB/s | 17 MB 00:02 2024-09-13T09:10:04.169 INFO:teuthology.orchestra.run.smithi044.stdout:(7/33): python3-cryptography-36.0.1-4.el9.x86_6 1.1 MB/s | 1.2 MB 00:01 2024-09-13T09:10:04.186 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-13T09:10:04.240 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-13T09:10:04.286 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-13T09:10:04.335 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-13T09:10:04.391 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-13T09:10:04.578 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-13T09:10:04.581 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 30/33 2024-09-13T09:10:04.610 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 30/33 2024-09-13T09:10:04.610 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:04.610 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T09:10:04.610 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T09:10:04.610 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T09:10:04.610 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:04.632 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-13T09:10:04.710 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-13T09:10:04.808 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-13T09:10:04.810 INFO:teuthology.orchestra.run.smithi044.stdout:(8/33): ceph-mon-18.2.4-820.g48a7de7e.el9.x86_6 1.4 MB/s | 4.7 MB 00:03 2024-09-13T09:10:04.835 INFO:teuthology.orchestra.run.smithi044.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 150 kB/s | 106 kB 00:00 2024-09-13T09:10:04.850 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-13T09:10:04.861 INFO:teuthology.orchestra.run.smithi044.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 195 kB/s | 135 kB 00:00 2024-09-13T09:10:04.982 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-13T09:10:05.053 INFO:teuthology.orchestra.run.smithi044.stdout:(11/33): python3-mako-1.1.4-6.el9.noarch.rpm 900 kB/s | 172 kB 00:00 2024-09-13T09:10:05.057 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 31/33 2024-09-13T09:10:05.081 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 31/33 2024-09-13T09:10:05.082 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:05.082 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T09:10:05.082 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T09:10:05.082 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T09:10:05.082 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:05.112 INFO:teuthology.orchestra.run.smithi044.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 691 kB/s | 35 kB 00:00 2024-09-13T09:10:05.147 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.n 28/33 2024-09-13T09:10:05.162 INFO:teuthology.orchestra.run.smithi044.stdout:(13/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-13T09:10:05.174 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 29/33 2024-09-13T09:10:05.200 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 29/33 2024-09-13T09:10:05.200 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:05.200 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T09:10:05.200 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T09:10:05.200 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T09:10:05.200 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:05.213 INFO:teuthology.orchestra.run.smithi044.stdout:(14/33): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-09-13T09:10:05.254 INFO:teuthology.orchestra.run.smithi044.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.0 MB/s | 43 kB 00:00 2024-09-13T09:10:05.279 INFO:teuthology.orchestra.run.smithi044.stdout:(16/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 6.7 MB/s | 172 kB 00:00 2024-09-13T09:10:05.282 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 32/33 2024-09-13T09:10:05.307 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 32/33 2024-09-13T09:10:05.307 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:05.307 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T09:10:05.307 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T09:10:05.307 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T09:10:05.307 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:05.313 INFO:teuthology.orchestra.run.smithi044.stdout:(17/33): python3-cherrypy-18.6.1-2.el9.noarch.r 11 MB/s | 358 kB 00:00 2024-09-13T09:10:05.338 INFO:teuthology.orchestra.run.smithi044.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-13T09:10:05.363 INFO:teuthology.orchestra.run.smithi044.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 707 kB/s | 18 kB 00:00 2024-09-13T09:10:05.372 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 33/33 2024-09-13T09:10:05.389 INFO:teuthology.orchestra.run.smithi044.stdout:(20/33): python3-jaraco-collections-3.0.0-8.el9 928 kB/s | 23 kB 00:00 2024-09-13T09:10:05.414 INFO:teuthology.orchestra.run.smithi044.stdout:(21/33): python3-jaraco-functools-3.5.0-2.el9.n 782 kB/s | 19 kB 00:00 2024-09-13T09:10:05.439 INFO:teuthology.orchestra.run.smithi044.stdout:(22/33): python3-jaraco-text-3.2.0-6.el9.noarch 793 kB/s | 20 kB 00:00 2024-09-13T09:10:05.473 INFO:teuthology.orchestra.run.smithi044.stdout:(23/33): python3-logutils-0.3.5-21.el9.noarch.r 1.4 MB/s | 46 kB 00:00 2024-09-13T09:10:05.490 INFO:teuthology.orchestra.run.smithi044.stdout:(24/33): python3-more-itertools-8.12.0-2.el9.no 4.6 MB/s | 79 kB 00:00 2024-09-13T09:10:05.515 INFO:teuthology.orchestra.run.smithi044.stdout:(25/33): python3-requests-2.25.1-8.el9.noarch.r 177 kB/s | 125 kB 00:00 2024-09-13T09:10:05.549 INFO:teuthology.orchestra.run.smithi044.stdout:(26/33): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-09-13T09:10:05.574 INFO:teuthology.orchestra.run.smithi044.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 281 kB/s | 16 kB 00:00 2024-09-13T09:10:05.599 INFO:teuthology.orchestra.run.smithi044.stdout:(28/33): python3-urllib3-1.26.5-6.el9.noarch.rp 281 kB/s | 215 kB 00:00 2024-09-13T09:10:05.625 INFO:teuthology.orchestra.run.smithi044.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-13T09:10:05.650 INFO:teuthology.orchestra.run.smithi044.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 474 kB/s | 36 kB 00:00 2024-09-13T09:10:05.683 INFO:teuthology.orchestra.run.smithi044.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 810 kB/s | 20 kB 00:00 2024-09-13T09:10:05.717 INFO:teuthology.orchestra.run.smithi044.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-13T09:10:05.750 INFO:teuthology.orchestra.run.smithi044.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-09-13T09:10:05.757 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:05.757 INFO:teuthology.orchestra.run.smithi044.stdout:Total 6.0 MB/s | 29 MB 00:04 2024-09-13T09:10:05.881 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:10:05.922 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:10:05.922 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:10:06.343 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:10:06.343 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:10:06.659 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 30/33 2024-09-13T09:10:06.689 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 30/33 2024-09-13T09:10:06.690 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:06.690 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T09:10:06.690 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T09:10:06.690 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T09:10:06.690 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 33/33 2024-09-13T09:10:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 1/33 2024-09-13T09:10:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 2/33 2024-09-13T09:10:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 3/33 2024-09-13T09:10:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 4/33 2024-09-13T09:10:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 5/33 2024-09-13T09:10:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.n 6/33 2024-09-13T09:10:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-13T09:10:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-13T09:10:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-13T09:10:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-13T09:10:06.954 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-13T09:10:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-13T09:10:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-13T09:10:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-13T09:10:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-13T09:10:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-13T09:10:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-13T09:10:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-13T09:10:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-13T09:10:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-13T09:10:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-13T09:10:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-13T09:10:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-13T09:10:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-13T09:10:06.955 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-13T09:10:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-13T09:10:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-13T09:10:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-13T09:10:06.956 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-13T09:10:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-13T09:10:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-13T09:10:06.957 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-13T09:10:07.127 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 31/33 2024-09-13T09:10:07.150 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 31/33 2024-09-13T09:10:07.150 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:07.150 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T09:10:07.150 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T09:10:07.150 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T09:10:07.150 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:07.361 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 32/33 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 32/33 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:07.449 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 33/33 2024-09-13T09:10:07.587 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:10:07.637 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-13T09:10:07.670 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-13T09:10:07.703 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-13T09:10:07.800 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-13T09:10:07.877 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-13T09:10:07.939 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-13T09:10:07.974 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-13T09:10:08.003 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:08.030 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:08.038 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-13T09:10:08.071 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-13T09:10:08.112 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-13T09:10:08.156 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-13T09:10:08.193 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-13T09:10:08.237 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-13T09:10:08.317 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-13T09:10:08.362 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-base 2024-09-13T09:10:08.471 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-13T09:10:08.559 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-13T09:10:08.610 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-13T09:10:08.647 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-13T09:10:08.706 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-13T09:10:08.770 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-13T09:10:08.932 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:48 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:08.970 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-13T09:10:09.018 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-13T09:10:09.022 INFO:teuthology.orchestra.run.smithi129.stdout:Package ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:09.063 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:09.064 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-13T09:10:09.064 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:09.100 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-13T09:10:09.136 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install cephadm 2024-09-13T09:10:09.176 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 33/33 2024-09-13T09:10:09.176 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 1/33 2024-09-13T09:10:09.176 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 2/33 2024-09-13T09:10:09.176 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 3/33 2024-09-13T09:10:09.176 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 4/33 2024-09-13T09:10:09.177 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 5/33 2024-09-13T09:10:09.177 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.n 6/33 2024-09-13T09:10:09.177 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-13T09:10:09.177 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-13T09:10:09.177 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-13T09:10:09.177 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-13T09:10:09.177 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-13T09:10:09.177 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-13T09:10:09.177 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-13T09:10:09.177 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-13T09:10:09.177 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-13T09:10:09.177 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-13T09:10:09.178 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-13T09:10:09.178 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-13T09:10:09.178 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-13T09:10:09.178 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-13T09:10:09.178 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-13T09:10:09.178 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-13T09:10:09.178 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-13T09:10:09.178 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-13T09:10:09.178 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-13T09:10:09.178 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-13T09:10:09.178 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-13T09:10:09.179 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-13T09:10:09.179 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-13T09:10:09.179 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-13T09:10:09.179 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-13T09:10:09.179 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-13T09:10:09.197 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-13T09:10:09.238 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-13T09:10:09.365 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-13T09:10:09.542 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.n 28/33 2024-09-13T09:10:09.568 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 29/33 2024-09-13T09:10:09.595 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 29/33 2024-09-13T09:10:09.595 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:09.595 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T09:10:09.595 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T09:10:09.596 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T09:10:09.596 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:09.695 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:49 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:09.827 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:09.828 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:09.828 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-13T09:10:09.828 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:09.828 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:10:09.828 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 226 k 2024-09-13T09:10:09.828 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:09.828 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:10:09.828 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:09.828 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-13T09:10:09.828 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:09.829 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 226 k 2024-09-13T09:10:09.829 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 217 k 2024-09-13T09:10:09.829 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:10:10.049 INFO:teuthology.orchestra.run.smithi129.stdout:cephadm-18.2.4-820.g48a7de7e.el9.noarch.rpm 1.0 MB/s | 226 kB 00:00 2024-09-13T09:10:10.049 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:10.050 INFO:teuthology.orchestra.run.smithi129.stdout:Total 1.0 MB/s | 226 kB 00:00 2024-09-13T09:10:10.050 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:10:10.055 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:10:10.055 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:10:10.058 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:10:10.058 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:10:10.098 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-13T09:10:10.098 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:10.098 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:10:10.098 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:10.099 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:10.099 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:10.099 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:10.099 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:10.099 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:10.099 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T09:10:10.099 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T09:10:10.099 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T09:10:10.099 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T09:10:10.099 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T09:10:10.099 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T09:10:10.099 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T09:10:10.099 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T09:10:10.099 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T09:10:10.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T09:10:10.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T09:10:10.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T09:10:10.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T09:10:10.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T09:10:10.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T09:10:10.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T09:10:10.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T09:10:10.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T09:10:10.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T09:10:10.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T09:10:10.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T09:10:10.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T09:10:10.101 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T09:10:10.101 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T09:10:10.101 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T09:10:10.101 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T09:10:10.101 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T09:10:10.101 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:10.101 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:10.127 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:10:10.449 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-base 2024-09-13T09:10:10.604 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T09:10:10.686 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T09:10:11.025 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:49 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:11.038 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T09:10:11.113 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 30/33 2024-09-13T09:10:11.114 INFO:teuthology.orchestra.run.smithi157.stdout:Package ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:11.143 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 30/33 2024-09-13T09:10:11.144 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:11.144 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T09:10:11.144 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T09:10:11.144 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T09:10:11.144 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:11.155 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:11.157 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-09-13T09:10:11.157 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:11.221 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install cephadm 2024-09-13T09:10:11.305 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T09:10:11.305 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:11.546 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-immutable-object-cache 2024-09-13T09:10:11.612 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 31/33 2024-09-13T09:10:11.638 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 31/33 2024-09-13T09:10:11.638 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:11.638 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T09:10:11.638 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T09:10:11.638 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T09:10:11.638 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:11.786 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:49 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:11.846 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 32/33 2024-09-13T09:10:11.872 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 32/33 2024-09-13T09:10:11.872 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:11.872 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T09:10:11.872 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T09:10:11.872 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T09:10:11.872 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:11.919 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:11.920 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:11.920 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-09-13T09:10:11.920 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:11.920 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:10:11.921 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 226 k 2024-09-13T09:10:11.921 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:11.921 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:10:11.921 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:11.921 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-09-13T09:10:11.921 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:11.921 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 226 k 2024-09-13T09:10:11.921 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 217 k 2024-09-13T09:10:11.921 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:10:11.938 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 33/33 2024-09-13T09:10:12.123 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:52 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:12.159 INFO:teuthology.orchestra.run.smithi157.stdout:cephadm-18.2.4-820.g48a7de7e.el9.noarch.rpm 951 kB/s | 226 kB 00:00 2024-09-13T09:10:12.159 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:12.160 INFO:teuthology.orchestra.run.smithi157.stdout:Total 945 kB/s | 226 kB 00:00 2024-09-13T09:10:12.160 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:10:12.165 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:10:12.165 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:10:12.168 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:10:12.168 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:10:12.234 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:10:12.255 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:12.256 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-13T09:10:12.256 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:12.256 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:10:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 143 k 2024-09-13T09:10:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:12.256 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:10:12.256 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:12.256 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-13T09:10:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:12.257 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 143 k 2024-09-13T09:10:12.257 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 395 k 2024-09-13T09:10:12.257 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:10:12.464 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-immutable-object-cache-18.2.4-820.g48a7de7 690 kB/s | 143 kB 00:00 2024-09-13T09:10:12.465 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:12.465 INFO:teuthology.orchestra.run.smithi129.stdout:Total 684 kB/s | 143 kB 00:00 2024-09-13T09:10:12.466 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:10:12.473 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:10:12.473 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:10:12.529 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:10:12.530 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:10:12.672 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:10:12.713 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T09:10:12.749 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T09:10:12.774 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T09:10:12.774 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:12.774 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T09:10:12.774 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:12.803 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T09:10:13.149 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T09:10:13.382 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T09:10:13.383 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:13.383 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:10:13.383 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:13.383 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:13.383 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:13.572 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-immutable-object-cache 2024-09-13T09:10:13.630 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 33/33 2024-09-13T09:10:13.630 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 1/33 2024-09-13T09:10:13.630 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 2/33 2024-09-13T09:10:13.630 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 3/33 2024-09-13T09:10:13.630 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 4/33 2024-09-13T09:10:13.630 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 5/33 2024-09-13T09:10:13.631 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.n 6/33 2024-09-13T09:10:13.631 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-13T09:10:13.631 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-13T09:10:13.631 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-13T09:10:13.631 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-13T09:10:13.631 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-13T09:10:13.631 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-13T09:10:13.631 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-13T09:10:13.631 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-13T09:10:13.631 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-13T09:10:13.631 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-13T09:10:13.631 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-13T09:10:13.631 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-13T09:10:13.631 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-13T09:10:13.632 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-13T09:10:13.632 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-13T09:10:13.632 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-13T09:10:13.632 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-13T09:10:13.632 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-13T09:10:13.632 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-13T09:10:13.632 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-13T09:10:13.632 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-13T09:10:13.632 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-13T09:10:13.633 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-13T09:10:13.633 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-13T09:10:13.633 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-13T09:10:13.633 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-13T09:10:13.725 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T09:10:13.725 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:13.725 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:10:13.725 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:13.725 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:13.725 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:13.960 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr 2024-09-13T09:10:14.147 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:52 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:14.279 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:14.280 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:14.280 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2024-09-13T09:10:14.280 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:14.280 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:10:14.280 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 143 k 2024-09-13T09:10:14.280 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:14.280 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:10:14.281 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:14.281 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-09-13T09:10:14.281 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:14.281 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 143 k 2024-09-13T09:10:14.281 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 395 k 2024-09-13T09:10:14.281 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:10:14.499 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-immutable-object-cache-18.2.4-820.g48a7de7 655 kB/s | 143 kB 00:00 2024-09-13T09:10:14.500 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:14.500 INFO:teuthology.orchestra.run.smithi157.stdout:Total 649 kB/s | 143 kB 00:00 2024-09-13T09:10:14.500 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:10:14.508 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:10:14.508 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:10:14.555 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:54 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-13T09:10:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:14.559 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:10:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:14.559 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T09:10:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T09:10:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T09:10:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T09:10:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T09:10:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T09:10:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T09:10:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T09:10:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T09:10:14.560 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T09:10:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T09:10:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T09:10:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T09:10:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T09:10:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T09:10:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T09:10:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T09:10:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T09:10:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T09:10:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T09:10:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T09:10:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T09:10:14.561 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T09:10:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T09:10:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T09:10:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T09:10:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T09:10:14.562 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:14.562 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:10:14.565 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:10:14.565 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:10:14.645 INFO:teuthology.orchestra.run.smithi129.stdout:Package ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:14.686 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:14.687 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-13T09:10:14.687 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:14.712 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:10:14.760 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-dashboard 2024-09-13T09:10:14.792 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T09:10:14.817 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T09:10:14.818 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:14.818 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T09:10:14.818 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:14.976 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-base 2024-09-13T09:10:15.316 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:55 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:15.448 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:15.448 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:15.449 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-13T09:10:15.449 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:15.449 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:10:15.449 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 3.5 M 2024-09-13T09:10:15.449 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-13T09:10:15.449 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 24 k 2024-09-13T09:10:15.449 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 16 k 2024-09-13T09:10:15.449 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T09:10:15.449 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T09:10:15.449 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:15.449 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:10:15.449 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:15.450 INFO:teuthology.orchestra.run.smithi129.stdout:Install 5 Packages 2024-09-13T09:10:15.450 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:15.450 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 3.8 M 2024-09-13T09:10:15.450 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 87 M 2024-09-13T09:10:15.450 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:10:15.569 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:54 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:10:15.630 INFO:teuthology.orchestra.run.smithi129.stdout:(1/5): ceph-prometheus-alerts-18.2.4-820.g48a7d 108 kB/s | 16 kB 00:00 2024-09-13T09:10:15.655 INFO:teuthology.orchestra.run.smithi129.stdout:(2/5): ceph-grafana-dashboards-18.2.4-820.g48a7 137 kB/s | 24 kB 00:00 2024-09-13T09:10:15.665 INFO:teuthology.orchestra.run.smithi044.stdout:Package ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:15.689 INFO:teuthology.orchestra.run.smithi129.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 527 kB/s | 31 kB 00:00 2024-09-13T09:10:15.709 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:10:15.710 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-09-13T09:10:15.710 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:10:15.714 INFO:teuthology.orchestra.run.smithi129.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 3.1 MB/s | 188 kB 00:00 2024-09-13T09:10:15.732 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T09:10:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:15.733 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:10:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:15.733 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:15.733 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:15.776 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install cephadm 2024-09-13T09:10:15.873 INFO:teuthology.orchestra.run.smithi129.stdout:(5/5): ceph-mgr-dashboard-18.2.4-820.g48a7de7e. 8.9 MB/s | 3.5 MB 00:00 2024-09-13T09:10:15.876 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:15.876 INFO:teuthology.orchestra.run.smithi129.stdout:Total 8.9 MB/s | 3.8 MB 00:00 2024-09-13T09:10:15.876 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:10:15.883 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:10:15.883 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:10:15.943 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr 2024-09-13T09:10:15.968 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:10:15.968 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:10:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:10:16.312 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-13T09:10:16.352 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-13T09:10:16.365 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:55 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:10:16.381 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.no 3/5 2024-09-13T09:10:16.502 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:10:16.503 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:16.503 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-09-13T09:10:16.503 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:16.503 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:10:16.503 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 226 k 2024-09-13T09:10:16.503 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:16.503 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:10:16.503 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:16.503 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-09-13T09:10:16.503 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:16.503 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 226 k 2024-09-13T09:10:16.503 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 217 k 2024-09-13T09:10:16.503 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:10:16.513 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:54 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:16.606 INFO:teuthology.orchestra.run.smithi157.stdout:Package ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:16.649 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:16.650 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-09-13T09:10:16.650 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:16.711 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-dashboard 2024-09-13T09:10:16.738 INFO:teuthology.orchestra.run.smithi044.stdout:cephadm-18.2.4-820.g48a7de7e.el9.noarch.rpm 966 kB/s | 226 kB 00:00 2024-09-13T09:10:16.738 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:16.739 INFO:teuthology.orchestra.run.smithi044.stdout:Total 959 kB/s | 226 kB 00:00 2024-09-13T09:10:16.739 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:10:16.744 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:10:16.745 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:10:16.747 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:10:16.748 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:10:16.816 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:10:17.280 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:55 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:17.288 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T09:10:17.378 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T09:10:17.414 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:17.414 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:17.414 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-09-13T09:10:17.414 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:17.415 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:10:17.415 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 3.5 M 2024-09-13T09:10:17.415 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-09-13T09:10:17.415 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 24 k 2024-09-13T09:10:17.415 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 16 k 2024-09-13T09:10:17.415 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T09:10:17.415 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T09:10:17.415 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:17.415 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:10:17.415 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:17.415 INFO:teuthology.orchestra.run.smithi157.stdout:Install 5 Packages 2024-09-13T09:10:17.415 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:17.416 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 3.8 M 2024-09-13T09:10:17.416 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 87 M 2024-09-13T09:10:17.416 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:10:17.427 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.n 4/5 2024-09-13T09:10:17.460 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 5/5 2024-09-13T09:10:17.752 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T09:10:17.919 INFO:teuthology.orchestra.run.smithi157.stdout:(1/5): ceph-prometheus-alerts-18.2.4-820.g48a7d 101 kB/s | 16 kB 00:00 2024-09-13T09:10:17.945 INFO:teuthology.orchestra.run.smithi157.stdout:(2/5): ceph-grafana-dashboards-18.2.4-820.g48a7 129 kB/s | 24 kB 00:00 2024-09-13T09:10:17.984 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T09:10:17.984 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:17.984 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:10:17.984 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:17.984 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:17.984 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:10:17.986 INFO:teuthology.orchestra.run.smithi157.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 459 kB/s | 31 kB 00:00 2024-09-13T09:10:18.012 INFO:teuthology.orchestra.run.smithi157.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-13T09:10:18.137 INFO:teuthology.orchestra.run.smithi157.stdout:(5/5): ceph-mgr-dashboard-18.2.4-820.g48a7de7e. 9.3 MB/s | 3.5 MB 00:00 2024-09-13T09:10:18.141 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:18.141 INFO:teuthology.orchestra.run.smithi157.stdout:Total 5.2 MB/s | 3.8 MB 00:00 2024-09-13T09:10:18.141 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:10:18.148 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:10:18.148 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:10:18.234 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:10:18.234 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:10:18.273 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-immutable-object-cache 2024-09-13T09:10:18.490 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:10:18.533 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 5/5 2024-09-13T09:10:18.533 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.n 1/5 2024-09-13T09:10:18.533 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 2/5 2024-09-13T09:10:18.533 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.no 3/5 2024-09-13T09:10:18.533 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-13T09:10:18.601 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-13T09:10:18.633 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-13T09:10:18.662 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.no 3/5 2024-09-13T09:10:18.786 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-13T09:10:18.786 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:18.786 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:10:18.786 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:18.786 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:18.786 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:18.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T09:10:18.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T09:10:18.787 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:18.787 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:18.860 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:57 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:10:18.997 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:10:18.998 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:18.998 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repo Size 2024-09-13T09:10:18.999 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:18.999 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:10:18.999 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 143 k 2024-09-13T09:10:18.999 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:18.999 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:10:18.999 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:18.999 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-09-13T09:10:18.999 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:18.999 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 143 k 2024-09-13T09:10:18.999 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 395 k 2024-09-13T09:10:18.999 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:10:19.016 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-13T09:10:19.212 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-immutable-object-cache-18.2.4-820.g48a7de7 672 kB/s | 143 kB 00:00 2024-09-13T09:10:19.212 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:19.212 INFO:teuthology.orchestra.run.smithi044.stdout:Total 668 kB/s | 143 kB 00:00 2024-09-13T09:10:19.213 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:10:19.220 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:10:19.221 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:10:19.281 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:10:19.281 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:10:19.426 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:10:19.479 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T09:10:19.506 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T09:10:19.506 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:19.506 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T09:10:19.506 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:19.581 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:59 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:19.712 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.n 4/5 2024-09-13T09:10:19.714 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:19.715 INFO:teuthology.orchestra.run.smithi129.stdout:====================================================================================== 2024-09-13T09:10:19.715 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-13T09:10:19.715 INFO:teuthology.orchestra.run.smithi129.stdout:====================================================================================== 2024-09-13T09:10:19.716 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:10:19.716 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 7.4 M 2024-09-13T09:10:19.716 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-13T09:10:19.716 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T09:10:19.716 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T09:10:19.716 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T09:10:19.716 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T09:10:19.716 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T09:10:19.716 INFO:teuthology.orchestra.run.smithi129.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T09:10:19.716 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T09:10:19.716 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T09:10:19.716 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T09:10:19.716 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T09:10:19.717 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T09:10:19.717 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:19.717 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:10:19.717 INFO:teuthology.orchestra.run.smithi129.stdout:====================================================================================== 2024-09-13T09:10:19.717 INFO:teuthology.orchestra.run.smithi129.stdout:Install 12 Packages 2024-09-13T09:10:19.717 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:19.717 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 38 M 2024-09-13T09:10:19.718 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 212 M 2024-09-13T09:10:19.718 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:10:19.749 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 5/5 2024-09-13T09:10:20.147 INFO:teuthology.orchestra.run.smithi129.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-13T09:10:20.255 INFO:teuthology.orchestra.run.smithi129.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-09-13T09:10:20.289 INFO:teuthology.orchestra.run.smithi129.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.4 MB/s | 801 kB 00:00 2024-09-13T09:10:20.431 INFO:teuthology.orchestra.run.smithi129.stdout:(4/12): ceph-mgr-diskprediction-local-18.2.4-82 16 MB/s | 7.4 MB 00:00 2024-09-13T09:10:20.434 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T09:10:20.434 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:20.434 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:10:20.434 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:20.434 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:20.434 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:10:20.456 INFO:teuthology.orchestra.run.smithi129.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 109 kB/s | 18 kB 00:00 2024-09-13T09:10:20.573 INFO:teuthology.orchestra.run.smithi129.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.4 MB/s | 3.0 MB 00:00 2024-09-13T09:10:20.598 INFO:teuthology.orchestra.run.smithi129.stdout:(7/12): openblas-0.3.26-2.el9.x86_64.rpm 231 kB/s | 38 kB 00:00 2024-09-13T09:10:20.640 INFO:teuthology.orchestra.run.smithi129.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-13T09:10:20.642 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr 2024-09-13T09:10:20.716 INFO:teuthology.orchestra.run.smithi129.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 3.3 MB/s | 250 kB 00:00 2024-09-13T09:10:20.800 INFO:teuthology.orchestra.run.smithi129.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-09-13T09:10:20.810 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 5/5 2024-09-13T09:10:20.810 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.n 1/5 2024-09-13T09:10:20.810 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 2/5 2024-09-13T09:10:20.810 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.no 3/5 2024-09-13T09:10:20.810 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-13T09:10:20.875 INFO:teuthology.orchestra.run.smithi129.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 12 MB/s | 4.9 MB 00:00 2024-09-13T09:10:21.061 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-13T09:10:21.061 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:21.061 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:10:21.061 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:21.061 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:21.061 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:21.061 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T09:10:21.061 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T09:10:21.061 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:21.061 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:21.200 INFO:teuthology.orchestra.run.smithi129.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-13T09:10:21.201 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:21.201 INFO:teuthology.orchestra.run.smithi129.stdout:Total 26 MB/s | 38 MB 00:01 2024-09-13T09:10:21.233 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:00 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:10:21.280 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-13T09:10:21.327 INFO:teuthology.orchestra.run.smithi044.stdout:Package ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:21.372 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:10:21.373 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-09-13T09:10:21.373 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:10:21.442 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-dashboard 2024-09-13T09:10:21.556 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:10:21.582 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:10:21.582 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:10:21.842 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:59 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:21.976 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:21.977 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================== 2024-09-13T09:10:21.977 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-09-13T09:10:21.977 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================== 2024-09-13T09:10:21.977 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:10:21.977 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 7.4 M 2024-09-13T09:10:21.977 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-09-13T09:10:21.977 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T09:10:21.977 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T09:10:21.977 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T09:10:21.977 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T09:10:21.978 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T09:10:21.978 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T09:10:21.978 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T09:10:21.978 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T09:10:21.978 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T09:10:21.978 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T09:10:21.978 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T09:10:21.978 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:21.978 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:10:21.978 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================== 2024-09-13T09:10:21.978 INFO:teuthology.orchestra.run.smithi157.stdout:Install 12 Packages 2024-09-13T09:10:21.978 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:21.979 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 38 M 2024-09-13T09:10:21.979 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 212 M 2024-09-13T09:10:21.979 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:10:22.029 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:01 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:10:22.088 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:10:22.088 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:10:22.169 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:10:22.169 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:22.169 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-09-13T09:10:22.169 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:22.170 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:10:22.170 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 3.5 M 2024-09-13T09:10:22.170 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-09-13T09:10:22.170 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 24 k 2024-09-13T09:10:22.170 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 16 k 2024-09-13T09:10:22.170 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T09:10:22.170 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T09:10:22.170 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:22.170 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:10:22.170 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:22.170 INFO:teuthology.orchestra.run.smithi044.stdout:Install 5 Packages 2024-09-13T09:10:22.170 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:22.171 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 3.8 M 2024-09-13T09:10:22.171 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 87 M 2024-09-13T09:10:22.171 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:10:22.374 INFO:teuthology.orchestra.run.smithi044.stdout:(1/5): ceph-grafana-dashboards-18.2.4-820.g48a7 163 kB/s | 24 kB 00:00 2024-09-13T09:10:22.399 INFO:teuthology.orchestra.run.smithi044.stdout:(2/5): ceph-prometheus-alerts-18.2.4-820.g48a7d 94 kB/s | 16 kB 00:00 2024-09-13T09:10:22.441 INFO:teuthology.orchestra.run.smithi044.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 460 kB/s | 31 kB 00:00 2024-09-13T09:10:22.452 INFO:teuthology.orchestra.run.smithi157.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 879 kB/s | 191 kB 00:00 2024-09-13T09:10:22.466 INFO:teuthology.orchestra.run.smithi044.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-13T09:10:22.561 INFO:teuthology.orchestra.run.smithi157.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.4 MB/s | 801 kB 00:00 2024-09-13T09:10:22.611 INFO:teuthology.orchestra.run.smithi157.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 206 kB/s | 33 kB 00:00 2024-09-13T09:10:22.662 INFO:teuthology.orchestra.run.smithi157.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 363 kB/s | 18 kB 00:00 2024-09-13T09:10:22.667 INFO:teuthology.orchestra.run.smithi044.stdout:(5/5): ceph-mgr-dashboard-18.2.4-820.g48a7de7e. 8.0 MB/s | 3.5 MB 00:00 2024-09-13T09:10:22.670 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:22.670 INFO:teuthology.orchestra.run.smithi044.stdout:Total 7.6 MB/s | 3.8 MB 00:00 2024-09-13T09:10:22.671 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:10:22.679 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:10:22.679 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:10:22.761 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:10:22.761 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:10:22.787 INFO:teuthology.orchestra.run.smithi157.stdout:(5/12): ceph-mgr-diskprediction-local-18.2.4-82 13 MB/s | 7.4 MB 00:00 2024-09-13T09:10:22.813 INFO:teuthology.orchestra.run.smithi157.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 256 kB/s | 38 kB 00:00 2024-09-13T09:10:22.843 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:10:22.922 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-13T09:10:22.929 INFO:teuthology.orchestra.run.smithi157.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-13T09:10:22.965 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-13T09:10:23.021 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:10:23.065 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-13T09:10:23.071 INFO:teuthology.orchestra.run.smithi157.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.9 MB/s | 3.0 MB 00:00 2024-09-13T09:10:23.100 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-13T09:10:23.100 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-13T09:10:23.129 INFO:teuthology.orchestra.run.smithi157.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.2 MB/s | 250 kB 00:00 2024-09-13T09:10:23.130 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-13T09:10:23.159 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.no 3/5 2024-09-13T09:10:23.246 INFO:teuthology.orchestra.run.smithi157.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-09-13T09:10:23.284 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-13T09:10:23.322 INFO:teuthology.orchestra.run.smithi157.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 9.2 MB/s | 4.9 MB 00:00 2024-09-13T09:10:23.340 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-13T09:10:23.418 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-13T09:10:23.680 INFO:teuthology.orchestra.run.smithi157.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-09-13T09:10:23.681 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:23.681 INFO:teuthology.orchestra.run.smithi157.stdout:Total 22 MB/s | 38 MB 00:01 2024-09-13T09:10:24.038 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:10:24.064 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-13T09:10:24.064 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:10:24.064 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:10:24.178 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-13T09:10:24.271 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.n 4/5 2024-09-13T09:10:24.304 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 5/5 2024-09-13T09:10:24.575 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:10:24.575 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:10:25.338 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:10:25.386 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 5/5 2024-09-13T09:10:25.386 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.n 1/5 2024-09-13T09:10:25.386 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 2/5 2024-09-13T09:10:25.387 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.no 3/5 2024-09-13T09:10:25.387 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-13T09:10:25.420 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-13T09:10:25.428 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T09:10:25.486 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-13T09:10:25.594 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-13T09:10:25.629 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-13T09:10:25.650 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-13T09:10:25.650 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:25.650 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:10:25.650 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:25.650 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:25.650 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:25.650 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T09:10:25.650 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T09:10:25.650 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:25.651 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:10:25.816 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-13T09:10:25.845 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-13T09:10:25.885 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-13T09:10:25.923 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-13T09:10:26.445 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-13T09:10:26.466 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 12/12 2024-09-13T09:10:26.472 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:05 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:10:26.570 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-13T09:10:26.612 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:10:26.613 INFO:teuthology.orchestra.run.smithi044.stdout:====================================================================================== 2024-09-13T09:10:26.613 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-09-13T09:10:26.613 INFO:teuthology.orchestra.run.smithi044.stdout:====================================================================================== 2024-09-13T09:10:26.613 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:10:26.613 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 7.4 M 2024-09-13T09:10:26.613 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-09-13T09:10:26.613 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T09:10:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T09:10:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T09:10:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T09:10:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T09:10:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T09:10:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T09:10:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T09:10:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T09:10:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T09:10:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T09:10:26.614 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:26.614 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:10:26.614 INFO:teuthology.orchestra.run.smithi044.stdout:====================================================================================== 2024-09-13T09:10:26.614 INFO:teuthology.orchestra.run.smithi044.stdout:Install 12 Packages 2024-09-13T09:10:26.615 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:26.615 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 38 M 2024-09-13T09:10:26.615 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 212 M 2024-09-13T09:10:26.615 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:10:26.708 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-13T09:10:27.207 INFO:teuthology.orchestra.run.smithi044.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 914 kB/s | 191 kB 00:00 2024-09-13T09:10:27.316 INFO:teuthology.orchestra.run.smithi044.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-13T09:10:27.466 INFO:teuthology.orchestra.run.smithi044.stdout:(3/12): ceph-mgr-diskprediction-local-18.2.4-82 16 MB/s | 7.4 MB 00:00 2024-09-13T09:10:27.516 INFO:teuthology.orchestra.run.smithi044.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 106 kB/s | 33 kB 00:00 2024-09-13T09:10:27.716 INFO:teuthology.orchestra.run.smithi044.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 192 kB/s | 38 kB 00:00 2024-09-13T09:10:27.750 INFO:teuthology.orchestra.run.smithi044.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 64 kB/s | 18 kB 00:00 2024-09-13T09:10:27.971 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T09:10:28.209 INFO:teuthology.orchestra.run.smithi044.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 534 kB/s | 245 kB 00:00 2024-09-13T09:10:28.459 INFO:teuthology.orchestra.run.smithi044.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2024-09-13T09:10:28.576 INFO:teuthology.orchestra.run.smithi044.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-09-13T09:10:28.776 INFO:teuthology.orchestra.run.smithi044.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.7 MB/s | 4.9 MB 00:01 2024-09-13T09:10:28.934 INFO:teuthology.orchestra.run.smithi044.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.0 MB/s | 5.1 MB 00:00 2024-09-13T09:10:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 12/12 2024-09-13T09:10:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 1/12 2024-09-13T09:10:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-13T09:10:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-13T09:10:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-13T09:10:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-13T09:10:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-13T09:10:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-13T09:10:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-13T09:10:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-13T09:10:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T09:10:28.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-13T09:10:29.018 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-13T09:10:29.044 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 12/12 2024-09-13T09:10:29.310 INFO:teuthology.orchestra.run.smithi044.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-13T09:10:29.311 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:29.311 INFO:teuthology.orchestra.run.smithi044.stdout:Total 14 MB/s | 38 MB 00:02 2024-09-13T09:10:29.374 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-13T09:10:29.374 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:29.374 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:10:29.374 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:29.374 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T09:10:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T09:10:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T09:10:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T09:10:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T09:10:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T09:10:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T09:10:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T09:10:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T09:10:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T09:10:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T09:10:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:29.375 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:29.680 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:10:29.707 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:10:29.707 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:10:29.749 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-rook 2024-09-13T09:10:30.257 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:10:30.258 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:10:30.324 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:10 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:30.457 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:30.459 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:30.459 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-13T09:10:30.459 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:30.459 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:10:30.459 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 49 k 2024-09-13T09:10:30.459 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-13T09:10:30.459 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T09:10:30.459 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T09:10:30.459 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T09:10:30.459 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T09:10:30.459 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T09:10:30.459 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T09:10:30.460 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T09:10:30.460 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T09:10:30.460 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T09:10:30.460 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T09:10:30.460 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T09:10:30.460 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T09:10:30.460 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:30.460 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:10:30.460 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:30.460 INFO:teuthology.orchestra.run.smithi129.stdout:Install 13 Packages 2024-09-13T09:10:30.460 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:30.461 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 2.2 M 2024-09-13T09:10:30.461 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 26 M 2024-09-13T09:10:30.461 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:10:30.826 INFO:teuthology.orchestra.run.smithi129.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 341 kB/s | 26 kB 00:00 2024-09-13T09:10:30.851 INFO:teuthology.orchestra.run.smithi129.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 191 kB/s | 19 kB 00:00 2024-09-13T09:10:30.926 INFO:teuthology.orchestra.run.smithi129.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.9 MB/s | 222 kB 00:00 2024-09-13T09:10:30.952 INFO:teuthology.orchestra.run.smithi129.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-09-13T09:10:30.977 INFO:teuthology.orchestra.run.smithi129.stdout:(5/13): ceph-mgr-rook-18.2.4-820.g48a7de7e.el9. 218 kB/s | 49 kB 00:00 2024-09-13T09:10:31.002 INFO:teuthology.orchestra.run.smithi129.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-13T09:10:31.028 INFO:teuthology.orchestra.run.smithi129.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 279 kB 00:00 2024-09-13T09:10:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:10:31.078 INFO:teuthology.orchestra.run.smithi129.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 320 kB/s | 32 kB 00:00 2024-09-13T09:10:31.103 INFO:teuthology.orchestra.run.smithi129.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-13T09:10:31.127 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-13T09:10:31.128 INFO:teuthology.orchestra.run.smithi129.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-13T09:10:31.153 INFO:teuthology.orchestra.run.smithi129.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.9 MB/s | 220 kB 00:00 2024-09-13T09:10:31.168 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-13T09:10:31.179 INFO:teuthology.orchestra.run.smithi129.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-13T09:10:31.221 INFO:teuthology.orchestra.run.smithi129.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-13T09:10:31.224 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:31.224 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.9 MB/s | 2.2 MB 00:00 2024-09-13T09:10:31.272 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-13T09:10:31.288 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:10:31.303 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:10:31.303 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:10:31.318 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-13T09:10:31.391 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:10:31.391 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:10:31.515 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-13T09:10:31.550 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-13T09:10:31.643 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 12/12 2024-09-13T09:10:31.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 1/12 2024-09-13T09:10:31.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-13T09:10:31.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-13T09:10:31.643 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-13T09:10:31.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-13T09:10:31.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-13T09:10:31.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-13T09:10:31.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-13T09:10:31.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-13T09:10:31.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T09:10:31.644 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-13T09:10:31.664 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-13T09:10:31.896 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:10:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-13T09:10:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-13T09:10:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:32.073 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:10:32.073 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T09:10:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T09:10:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T09:10:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T09:10:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T09:10:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T09:10:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T09:10:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T09:10:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T09:10:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T09:10:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T09:10:32.074 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:32.074 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:32.113 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-13T09:10:32.169 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-13T09:10:32.214 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-13T09:10:32.256 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-13T09:10:32.356 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-13T09:10:32.359 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-13T09:10:32.439 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-rook 2024-09-13T09:10:32.469 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-13T09:10:32.484 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-13T09:10:32.532 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-13T09:10:33.021 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:11 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:33.154 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:33.155 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:33.155 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-09-13T09:10:33.155 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:33.155 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:10:33.155 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 49 k 2024-09-13T09:10:33.155 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-09-13T09:10:33.155 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T09:10:33.155 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T09:10:33.155 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T09:10:33.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T09:10:33.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T09:10:33.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T09:10:33.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T09:10:33.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T09:10:33.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T09:10:33.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T09:10:33.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T09:10:33.156 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T09:10:33.156 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:33.156 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:10:33.156 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:33.157 INFO:teuthology.orchestra.run.smithi157.stdout:Install 13 Packages 2024-09-13T09:10:33.157 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:33.157 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 2.2 M 2024-09-13T09:10:33.157 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 26 M 2024-09-13T09:10:33.157 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:10:33.242 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-13T09:10:33.289 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-13T09:10:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-13T09:10:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-13T09:10:33.401 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 13/13 2024-09-13T09:10:33.460 INFO:teuthology.orchestra.run.smithi157.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 186 kB/s | 19 kB 00:00 2024-09-13T09:10:33.485 INFO:teuthology.orchestra.run.smithi157.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 200 kB/s | 26 kB 00:00 2024-09-13T09:10:33.544 INFO:teuthology.orchestra.run.smithi157.stdout:(3/13): ceph-mgr-rook-18.2.4-820.g48a7de7e.el9. 264 kB/s | 49 kB 00:00 2024-09-13T09:10:33.569 INFO:teuthology.orchestra.run.smithi157.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-13T09:10:33.594 INFO:teuthology.orchestra.run.smithi157.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-13T09:10:33.636 INFO:teuthology.orchestra.run.smithi157.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.0 MB/s | 279 kB 00:00 2024-09-13T09:10:33.661 INFO:teuthology.orchestra.run.smithi157.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-13T09:10:33.695 INFO:teuthology.orchestra.run.smithi157.stdout:(8/13): python3-requests-oauthlib-1.3.0-12.el9. 427 kB/s | 54 kB 00:00 2024-09-13T09:10:33.720 INFO:teuthology.orchestra.run.smithi157.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 168 kB/s | 14 kB 00:00 2024-09-13T09:10:33.754 INFO:teuthology.orchestra.run.smithi157.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.5 MB/s | 220 kB 00:00 2024-09-13T09:10:33.779 INFO:teuthology.orchestra.run.smithi157.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-13T09:10:33.804 INFO:teuthology.orchestra.run.smithi157.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-13T09:10:33.822 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T09:10:33.846 INFO:teuthology.orchestra.run.smithi157.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-13T09:10:33.849 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:33.849 INFO:teuthology.orchestra.run.smithi157.stdout:Total 3.2 MB/s | 2.2 MB 00:00 2024-09-13T09:10:33.912 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:10:33.928 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:10:33.928 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:10:34.017 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:10:34.019 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:10:34.191 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 13/13 2024-09-13T09:10:34.191 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 1/13 2024-09-13T09:10:34.192 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-13T09:10:34.192 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-13T09:10:34.192 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-13T09:10:34.192 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-13T09:10:34.192 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-13T09:10:34.192 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-13T09:10:34.192 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-13T09:10:34.192 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-13T09:10:34.192 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-13T09:10:34.192 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-13T09:10:34.192 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-13T09:10:34.521 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:10:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-13T09:10:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:34.546 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:10:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T09:10:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T09:10:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T09:10:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T09:10:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T09:10:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T09:10:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T09:10:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T09:10:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T09:10:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T09:10:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T09:10:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T09:10:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:34.547 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:34.701 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-13T09:10:34.756 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-13T09:10:34.811 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-13T09:10:34.832 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-cephadm 2024-09-13T09:10:34.856 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-13T09:10:34.898 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-13T09:10:34.911 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-13T09:10:34.932 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 12/12 2024-09-13T09:10:34.999 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-13T09:10:35.109 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-13T09:10:35.166 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-13T09:10:35.417 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:15 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:35.551 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:35.552 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:35.552 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-13T09:10:35.552 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:35.552 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:10:35.552 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 139 k 2024-09-13T09:10:35.552 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-13T09:10:35.552 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T09:10:35.552 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T09:10:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T09:10:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T09:10:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T09:10:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:35.553 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:10:35.553 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:35.553 INFO:teuthology.orchestra.run.smithi129.stdout:Install 6 Packages 2024-09-13T09:10:35.553 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:35.554 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 7.0 M 2024-09-13T09:10:35.554 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 33 M 2024-09-13T09:10:35.554 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:10:35.880 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-13T09:10:35.930 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-13T09:10:35.964 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-13T09:10:36.012 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-13T09:10:36.042 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 13/13 2024-09-13T09:10:36.100 INFO:teuthology.orchestra.run.smithi129.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.8 MB/s | 248 kB 00:00 2024-09-13T09:10:36.150 INFO:teuthology.orchestra.run.smithi129.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-13T09:10:36.175 INFO:teuthology.orchestra.run.smithi129.stdout:(3/6): ceph-mgr-cephadm-18.2.4-820.g48a7de7e.el 668 kB/s | 139 kB 00:00 2024-09-13T09:10:36.201 INFO:teuthology.orchestra.run.smithi129.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-13T09:10:36.242 INFO:teuthology.orchestra.run.smithi129.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-13T09:10:36.320 INFO:teuthology.orchestra.run.smithi129.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-13T09:10:36.321 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:36.321 INFO:teuthology.orchestra.run.smithi129.stdout:Total 9.1 MB/s | 7.0 MB 00:00 2024-09-13T09:10:36.391 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:10:36.409 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:10:36.409 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:10:36.500 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:10:36.502 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:10:36.822 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:10:36.831 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 13/13 2024-09-13T09:10:36.832 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 1/13 2024-09-13T09:10:36.832 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-13T09:10:36.832 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-13T09:10:36.832 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-13T09:10:36.832 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-13T09:10:36.832 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-13T09:10:36.832 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-13T09:10:36.832 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-13T09:10:36.832 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-13T09:10:36.832 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-13T09:10:36.832 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-13T09:10:36.832 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-13T09:10:36.984 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-13T09:10:37.036 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-13T09:10:37.168 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-13T09:10:37.168 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:37.168 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:10:37.168 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:37.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T09:10:37.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T09:10:37.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T09:10:37.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T09:10:37.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T09:10:37.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T09:10:37.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T09:10:37.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T09:10:37.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T09:10:37.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T09:10:37.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T09:10:37.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T09:10:37.169 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:37.169 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:37.402 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 12/12 2024-09-13T09:10:37.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 1/12 2024-09-13T09:10:37.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-13T09:10:37.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-13T09:10:37.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-13T09:10:37.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-13T09:10:37.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-13T09:10:37.402 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-13T09:10:37.403 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-13T09:10:37.403 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-13T09:10:37.403 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T09:10:37.403 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-13T09:10:37.413 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-cephadm 2024-09-13T09:10:37.560 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-13T09:10:37.639 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-13T09:10:37.710 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-13T09:10:37.741 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 6/6 2024-09-13T09:10:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-13T09:10:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:37.818 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:10:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T09:10:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T09:10:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T09:10:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T09:10:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T09:10:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T09:10:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T09:10:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T09:10:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T09:10:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T09:10:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T09:10:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:37.819 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:10:37.996 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:15 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:38.129 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:38.130 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:38.130 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-09-13T09:10:38.130 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:38.131 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:10:38.131 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 139 k 2024-09-13T09:10:38.131 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-09-13T09:10:38.131 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T09:10:38.131 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T09:10:38.131 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T09:10:38.131 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T09:10:38.131 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T09:10:38.131 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:38.131 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:10:38.131 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:38.131 INFO:teuthology.orchestra.run.smithi157.stdout:Install 6 Packages 2024-09-13T09:10:38.131 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:38.132 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 7.0 M 2024-09-13T09:10:38.132 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 33 M 2024-09-13T09:10:38.132 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:10:38.188 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-rook 2024-09-13T09:10:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 6/6 2024-09-13T09:10:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/6 2024-09-13T09:10:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-13T09:10:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-13T09:10:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-13T09:10:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-13T09:10:38.591 INFO:teuthology.orchestra.run.smithi157.stdout:(1/6): ceph-mgr-cephadm-18.2.4-820.g48a7de7e.el 719 kB/s | 139 kB 00:00 2024-09-13T09:10:38.649 INFO:teuthology.orchestra.run.smithi157.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-13T09:10:38.683 INFO:teuthology.orchestra.run.smithi157.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.1 MB/s | 58 kB 00:00 2024-09-13T09:10:38.708 INFO:teuthology.orchestra.run.smithi157.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-13T09:10:38.760 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-13T09:10:38.760 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:38.760 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:10:38.760 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:38.760 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T09:10:38.760 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T09:10:38.760 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T09:10:38.760 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T09:10:38.760 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T09:10:38.761 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:38.761 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:38.783 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:17 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:10:38.923 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:10:38.924 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-09-13T09:10:38.924 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:38.924 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:10:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 49 k 2024-09-13T09:10:38.924 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-09-13T09:10:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T09:10:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T09:10:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T09:10:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T09:10:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T09:10:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T09:10:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T09:10:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T09:10:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T09:10:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T09:10:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T09:10:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T09:10:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:38.926 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:10:38.926 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:38.926 INFO:teuthology.orchestra.run.smithi044.stdout:Install 13 Packages 2024-09-13T09:10:38.926 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:38.926 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 2.2 M 2024-09-13T09:10:38.926 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 26 M 2024-09-13T09:10:38.926 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:10:39.002 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-fuse 2024-09-13T09:10:39.200 INFO:teuthology.orchestra.run.smithi157.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 309 kB/s | 248 kB 00:00 2024-09-13T09:10:39.232 INFO:teuthology.orchestra.run.smithi044.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 274 kB/s | 26 kB 00:00 2024-09-13T09:10:39.298 INFO:teuthology.orchestra.run.smithi044.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 119 kB/s | 19 kB 00:00 2024-09-13T09:10:39.324 INFO:teuthology.orchestra.run.smithi044.stdout:(3/13): ceph-mgr-rook-18.2.4-820.g48a7de7e.el9. 265 kB/s | 49 kB 00:00 2024-09-13T09:10:39.349 INFO:teuthology.orchestra.run.smithi044.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-09-13T09:10:39.408 INFO:teuthology.orchestra.run.smithi044.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-13T09:10:39.433 INFO:teuthology.orchestra.run.smithi044.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 640 kB/s | 54 kB 00:00 2024-09-13T09:10:39.475 INFO:teuthology.orchestra.run.smithi044.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-13T09:10:39.500 INFO:teuthology.orchestra.run.smithi044.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 279 kB 00:00 2024-09-13T09:10:39.525 INFO:teuthology.orchestra.run.smithi044.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 153 kB/s | 14 kB 00:00 2024-09-13T09:10:39.559 INFO:teuthology.orchestra.run.smithi044.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.2 MB/s | 59 kB 00:00 2024-09-13T09:10:39.569 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:19 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:39.584 INFO:teuthology.orchestra.run.smithi044.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 2.0 MB/s | 220 kB 00:00 2024-09-13T09:10:39.609 INFO:teuthology.orchestra.run.smithi044.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-13T09:10:39.651 INFO:teuthology.orchestra.run.smithi044.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-13T09:10:39.654 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:39.654 INFO:teuthology.orchestra.run.smithi044.stdout:Total 3.0 MB/s | 2.2 MB 00:00 2024-09-13T09:10:39.675 INFO:teuthology.orchestra.run.smithi157.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.7 MB/s | 6.0 MB 00:01 2024-09-13T09:10:39.677 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:39.677 INFO:teuthology.orchestra.run.smithi157.stdout:Total 4.5 MB/s | 7.0 MB 00:01 2024-09-13T09:10:39.704 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:39.704 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:39.704 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-13T09:10:39.704 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:39.704 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:10:39.704 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 852 k 2024-09-13T09:10:39.704 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-13T09:10:39.705 INFO:teuthology.orchestra.run.smithi129.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T09:10:39.705 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:39.705 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:10:39.705 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:39.705 INFO:teuthology.orchestra.run.smithi129.stdout:Install 2 Packages 2024-09-13T09:10:39.705 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:39.705 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 932 k 2024-09-13T09:10:39.706 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 2.7 M 2024-09-13T09:10:39.706 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:10:39.722 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:10:39.738 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:10:39.738 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:10:39.748 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:10:39.759 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:10:39.759 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:10:39.836 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:10:39.837 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:10:39.858 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:10:39.858 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:10:40.124 INFO:teuthology.orchestra.run.smithi129.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 488 kB/s | 80 kB 00:00 2024-09-13T09:10:40.160 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:10:40.324 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-13T09:10:40.356 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:10:40.377 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-13T09:10:40.534 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-13T09:10:40.579 INFO:teuthology.orchestra.run.smithi129.stdout:(2/2): ceph-fuse-18.2.4-820.g48a7de7e.el9.x86_6 1.3 MB/s | 852 kB 00:00 2024-09-13T09:10:40.580 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:40.580 INFO:teuthology.orchestra.run.smithi129.stdout:Total 1.0 MB/s | 932 kB 00:00 2024-09-13T09:10:40.587 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-13T09:10:40.592 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:10:40.614 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:10:40.614 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:10:40.643 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-13T09:10:40.679 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-13T09:10:40.696 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:10:40.696 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:10:40.721 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-13T09:10:40.828 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-13T09:10:40.888 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:10:40.912 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-13T09:10:40.939 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-13T09:10:40.996 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-13T09:10:40.998 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-13T09:10:41.019 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-13T09:10:41.058 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-13T09:10:41.095 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2/2 2024-09-13T09:10:41.097 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 6/6 2024-09-13T09:10:41.758 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-13T09:10:41.788 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2/2 2024-09-13T09:10:41.788 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/2 2024-09-13T09:10:41.803 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-13T09:10:41.836 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-13T09:10:41.872 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 6/6 2024-09-13T09:10:41.873 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/6 2024-09-13T09:10:41.873 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-13T09:10:41.873 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-13T09:10:41.873 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-13T09:10:41.873 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-13T09:10:41.885 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-13T09:10:41.914 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 13/13 2024-09-13T09:10:42.043 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-13T09:10:42.043 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:42.043 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:10:42.043 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-13T09:10:42.043 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:42.043 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:42.136 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-13T09:10:42.136 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:42.136 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:10:42.136 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:42.136 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T09:10:42.136 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T09:10:42.137 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T09:10:42.137 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T09:10:42.137 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T09:10:42.137 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:42.137 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:42.316 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-volume 2024-09-13T09:10:42.372 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-fuse 2024-09-13T09:10:42.786 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 13/13 2024-09-13T09:10:42.786 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 1/13 2024-09-13T09:10:42.786 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-13T09:10:42.786 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-13T09:10:42.786 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-13T09:10:42.786 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-13T09:10:42.786 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-13T09:10:42.786 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-13T09:10:42.786 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-13T09:10:42.787 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-13T09:10:42.787 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-13T09:10:42.787 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-13T09:10:42.787 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-13T09:10:42.898 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:22 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:42.939 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:20 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:43.038 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:43.038 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:43.038 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-13T09:10:43.038 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:43.039 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:10:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 264 k 2024-09-13T09:10:43.039 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-13T09:10:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T09:10:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:43.039 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:10:43.039 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:43.039 INFO:teuthology.orchestra.run.smithi129.stdout:Install 2 Packages 2024-09-13T09:10:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:43.040 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 342 k 2024-09-13T09:10:43.040 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 1.5 M 2024-09-13T09:10:43.040 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:10:43.072 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:43.073 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:43.073 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-09-13T09:10:43.073 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:43.074 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:10:43.074 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 852 k 2024-09-13T09:10:43.074 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-09-13T09:10:43.074 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T09:10:43.074 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:43.074 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:10:43.074 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:43.074 INFO:teuthology.orchestra.run.smithi157.stdout:Install 2 Packages 2024-09-13T09:10:43.074 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:43.074 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 932 k 2024-09-13T09:10:43.074 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 2.7 M 2024-09-13T09:10:43.074 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:10:43.124 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-13T09:10:43.124 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:43.124 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:10:43.124 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:43.124 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T09:10:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T09:10:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T09:10:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T09:10:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T09:10:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T09:10:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T09:10:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T09:10:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T09:10:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T09:10:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T09:10:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T09:10:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:43.126 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:10:43.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-cephadm 2024-09-13T09:10:43.601 INFO:teuthology.orchestra.run.smithi157.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 168 kB/s | 80 kB 00:00 2024-09-13T09:10:43.643 INFO:teuthology.orchestra.run.smithi129.stdout:(1/2): ceph-volume-18.2.4-820.g48a7de7e.el9.noa 454 kB/s | 264 kB 00:00 2024-09-13T09:10:43.694 INFO:teuthology.orchestra.run.smithi129.stdout:(2/2): python3-packaging-20.9-5.el9.noarch.rpm 122 kB/s | 77 kB 00:00 2024-09-13T09:10:43.694 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:43.695 INFO:teuthology.orchestra.run.smithi129.stdout:Total 521 kB/s | 342 kB 00:00 2024-09-13T09:10:43.706 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:10:43.716 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:10:43.716 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:10:43.750 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:10:43.750 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:10:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:10:43.999 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:22 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:10:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-13T09:10:44.122 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2/2 2024-09-13T09:10:44.137 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2/2 2024-09-13T09:10:44.137 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:44.137 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T09:10:44.137 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:44.140 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:10:44.141 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:44.141 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-09-13T09:10:44.141 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:44.141 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:10:44.141 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 139 k 2024-09-13T09:10:44.141 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-09-13T09:10:44.141 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T09:10:44.141 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T09:10:44.141 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T09:10:44.141 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T09:10:44.141 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T09:10:44.141 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:44.142 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:10:44.142 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:44.142 INFO:teuthology.orchestra.run.smithi044.stdout:Install 6 Packages 2024-09-13T09:10:44.142 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:44.142 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 7.0 M 2024-09-13T09:10:44.142 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 33 M 2024-09-13T09:10:44.142 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:10:44.201 INFO:teuthology.orchestra.run.smithi157.stdout:(2/2): ceph-fuse-18.2.4-820.g48a7de7e.el9.x86_6 792 kB/s | 852 kB 00:01 2024-09-13T09:10:44.202 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:44.202 INFO:teuthology.orchestra.run.smithi157.stdout:Total 825 kB/s | 932 kB 00:01 2024-09-13T09:10:44.216 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:10:44.240 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:10:44.240 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:10:44.323 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:10:44.323 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:10:44.495 INFO:teuthology.orchestra.run.smithi044.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-09-13T09:10:44.510 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:10:44.545 INFO:teuthology.orchestra.run.smithi044.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-13T09:10:44.570 INFO:teuthology.orchestra.run.smithi044.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-13T09:10:44.595 INFO:teuthology.orchestra.run.smithi044.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-13T09:10:44.641 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-13T09:10:44.696 INFO:teuthology.orchestra.run.smithi044.stdout:(5/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-13T09:10:44.707 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2/2 2024-09-13T09:10:44.821 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 1/2 2024-09-13T09:10:44.913 INFO:teuthology.orchestra.run.smithi044.stdout:(6/6): ceph-mgr-cephadm-18.2.4-820.g48a7de7e.el 232 kB/s | 139 kB 00:00 2024-09-13T09:10:44.914 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:44.915 INFO:teuthology.orchestra.run.smithi044.stdout:Total 9.1 MB/s | 7.0 MB 00:00 2024-09-13T09:10:44.988 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:10:45.000 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:10:45.000 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:10:45.105 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:10:45.105 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:10:45.138 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-13T09:10:45.138 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:45.138 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:10:45.138 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:45.138 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T09:10:45.138 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:45.138 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:45.360 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2/2 2024-09-13T09:10:45.360 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/2 2024-09-13T09:10:45.426 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install librados-devel 2024-09-13T09:10:45.556 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:10:45.648 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-13T09:10:45.648 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:45.648 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:10:45.649 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-13T09:10:45.649 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:45.649 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:45.737 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-13T09:10:45.858 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-13T09:10:45.934 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-volume 2024-09-13T09:10:46.007 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:26 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:46.143 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:46.144 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:46.144 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-13T09:10:46.144 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:46.144 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:10:46.144 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 127 k 2024-09-13T09:10:46.144 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:46.144 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:10:46.145 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:46.145 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-13T09:10:46.145 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:46.145 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 127 k 2024-09-13T09:10:46.145 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 456 k 2024-09-13T09:10:46.145 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:10:46.376 INFO:teuthology.orchestra.run.smithi129.stdout:librados-devel-18.2.4-820.g48a7de7e.el9.x86_64. 549 kB/s | 127 kB 00:00 2024-09-13T09:10:46.377 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:46.377 INFO:teuthology.orchestra.run.smithi129.stdout:Total 546 kB/s | 127 kB 00:00 2024-09-13T09:10:46.377 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:10:46.382 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:10:46.382 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:10:46.416 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-13T09:10:46.445 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:10:46.445 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:10:46.512 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-13T09:10:46.516 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:24 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:46.582 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-13T09:10:46.595 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:10:46.610 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 6/6 2024-09-13T09:10:46.651 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:46.652 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-09-13T09:10:46.652 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:46.652 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:10:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 264 k 2024-09-13T09:10:46.652 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-09-13T09:10:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T09:10:46.652 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:46.652 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:10:46.653 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:46.653 INFO:teuthology.orchestra.run.smithi157.stdout:Install 2 Packages 2024-09-13T09:10:46.653 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:46.653 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 342 k 2024-09-13T09:10:46.653 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 1.5 M 2024-09-13T09:10:46.653 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:10:46.689 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:46.855 INFO:teuthology.orchestra.run.smithi157.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 496 kB/s | 77 kB 00:00 2024-09-13T09:10:46.947 INFO:teuthology.orchestra.run.smithi157.stdout:(2/2): ceph-volume-18.2.4-820.g48a7de7e.el9.noa 1.0 MB/s | 264 kB 00:00 2024-09-13T09:10:46.948 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:46.948 INFO:teuthology.orchestra.run.smithi157.stdout:Total 1.1 MB/s | 342 kB 00:00 2024-09-13T09:10:46.960 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:10:46.970 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:10:46.970 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:10:47.006 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:10:47.006 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:10:47.120 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:47.156 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:10:47.327 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-13T09:10:47.367 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2/2 2024-09-13T09:10:47.375 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 6/6 2024-09-13T09:10:47.375 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/6 2024-09-13T09:10:47.375 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-13T09:10:47.375 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-13T09:10:47.375 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-13T09:10:47.376 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-13T09:10:47.383 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2/2 2024-09-13T09:10:47.383 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:47.383 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T09:10:47.383 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:47.385 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:47.385 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:47.385 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:10:47.385 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:47.385 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:47.385 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:47.632 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install libcephfs2 2024-09-13T09:10:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-13T09:10:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:47.665 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:10:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T09:10:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T09:10:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T09:10:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T09:10:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T09:10:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:47.666 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:10:47.918 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-fuse 2024-09-13T09:10:48.119 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 1/2 2024-09-13T09:10:48.214 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:28 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:48.308 INFO:teuthology.orchestra.run.smithi129.stdout:Package libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:48.353 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:48.354 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-13T09:10:48.354 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:48.425 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-13T09:10:48.425 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:48.425 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:10:48.425 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:48.425 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T09:10:48.425 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:48.426 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:48.431 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install libcephfs-devel 2024-09-13T09:10:48.512 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:27 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:10:48.654 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:10:48.655 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:48.655 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-09-13T09:10:48.655 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:48.655 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:10:48.655 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 852 k 2024-09-13T09:10:48.655 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-09-13T09:10:48.655 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T09:10:48.656 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:48.656 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:10:48.656 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:48.656 INFO:teuthology.orchestra.run.smithi044.stdout:Install 2 Packages 2024-09-13T09:10:48.656 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:48.656 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 932 k 2024-09-13T09:10:48.656 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 2.7 M 2024-09-13T09:10:48.657 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:10:48.728 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install librados-devel 2024-09-13T09:10:48.931 INFO:teuthology.orchestra.run.smithi044.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 523 kB/s | 80 kB 00:00 2024-09-13T09:10:48.995 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:28 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:49.073 INFO:teuthology.orchestra.run.smithi044.stdout:(2/2): ceph-fuse-18.2.4-820.g48a7de7e.el9.x86_6 2.8 MB/s | 852 kB 00:00 2024-09-13T09:10:49.073 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:49.074 INFO:teuthology.orchestra.run.smithi044.stdout:Total 2.2 MB/s | 932 kB 00:00 2024-09-13T09:10:49.086 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:10:49.109 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:10:49.109 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:10:49.127 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:49.128 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-13T09:10:49.128 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:49.128 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:10:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 31 k 2024-09-13T09:10:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:49.128 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:10:49.128 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:49.128 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-13T09:10:49.128 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:49.129 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 31 k 2024-09-13T09:10:49.129 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 139 k 2024-09-13T09:10:49.129 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:10:49.194 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:10:49.195 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:10:49.272 INFO:teuthology.orchestra.run.smithi129.stdout:libcephfs-devel-18.2.4-820.g48a7de7e.el9.x86_64 217 kB/s | 31 kB 00:00 2024-09-13T09:10:49.273 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:49.273 INFO:teuthology.orchestra.run.smithi129.stdout:Total 215 kB/s | 31 kB 00:00 2024-09-13T09:10:49.273 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:10:49.276 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:10:49.276 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:10:49.294 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:10:49.294 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:10:49.300 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:27 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:49.377 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:10:49.389 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:10:49.441 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:49.441 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:49.442 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2024-09-13T09:10:49.442 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:49.442 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:10:49.442 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 127 k 2024-09-13T09:10:49.442 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:49.442 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:10:49.442 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:49.442 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-09-13T09:10:49.442 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:49.442 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 127 k 2024-09-13T09:10:49.442 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 456 k 2024-09-13T09:10:49.442 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:10:49.501 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:49.534 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-13T09:10:49.616 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2/2 2024-09-13T09:10:49.653 INFO:teuthology.orchestra.run.smithi157.stdout:librados-devel-18.2.4-820.g48a7de7e.el9.x86_64. 600 kB/s | 127 kB 00:00 2024-09-13T09:10:49.653 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:49.654 INFO:teuthology.orchestra.run.smithi157.stdout:Total 595 kB/s | 127 kB 00:00 2024-09-13T09:10:49.654 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:10:49.659 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:10:49.659 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:10:49.722 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:10:49.723 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:10:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:49.875 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:10:49.975 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:50.092 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:10:50.092 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:50.093 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:50.093 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:50.278 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install librados2 2024-09-13T09:10:50.308 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2/2 2024-09-13T09:10:50.308 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/2 2024-09-13T09:10:50.419 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:50.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-13T09:10:50.575 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:50.575 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:10:50.576 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-13T09:10:50.576 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:50.576 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:10:50.677 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:50.677 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:50.677 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:10:50.677 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:50.677 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:50.677 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:50.850 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:30 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:50.871 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-volume 2024-09-13T09:10:50.936 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install libcephfs2 2024-09-13T09:10:50.940 INFO:teuthology.orchestra.run.smithi129.stdout:Package librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:50.982 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:50.984 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-13T09:10:50.984 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:51.053 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install librbd1 2024-09-13T09:10:51.469 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:30 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:10:51.510 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:29 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:51.603 INFO:teuthology.orchestra.run.smithi157.stdout:Package libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:51.608 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:10:51.609 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:51.609 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-09-13T09:10:51.609 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:51.609 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:10:51.609 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 264 k 2024-09-13T09:10:51.609 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-09-13T09:10:51.609 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T09:10:51.609 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:51.609 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:10:51.609 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:51.609 INFO:teuthology.orchestra.run.smithi044.stdout:Install 2 Packages 2024-09-13T09:10:51.610 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:51.610 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 342 k 2024-09-13T09:10:51.610 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 1.5 M 2024-09-13T09:10:51.610 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:10:51.622 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:31 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:51.645 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:51.646 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-09-13T09:10:51.646 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:51.713 INFO:teuthology.orchestra.run.smithi129.stdout:Package librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:51.717 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install libcephfs-devel 2024-09-13T09:10:51.756 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:51.757 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-13T09:10:51.757 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:51.832 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-rados 2024-09-13T09:10:51.894 INFO:teuthology.orchestra.run.smithi044.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 515 kB/s | 77 kB 00:00 2024-09-13T09:10:51.994 INFO:teuthology.orchestra.run.smithi044.stdout:(2/2): ceph-volume-18.2.4-820.g48a7de7e.el9.noa 1.0 MB/s | 264 kB 00:00 2024-09-13T09:10:51.994 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:51.995 INFO:teuthology.orchestra.run.smithi044.stdout:Total 888 kB/s | 342 kB 00:00 2024-09-13T09:10:52.007 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:10:52.017 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:10:52.018 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:10:52.054 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:10:52.054 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:10:52.207 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:10:52.278 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:30 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:52.397 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:32 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:52.402 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-13T09:10:52.410 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:52.411 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:52.411 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2024-09-13T09:10:52.411 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:52.411 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:10:52.411 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 31 k 2024-09-13T09:10:52.412 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:52.412 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:10:52.412 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:52.412 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-09-13T09:10:52.412 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:52.412 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 31 k 2024-09-13T09:10:52.412 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 139 k 2024-09-13T09:10:52.412 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:10:52.438 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2/2 2024-09-13T09:10:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2/2 2024-09-13T09:10:52.455 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:52.455 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T09:10:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:52.486 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:52.529 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:52.530 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-13T09:10:52.530 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:52.556 INFO:teuthology.orchestra.run.smithi157.stdout:libcephfs-devel-18.2.4-820.g48a7de7e.el9.x86_64 216 kB/s | 31 kB 00:00 2024-09-13T09:10:52.557 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:52.557 INFO:teuthology.orchestra.run.smithi157.stdout:Total 214 kB/s | 31 kB 00:00 2024-09-13T09:10:52.557 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:10:52.560 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:10:52.560 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:10:52.582 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:10:52.583 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:10:52.616 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-rgw 2024-09-13T09:10:52.665 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:10:52.786 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:53.151 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:53.159 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 1/2 2024-09-13T09:10:53.189 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:33 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:53.281 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:53.324 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:53.324 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-13T09:10:53.325 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:53.380 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:53.381 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:53.381 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:10:53.381 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:53.381 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:53.381 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:53.407 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-cephfs 2024-09-13T09:10:53.464 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-13T09:10:53.464 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:53.464 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:10:53.464 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T09:10:53.464 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T09:10:53.465 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:53.465 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:10:53.553 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install librados2 2024-09-13T09:10:53.758 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install librados-devel 2024-09-13T09:10:53.970 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:33 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:54.060 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:54.103 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:54.104 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-13T09:10:54.104 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:54.117 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:32 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:54.182 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-rbd 2024-09-13T09:10:54.208 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:54.251 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:54.251 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-09-13T09:10:54.252 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:54.322 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install librbd1 2024-09-13T09:10:54.348 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:33 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:10:54.487 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:10:54.488 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:54.488 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repo Size 2024-09-13T09:10:54.488 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:54.488 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:10:54.488 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 127 k 2024-09-13T09:10:54.488 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:54.488 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:10:54.488 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:54.488 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-09-13T09:10:54.489 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:54.489 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 127 k 2024-09-13T09:10:54.489 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 456 k 2024-09-13T09:10:54.489 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:10:54.674 INFO:teuthology.orchestra.run.smithi044.stdout:librados-devel-18.2.4-820.g48a7de7e.el9.x86_64. 683 kB/s | 127 kB 00:00 2024-09-13T09:10:54.675 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:54.675 INFO:teuthology.orchestra.run.smithi044.stdout:Total 677 kB/s | 127 kB 00:00 2024-09-13T09:10:54.676 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:10:54.681 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:10:54.681 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:10:54.748 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:10:54.748 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:10:54.749 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:34 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:54.841 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:54.884 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:54.885 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-13T09:10:54.885 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:54.886 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:32 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:54.903 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:10:54.957 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-fuse 2024-09-13T09:10:54.976 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:55.018 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:55.019 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-09-13T09:10:55.019 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:55.033 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:55.088 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-rados 2024-09-13T09:10:55.480 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:55.528 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:35 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:55.654 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:33 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:55.660 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:55.661 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:55.661 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-13T09:10:55.661 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:55.661 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:10:55.661 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 87 k 2024-09-13T09:10:55.661 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:55.661 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:10:55.661 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:55.662 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-13T09:10:55.662 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:55.662 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 87 k 2024-09-13T09:10:55.662 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 227 k 2024-09-13T09:10:55.662 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:10:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:55.733 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:10:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:55.733 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:10:55.743 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:55.786 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:55.787 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-09-13T09:10:55.787 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:55.854 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-rgw 2024-09-13T09:10:55.862 INFO:teuthology.orchestra.run.smithi129.stdout:rbd-fuse-18.2.4-820.g48a7de7e.el9.x86_64.rpm 435 kB/s | 87 kB 00:00 2024-09-13T09:10:55.862 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:55.863 INFO:teuthology.orchestra.run.smithi129.stdout:Total 432 kB/s | 87 kB 00:00 2024-09-13T09:10:55.863 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:10:55.870 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:10:55.870 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:10:55.933 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:10:55.933 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:10:55.986 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install libcephfs2 2024-09-13T09:10:56.080 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:10:56.189 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:56.414 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:34 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:56.503 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:56.546 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:56.547 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-09-13T09:10:56.547 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:56.562 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:56.577 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:35 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:10:56.615 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-cephfs 2024-09-13T09:10:56.673 INFO:teuthology.orchestra.run.smithi044.stdout:Package libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:56.720 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:10:56.721 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-09-13T09:10:56.721 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:10:56.790 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install libcephfs-devel 2024-09-13T09:10:56.816 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:56.816 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:56.816 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:10:56.816 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:56.816 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:56.816 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:57.132 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-mirror 2024-09-13T09:10:57.198 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:35 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:57.292 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:57.334 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:57.336 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-09-13T09:10:57.336 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:57.382 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:36 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:10:57.407 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-rbd 2024-09-13T09:10:57.520 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:10:57.521 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:57.521 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repo Size 2024-09-13T09:10:57.521 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:57.521 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:10:57.521 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 31 k 2024-09-13T09:10:57.521 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:57.522 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:10:57.522 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:10:57.522 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-09-13T09:10:57.522 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:57.522 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 31 k 2024-09-13T09:10:57.522 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 139 k 2024-09-13T09:10:57.522 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:10:57.669 INFO:teuthology.orchestra.run.smithi044.stdout:libcephfs-devel-18.2.4-820.g48a7de7e.el9.x86_64 211 kB/s | 31 kB 00:00 2024-09-13T09:10:57.670 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:57.670 INFO:teuthology.orchestra.run.smithi044.stdout:Total 209 kB/s | 31 kB 00:00 2024-09-13T09:10:57.670 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:10:57.674 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:10:57.674 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:10:57.694 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:10:57.694 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:10:57.718 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:37 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:10:57.790 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:10:57.855 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:10:57.856 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:57.856 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-13T09:10:57.856 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:57.856 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:10:57.856 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.0 M 2024-09-13T09:10:57.856 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:57.856 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:10:57.857 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:10:57.857 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-13T09:10:57.857 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:57.857 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 3.0 M 2024-09-13T09:10:57.857 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 12 M 2024-09-13T09:10:57.857 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:10:57.903 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:57.972 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:35 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:58.063 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:58.105 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:58.106 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-09-13T09:10:58.106 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:10:58.171 INFO:teuthology.orchestra.run.smithi129.stdout:rbd-mirror-18.2.4-820.g48a7de7e.el9.x86_64.rpm 9.5 MB/s | 3.0 MB 00:00 2024-09-13T09:10:58.172 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:58.173 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install rbd-fuse 2024-09-13T09:10:58.174 INFO:teuthology.orchestra.run.smithi129.stdout:Total 9.5 MB/s | 3.0 MB 00:00 2024-09-13T09:10:58.174 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:10:58.181 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:10:58.182 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:10:58.263 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:10:58.263 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:10:58.265 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:58.504 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:58.504 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:58.504 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:10:58.504 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:58.504 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:10:58.504 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:10:58.678 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:10:58.686 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install librados2 2024-09-13T09:10:58.734 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:36 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:10:58.739 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:58.764 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:58.764 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:10:58.764 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T09:10:58.764 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T09:10:58.764 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T09:10:58.764 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:58.867 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:10:58.868 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:58.868 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-09-13T09:10:58.868 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:58.868 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:10:58.868 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 87 k 2024-09-13T09:10:58.868 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:58.868 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:10:58.868 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:10:58.868 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-09-13T09:10:58.868 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:58.868 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 87 k 2024-09-13T09:10:58.868 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 227 k 2024-09-13T09:10:58.869 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:10:59.067 INFO:teuthology.orchestra.run.smithi157.stdout:rbd-fuse-18.2.4-820.g48a7de7e.el9.x86_64.rpm 437 kB/s | 87 kB 00:00 2024-09-13T09:10:59.068 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:10:59.068 INFO:teuthology.orchestra.run.smithi157.stdout:Total 434 kB/s | 87 kB 00:00 2024-09-13T09:10:59.068 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:10:59.075 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:10:59.075 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:10:59.140 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:10:59.140 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:10:59.281 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:38 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:10:59.286 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:10:59.372 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:59.375 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:10:59.421 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:10:59.422 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-09-13T09:10:59.422 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:10:59.490 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install librbd1 2024-09-13T09:10:59.739 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:59.866 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:59.866 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:59.866 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:10:59.866 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:59.866 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:10:59.866 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:10:59.987 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:10:59.987 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:59.987 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:10:59.987 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:10:59.987 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:10:59.987 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:11:00.083 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:39 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:11:00.117 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-nbd 2024-09-13T09:11:00.177 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:11:00.222 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:11:00.223 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-09-13T09:11:00.223 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:11:00.251 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install rbd-mirror 2024-09-13T09:11:00.293 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-rados 2024-09-13T09:11:00.695 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:40 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:11:00.822 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:38 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:11:00.830 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:11:00.831 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:11:00.831 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-13T09:11:00.831 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:11:00.831 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-13T09:11:00.831 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 172 k 2024-09-13T09:11:00.831 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:00.831 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-13T09:11:00.831 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-13T09:11:00.831 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-13T09:11:00.832 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:00.832 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 172 k 2024-09-13T09:11:00.832 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 494 k 2024-09-13T09:11:00.832 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-13T09:11:00.885 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:39 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:11:00.958 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:11:00.959 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:11:00.959 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2024-09-13T09:11:00.959 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:11:00.959 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:11:00.960 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.0 M 2024-09-13T09:11:00.960 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:00.960 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:11:00.960 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:11:00.960 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-09-13T09:11:00.960 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:00.960 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 3.0 M 2024-09-13T09:11:00.960 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 12 M 2024-09-13T09:11:00.960 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:11:00.979 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:11:01.024 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:11:01.025 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-09-13T09:11:01.025 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:11:01.043 INFO:teuthology.orchestra.run.smithi129.stdout:rbd-nbd-18.2.4-820.g48a7de7e.el9.x86_64.rpm 818 kB/s | 172 kB 00:00 2024-09-13T09:11:01.043 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:11:01.044 INFO:teuthology.orchestra.run.smithi129.stdout:Total 810 kB/s | 172 kB 00:00 2024-09-13T09:11:01.044 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-13T09:11:01.052 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-13T09:11:01.052 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-13T09:11:01.097 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-rgw 2024-09-13T09:11:01.116 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-13T09:11:01.116 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-13T09:11:01.278 INFO:teuthology.orchestra.run.smithi157.stdout:rbd-mirror-18.2.4-820.g48a7de7e.el9.x86_64.rpm 9.4 MB/s | 3.0 MB 00:00 2024-09-13T09:11:01.279 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:11:01.280 INFO:teuthology.orchestra.run.smithi157.stdout:Total 9.4 MB/s | 3.0 MB 00:00 2024-09-13T09:11:01.280 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:11:01.288 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:11:01.288 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:11:01.297 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-13T09:11:01.368 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:11:01.368 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:11:01.407 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:01.683 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:40 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:11:01.775 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:11:01.785 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:01.791 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:11:01.820 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:11:01.821 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-09-13T09:11:01.821 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:11:01.854 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:01.879 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:01.880 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:11:01.880 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T09:11:01.880 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T09:11:01.880 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T09:11:01.880 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:01.891 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-cephfs 2024-09-13T09:11:02.033 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:02.033 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:02.033 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-13T09:11:02.033 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:11:02.033 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:02.033 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:11:02.270 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-nbd 2024-09-13T09:11:02.502 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:41 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:11:02.596 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:11:02.642 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:11:02.643 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-09-13T09:11:02.643 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:11:02.715 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-rbd 2024-09-13T09:11:02.853 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:42 ago on Fri 13 Sep 2024 09:09:20 AM UTC. 2024-09-13T09:11:02.946 INFO:teuthology.orchestra.run.smithi129.stdout:Package rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:11:02.952 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:02.952 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:02.952 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:11:02.952 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:11:02.952 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:02.952 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:11:02.989 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-13T09:11:02.990 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-13T09:11:02.990 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-13T09:11:03.068 DEBUG:teuthology.parallel:result is None 2024-09-13T09:11:03.234 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install rbd-nbd 2024-09-13T09:11:03.303 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:42 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:11:03.397 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:11:03.441 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:11:03.442 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-09-13T09:11:03.442 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:11:03.511 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install rbd-fuse 2024-09-13T09:11:03.821 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:41 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:11:03.957 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:11:03.957 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:11:03.957 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-09-13T09:11:03.957 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:11:03.957 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-09-13T09:11:03.957 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 172 k 2024-09-13T09:11:03.957 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:03.957 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-09-13T09:11:03.957 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-09-13T09:11:03.957 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-09-13T09:11:03.958 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:03.958 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 172 k 2024-09-13T09:11:03.958 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 494 k 2024-09-13T09:11:03.958 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-09-13T09:11:04.099 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:43 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:11:04.173 INFO:teuthology.orchestra.run.smithi157.stdout:rbd-nbd-18.2.4-820.g48a7de7e.el9.x86_64.rpm 790 kB/s | 172 kB 00:00 2024-09-13T09:11:04.174 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:11:04.174 INFO:teuthology.orchestra.run.smithi157.stdout:Total 785 kB/s | 172 kB 00:00 2024-09-13T09:11:04.174 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-09-13T09:11:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-09-13T09:11:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-09-13T09:11:04.239 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:11:04.240 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:11:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-09-13T09:11:04.240 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:11:04.240 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:11:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 87 k 2024-09-13T09:11:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:04.241 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:11:04.241 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:11:04.241 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-09-13T09:11:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:04.241 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 87 k 2024-09-13T09:11:04.241 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 227 k 2024-09-13T09:11:04.241 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:11:04.245 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-09-13T09:11:04.245 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-09-13T09:11:04.401 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-09-13T09:11:04.431 INFO:teuthology.orchestra.run.smithi044.stdout:rbd-fuse-18.2.4-820.g48a7de7e.el9.x86_64.rpm 457 kB/s | 87 kB 00:00 2024-09-13T09:11:04.432 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:11:04.432 INFO:teuthology.orchestra.run.smithi044.stdout:Total 454 kB/s | 87 kB 00:00 2024-09-13T09:11:04.432 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:11:04.440 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:11:04.440 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:11:04.504 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:04.506 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:11:04.506 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:11:04.656 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:11:04.756 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:04.907 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:05.182 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:05.189 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:05.189 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:05.189 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-09-13T09:11:05.189 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:11:05.189 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:05.189 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:11:05.409 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install rbd-nbd 2024-09-13T09:11:05.454 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:05.455 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:11:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:11:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:05.455 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:11:05.699 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install rbd-mirror 2024-09-13T09:11:05.993 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:43 ago on Fri 13 Sep 2024 09:09:22 AM UTC. 2024-09-13T09:11:06.086 INFO:teuthology.orchestra.run.smithi157.stdout:Package rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:11:06.131 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-09-13T09:11:06.132 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-09-13T09:11:06.132 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-09-13T09:11:06.210 DEBUG:teuthology.parallel:result is None 2024-09-13T09:11:06.295 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:45 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:11:06.436 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:11:06.436 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:11:06.437 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repo Size 2024-09-13T09:11:06.437 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:11:06.437 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:11:06.437 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.0 M 2024-09-13T09:11:06.437 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:06.437 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:11:06.437 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:11:06.437 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-09-13T09:11:06.437 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:06.437 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 3.0 M 2024-09-13T09:11:06.437 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 12 M 2024-09-13T09:11:06.437 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:11:06.785 INFO:teuthology.orchestra.run.smithi044.stdout:rbd-mirror-18.2.4-820.g48a7de7e.el9.x86_64.rpm 8.6 MB/s | 3.0 MB 00:00 2024-09-13T09:11:06.785 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:11:06.786 INFO:teuthology.orchestra.run.smithi044.stdout:Total 8.6 MB/s | 3.0 MB 00:00 2024-09-13T09:11:06.786 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:11:06.795 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:11:06.796 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:11:06.880 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:11:06.880 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:11:07.319 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:11:07.386 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:07.413 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:07.413 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T09:11:07.413 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T09:11:07.413 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T09:11:07.413 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T09:11:07.413 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:08.469 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:08.469 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:08.469 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:11:08.469 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:11:08.469 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:08.469 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:11:08.707 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install rbd-nbd 2024-09-13T09:11:09.305 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:48 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:11:09.445 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:11:09.445 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:11:09.446 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-09-13T09:11:09.446 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:11:09.446 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-09-13T09:11:09.446 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 172 k 2024-09-13T09:11:09.446 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:09.446 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-09-13T09:11:09.446 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-09-13T09:11:09.446 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-09-13T09:11:09.446 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:09.446 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 172 k 2024-09-13T09:11:09.446 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 494 k 2024-09-13T09:11:09.446 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-09-13T09:11:09.647 INFO:teuthology.orchestra.run.smithi044.stdout:rbd-nbd-18.2.4-820.g48a7de7e.el9.x86_64.rpm 858 kB/s | 172 kB 00:00 2024-09-13T09:11:09.647 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-09-13T09:11:09.648 INFO:teuthology.orchestra.run.smithi044.stdout:Total 852 kB/s | 172 kB 00:00 2024-09-13T09:11:09.648 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-09-13T09:11:09.656 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-09-13T09:11:09.656 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-09-13T09:11:09.723 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-09-13T09:11:09.724 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-09-13T09:11:09.881 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-09-13T09:11:09.980 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:10.416 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T09:11:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:10.686 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-09-13T09:11:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T09:11:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:10.687 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:11:10.951 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install rbd-nbd 2024-09-13T09:11:11.543 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:50 ago on Fri 13 Sep 2024 09:09:21 AM UTC. 2024-09-13T09:11:11.637 INFO:teuthology.orchestra.run.smithi044.stdout:Package rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T09:11:11.682 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-09-13T09:11:11.683 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-09-13T09:11:11.683 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-09-13T09:11:11.754 DEBUG:teuthology.parallel:result is None 2024-09-13T09:11:11.755 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:11:11.901 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T09:11:11.923 INFO:teuthology.orchestra.run.smithi044.stdout:18.2.4-820.g48a7de7e.el9 2024-09-13T09:11:11.924 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820.g48a7de7e.el9 2024-09-13T09:11:11.924 INFO:teuthology.task.install:The correct ceph version 18.2.4-820.g48a7de7e is installed. 2024-09-13T09:11:11.926 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:11:12.070 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T09:11:12.097 INFO:teuthology.orchestra.run.smithi129.stdout:18.2.4-820.g48a7de7e.el9 2024-09-13T09:11:12.098 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820.g48a7de7e.el9 2024-09-13T09:11:12.098 INFO:teuthology.task.install:The correct ceph version 18.2.4-820.g48a7de7e is installed. 2024-09-13T09:11:12.100 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:11:12.247 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T09:11:12.273 INFO:teuthology.orchestra.run.smithi157.stdout:18.2.4-820.g48a7de7e.el9 2024-09-13T09:11:12.273 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820.g48a7de7e.el9 2024-09-13T09:11:12.273 INFO:teuthology.task.install:The correct ceph version 18.2.4-820.g48a7de7e is installed. 2024-09-13T09:11:12.275 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T09:11:12.275 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:12.275 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T09:11:12.301 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:11:12.301 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T09:11:12.333 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T09:11:12.333 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T09:11:12.364 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T09:11:12.364 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:12.364 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T09:11:12.390 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T09:11:12.455 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:11:12.455 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T09:11:12.485 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T09:11:12.554 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T09:11:12.555 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T09:11:12.585 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T09:11:12.653 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T09:11:12.654 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:12.654 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T09:11:12.680 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T09:11:12.745 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:11:12.745 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T09:11:12.774 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T09:11:12.842 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T09:11:12.842 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T09:11:12.872 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T09:11:12.942 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T09:11:12.942 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:12.943 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T09:11:12.968 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T09:11:13.033 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:11:13.033 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T09:11:13.061 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T09:11:13.130 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T09:11:13.130 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T09:11:13.160 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T09:11:13.230 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T09:11:13.312 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T09:11:13.312 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /var/log/ceph 2024-09-13T09:11:13.314 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 777 /var/log/ceph 2024-09-13T09:11:13.316 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /var/log/ceph 2024-09-13T09:11:13.346 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T09:11:13.347 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T09:11:13.380 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T09:11:13.384 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T09:11:13.416 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T09:11:13.417 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T09:11:13.445 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T09:11:13.451 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T09:11:13.494 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T09:11:13.494 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T09:11:13.494 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020', 'branch': 'reef', 'description': 'rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7901020', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'], ['client.0']], 'seed': 248, 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412', 'sleep_before_teardown': 0, 'subset': '116/128', 'suite': 'rbd', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880', 'targets': {'smithi044.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE7VOPSW1CnNGqTTu6lwBvPpvTowORkm2kLGFvlCNxgVPMrzKfuIs1e7QPHN6Z4K2zQ2jJ17k6OOo4Xg6jyXCZo=', 'smithi129.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE5s50LibChURJmLORumpBijimYXHR7QrZnPNtIhguPk2st/3rLA5Xuu7HaXHx1bwCQh6SzoLo9CPx79HW0ZPzQ=', 'smithi157.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLvWUU9S+ygGjt1DtQ4PPL453Ug0aVHB9+X+kQSXGTQI9jkAyTz3uh/I1zj1oD+eE48vgby6faEb99XcOJMEpBA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'holebdy': 512, 'nbd': True, 'ops': 6000, 'punch_holes': True, 'readbdy': 512, 'truncbdy': 512, 'writebdy': 512}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_22:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T09:11:13.495 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T09:11:13.511 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T09:11:13.518 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T09:11:13.544 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T09:11:13.567 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T09:11:13.574 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T09:11:13.611 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:13.611 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T09:11:13.649 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T09:11:13.649 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2024-09-13T09:11:13.704 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T09:11:13.705 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:11:13.705 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-13T09:11:13.705 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:11:13.705 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T09:11:13.705 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-09-13 09:11:07.672361426 +0000 2024-09-13T09:11:13.705 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-09-13 09:08:43.361610986 +0000 2024-09-13T09:11:13.705 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-09-13 09:08:43.361610986 +0000 2024-09-13T09:11:13.705 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-09-13 09:08:43.361610986 +0000 2024-09-13T09:11:13.705 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T09:11:13.769 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-09-13T09:11:13.769 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-09-13T09:11:13.769 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000259867 s, 2.0 MB/s 2024-09-13T09:11:13.770 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T09:11:13.827 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2024-09-13T09:11:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T09:11:13.884 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:11:13.884 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-13T09:11:13.884 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:11:13.884 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T09:11:13.884 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-09-13 09:11:07.673361409 +0000 2024-09-13T09:11:13.884 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-09-13 09:08:43.702605216 +0000 2024-09-13T09:11:13.884 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-09-13 09:08:43.702605216 +0000 2024-09-13T09:11:13.884 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-09-13 09:08:43.702605216 +0000 2024-09-13T09:11:13.884 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T09:11:13.947 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-09-13T09:11:13.947 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-09-13T09:11:13.948 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000190519 s, 2.7 MB/s 2024-09-13T09:11:13.949 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T09:11:14.005 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2024-09-13T09:11:14.060 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T09:11:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:11:14.061 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-13T09:11:14.061 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:11:14.061 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T09:11:14.061 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-09-13 09:11:07.673361409 +0000 2024-09-13T09:11:14.061 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-09-13 09:08:44.038599530 +0000 2024-09-13T09:11:14.061 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-09-13 09:08:44.038599530 +0000 2024-09-13T09:11:14.061 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-09-13 09:08:44.038599530 +0000 2024-09-13T09:11:14.061 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T09:11:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-09-13T09:11:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-09-13T09:11:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000189856 s, 2.7 MB/s 2024-09-13T09:11:14.126 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T09:11:14.182 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2024-09-13T09:11:14.238 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T09:11:14.238 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:11:14.238 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-13T09:11:14.239 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:11:14.239 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T09:11:14.239 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-09-13 09:11:07.674361392 +0000 2024-09-13T09:11:14.239 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-09-13 09:08:44.367593963 +0000 2024-09-13T09:11:14.239 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-09-13 09:08:44.367593963 +0000 2024-09-13T09:11:14.239 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-09-13 09:08:44.367593963 +0000 2024-09-13T09:11:14.239 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T09:11:14.302 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-09-13T09:11:14.303 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-09-13T09:11:14.303 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000218602 s, 2.3 MB/s 2024-09-13T09:11:14.304 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T09:11:14.360 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-13T09:11:14.360 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:11:14.360 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T09:11:14.380 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T09:11:14.380 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 2024-09-13T09:11:14.438 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T09:11:14.438 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:11:14.438 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-13T09:11:14.438 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:11:14.438 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T09:11:14.438 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-13 09:10:59.018549494 +0000 2024-09-13T09:11:14.438 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-13 09:08:41.686626720 +0000 2024-09-13T09:11:14.438 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-13 09:08:41.686626720 +0000 2024-09-13T09:11:14.439 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-13 09:08:41.686626720 +0000 2024-09-13T09:11:14.439 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T09:11:14.506 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-13T09:11:14.506 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-13T09:11:14.506 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000310279 s, 1.7 MB/s 2024-09-13T09:11:14.508 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T09:11:14.567 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 2024-09-13T09:11:14.625 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T09:11:14.626 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:11:14.626 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-13T09:11:14.626 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:11:14.626 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T09:11:14.626 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-13 09:10:59.019549472 +0000 2024-09-13T09:11:14.626 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-13 09:08:42.009619639 +0000 2024-09-13T09:11:14.626 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-13 09:08:42.009619639 +0000 2024-09-13T09:11:14.626 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-13 09:08:42.009619639 +0000 2024-09-13T09:11:14.626 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T09:11:14.691 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-13T09:11:14.691 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-13T09:11:14.691 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000195835 s, 2.6 MB/s 2024-09-13T09:11:14.693 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T09:11:14.751 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 2024-09-13T09:11:14.808 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T09:11:14.808 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:11:14.808 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-13T09:11:14.808 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:11:14.808 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T09:11:14.808 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-13 09:10:59.019549472 +0000 2024-09-13T09:11:14.808 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-13 09:08:42.345612272 +0000 2024-09-13T09:11:14.808 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-13 09:08:42.345612272 +0000 2024-09-13T09:11:14.809 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-13 09:08:42.345612272 +0000 2024-09-13T09:11:14.809 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T09:11:14.873 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-13T09:11:14.873 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-13T09:11:14.873 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.00020092 s, 2.5 MB/s 2024-09-13T09:11:14.874 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T09:11:14.931 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 2024-09-13T09:11:14.988 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T09:11:14.988 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:11:14.988 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-13T09:11:14.988 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:11:14.988 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T09:11:14.988 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-13 09:10:59.020549451 +0000 2024-09-13T09:11:14.988 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-13 09:08:42.671605125 +0000 2024-09-13T09:11:14.989 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-13 09:08:42.671605125 +0000 2024-09-13T09:11:14.989 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-13 09:08:42.671605125 +0000 2024-09-13T09:11:14.989 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T09:11:15.053 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-13T09:11:15.053 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-13T09:11:15.053 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000198945 s, 2.6 MB/s 2024-09-13T09:11:15.054 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T09:11:15.111 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-13T09:11:15.112 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi129.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-13T09:11:15.112 INFO:tasks.ceph:Generating config... 2024-09-13T09:11:15.113 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-13T09:11:15.113 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-13T09:11:15.113 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-13T09:11:15.113 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T09:11:15.113 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T09:11:15.113 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T09:11:15.113 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T09:11:15.114 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T09:11:15.114 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-13T09:11:15.114 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-13T09:11:15.114 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T09:11:15.114 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T09:11:15.114 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T09:11:15.114 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T09:11:15.114 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T09:11:15.114 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T09:11:15.114 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T09:11:15.114 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T09:11:15.114 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T09:11:15.114 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T09:11:15.114 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-13T09:11:15.114 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T09:11:15.115 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-13T09:11:15.115 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2024-09-13T09:11:15.115 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T09:11:15.115 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T09:11:15.115 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T09:11:15.166 INFO:teuthology.orchestra.run.smithi044.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T09:11:15.169 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T09:11:15.215 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T09:11:15.241 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.44'), ('mon.c', '[v2:172.21.15.44:3301,v1:172.21.15.44:6790]'), ('mon.b', '172.21.15.129')] 2024-09-13T09:11:15.241 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.44,[v2:172.21.15.44:3301,v1:172.21.15.44:6790],172.21.15.129', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-13T09:11:15.242 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:15.242 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T09:11:15.297 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.44 --addv c '[v2:172.21.15.44:3301,v1:172.21.15.44:6790]' --add b 172.21.15.129 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:11:15.383 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:11:15.384 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: generated fsid 06cda68b-dd73-4266-b5e6-93ae2b3f7bd5 2024-09-13T09:11:15.384 INFO:teuthology.orchestra.run.smithi044.stdout:setting min_mon_release = pacific 2024-09-13T09:11:15.384 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 0 2024-09-13T09:11:15.384 INFO:teuthology.orchestra.run.smithi044.stdout:fsid 06cda68b-dd73-4266-b5e6-93ae2b3f7bd5 2024-09-13T09:11:15.384 INFO:teuthology.orchestra.run.smithi044.stdout:last_changed 2024-09-13T09:11:15.383371+0000 2024-09-13T09:11:15.384 INFO:teuthology.orchestra.run.smithi044.stdout:created 2024-09-13T09:11:15.383371+0000 2024-09-13T09:11:15.384 INFO:teuthology.orchestra.run.smithi044.stdout:min_mon_release 16 (pacific) 2024-09-13T09:11:15.384 INFO:teuthology.orchestra.run.smithi044.stdout:election_strategy: 1 2024-09-13T09:11:15.384 INFO:teuthology.orchestra.run.smithi044.stdout:0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-09-13T09:11:15.384 INFO:teuthology.orchestra.run.smithi044.stdout:1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-09-13T09:11:15.384 INFO:teuthology.orchestra.run.smithi044.stdout:2: [v2:172.21.15.44:3301/0,v1:172.21.15.44:6790/0] mon.c 2024-09-13T09:11:15.384 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T09:11:15.386 DEBUG:teuthology.orchestra.run.smithi044:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T09:11:15.401 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 06cda68b-dd73-4266-b5e6-93ae2b3f7bd5... 2024-09-13T09:11:15.402 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T09:11:15.444 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T09:11:15.447 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T09:11:15.486 INFO:teuthology.orchestra.run.smithi044.stdout:[global] 2024-09-13T09:11:15.486 INFO:teuthology.orchestra.run.smithi044.stdout: chdir = "" 2024-09-13T09:11:15.487 INFO:teuthology.orchestra.run.smithi044.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T09:11:15.487 INFO:teuthology.orchestra.run.smithi044.stdout: auth supported = cephx 2024-09-13T09:11:15.487 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.487 INFO:teuthology.orchestra.run.smithi044.stdout: filestore xattr use omap = true 2024-09-13T09:11:15.487 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.487 INFO:teuthology.orchestra.run.smithi044.stdout: mon clock drift allowed = 1.000 2024-09-13T09:11:15.487 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.487 INFO:teuthology.orchestra.run.smithi044.stdout: osd crush chooseleaf type = 0 2024-09-13T09:11:15.487 INFO:teuthology.orchestra.run.smithi044.stdout: auth debug = true 2024-09-13T09:11:15.487 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.487 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on old message = true 2024-09-13T09:11:15.487 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on bug = true 2024-09-13T09:11:15.487 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.488 INFO:teuthology.orchestra.run.smithi044.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T09:11:15.488 INFO:teuthology.orchestra.run.smithi044.stdout: mon pg warn max object skew = 0 2024-09-13T09:11:15.488 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.488 INFO:teuthology.orchestra.run.smithi044.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T09:11:15.488 INFO:teuthology.orchestra.run.smithi044.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T09:11:15.488 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.488 INFO:teuthology.orchestra.run.smithi044.stdout: osd pool default size = 2 2024-09-13T09:11:15.488 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.488 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd allow primary affinity = true 2024-09-13T09:11:15.488 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd allow pg remap = true 2024-09-13T09:11:15.488 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on legacy crush tunables = false 2024-09-13T09:11:15.488 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on crush straw calc version zero = false 2024-09-13T09:11:15.488 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on no sortbitwise = false 2024-09-13T09:11:15.489 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on osd down out interval zero = false 2024-09-13T09:11:15.489 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on too few osds = false 2024-09-13T09:11:15.489 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T09:11:15.489 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T09:11:15.489 INFO:teuthology.orchestra.run.smithi044.stdout: mon_allow_pool_size_one = true 2024-09-13T09:11:15.489 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.489 INFO:teuthology.orchestra.run.smithi044.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T09:11:15.489 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.489 INFO:teuthology.orchestra.run.smithi044.stdout: osd default data pool replay window = 5 2024-09-13T09:11:15.489 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.489 INFO:teuthology.orchestra.run.smithi044.stdout: mon allow pool delete = true 2024-09-13T09:11:15.489 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.489 INFO:teuthology.orchestra.run.smithi044.stdout: mon cluster log file level = debug 2024-09-13T09:11:15.489 INFO:teuthology.orchestra.run.smithi044.stdout: debug asserts on shutdown = true 2024-09-13T09:11:15.490 INFO:teuthology.orchestra.run.smithi044.stdout: mon health detail to clog = false 2024-09-13T09:11:15.490 INFO:teuthology.orchestra.run.smithi044.stdout: mon host = "172.21.15.44,[v2:172.21.15.44:3301,v1:172.21.15.44:6790],172.21.15.129" 2024-09-13T09:11:15.490 INFO:teuthology.orchestra.run.smithi044.stdout: mon client directed command retry = 5 2024-09-13T09:11:15.490 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on pool no app = False 2024-09-13T09:11:15.490 INFO:teuthology.orchestra.run.smithi044.stdout: ms inject socket failures = 5000 2024-09-13T09:11:15.490 INFO:teuthology.orchestra.run.smithi044.stdout: fsid = 06cda68b-dd73-4266-b5e6-93ae2b3f7bd5 2024-09-13T09:11:15.490 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.490 INFO:teuthology.orchestra.run.smithi044.stdout:[osd] 2024-09-13T09:11:15.490 INFO:teuthology.orchestra.run.smithi044.stdout: osd journal size = 100 2024-09-13T09:11:15.490 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.490 INFO:teuthology.orchestra.run.smithi044.stdout: osd scrub load threshold = 5.0 2024-09-13T09:11:15.490 INFO:teuthology.orchestra.run.smithi044.stdout: osd scrub max interval = 600 2024-09-13T09:11:15.490 INFO:teuthology.orchestra.run.smithi044.stdout: osd mclock profile = high_recovery_ops 2024-09-13T09:11:15.490 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.491 INFO:teuthology.orchestra.run.smithi044.stdout: osd recover clone overlap = true 2024-09-13T09:11:15.491 INFO:teuthology.orchestra.run.smithi044.stdout: osd recovery max chunk = 1048576 2024-09-13T09:11:15.491 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.491 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug shutdown = true 2024-09-13T09:11:15.491 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug op order = true 2024-09-13T09:11:15.491 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug verify stray on activate = true 2024-09-13T09:11:15.491 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.491 INFO:teuthology.orchestra.run.smithi044.stdout: osd open classes on start = true 2024-09-13T09:11:15.491 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug pg log writeout = true 2024-09-13T09:11:15.491 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.491 INFO:teuthology.orchestra.run.smithi044.stdout: osd deep scrub update digest min age = 30 2024-09-13T09:11:15.491 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.491 INFO:teuthology.orchestra.run.smithi044.stdout: osd map max advance = 10 2024-09-13T09:11:15.491 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.492 INFO:teuthology.orchestra.run.smithi044.stdout: journal zero on create = true 2024-09-13T09:11:15.492 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.492 INFO:teuthology.orchestra.run.smithi044.stdout: filestore ondisk finisher threads = 3 2024-09-13T09:11:15.492 INFO:teuthology.orchestra.run.smithi044.stdout: filestore apply finisher threads = 3 2024-09-13T09:11:15.492 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.492 INFO:teuthology.orchestra.run.smithi044.stdout: bdev debug aio = true 2024-09-13T09:11:15.492 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug misdirected ops = true 2024-09-13T09:11:15.492 INFO:teuthology.orchestra.run.smithi044.stdout: bdev async discard = True 2024-09-13T09:11:15.492 INFO:teuthology.orchestra.run.smithi044.stdout: bdev enable discard = True 2024-09-13T09:11:15.492 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore block size = 96636764160 2024-09-13T09:11:15.492 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore fsck on mount = True 2024-09-13T09:11:15.492 INFO:teuthology.orchestra.run.smithi044.stdout: debug bluefs = 1/20 2024-09-13T09:11:15.492 INFO:teuthology.orchestra.run.smithi044.stdout: debug bluestore = 1/20 2024-09-13T09:11:15.492 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-09-13T09:11:15.493 INFO:teuthology.orchestra.run.smithi044.stdout: debug osd = 20 2024-09-13T09:11:15.493 INFO:teuthology.orchestra.run.smithi044.stdout: debug rocksdb = 4/10 2024-09-13T09:11:15.493 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T09:11:15.493 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd full ratio = 0.9 2024-09-13T09:11:15.493 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd nearfull ratio = 0.8 2024-09-13T09:11:15.493 INFO:teuthology.orchestra.run.smithi044.stdout: osd blocked scrub grace period = 3600 2024-09-13T09:11:15.493 INFO:teuthology.orchestra.run.smithi044.stdout: osd failsafe full ratio = 0.95 2024-09-13T09:11:15.493 INFO:teuthology.orchestra.run.smithi044.stdout: osd max markdown count = 1000 2024-09-13T09:11:15.493 INFO:teuthology.orchestra.run.smithi044.stdout: osd memory target = 1610612736 2024-09-13T09:11:15.493 INFO:teuthology.orchestra.run.smithi044.stdout: osd objectstore = bluestore 2024-09-13T09:11:15.493 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.493 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr] 2024-09-13T09:11:15.493 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-09-13T09:11:15.493 INFO:teuthology.orchestra.run.smithi044.stdout: debug mgr = 20 2024-09-13T09:11:15.493 INFO:teuthology.orchestra.run.smithi044.stdout: debug mon = 20 2024-09-13T09:11:15.494 INFO:teuthology.orchestra.run.smithi044.stdout: debug auth = 20 2024-09-13T09:11:15.494 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:11:15.494 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:11:15.494 INFO:teuthology.orchestra.run.smithi044.stdout: mgr/telemetry/nag = false 2024-09-13T09:11:15.494 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.494 INFO:teuthology.orchestra.run.smithi044.stdout:[mon] 2024-09-13T09:11:15.494 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-09-13T09:11:15.494 INFO:teuthology.orchestra.run.smithi044.stdout: debug mon = 20 2024-09-13T09:11:15.494 INFO:teuthology.orchestra.run.smithi044.stdout: debug paxos = 20 2024-09-13T09:11:15.494 INFO:teuthology.orchestra.run.smithi044.stdout: debug auth = 20 2024-09-13T09:11:15.494 INFO:teuthology.orchestra.run.smithi044.stdout: mon data avail warn = 5 2024-09-13T09:11:15.494 INFO:teuthology.orchestra.run.smithi044.stdout: mon mgr mkfs grace = 240 2024-09-13T09:11:15.494 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:11:15.494 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd reporter subtree level = osd 2024-09-13T09:11:15.495 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd prime pg temp = true 2024-09-13T09:11:15.495 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:11:15.495 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.495 INFO:teuthology.orchestra.run.smithi044.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T09:11:15.495 INFO:teuthology.orchestra.run.smithi044.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T09:11:15.495 INFO:teuthology.orchestra.run.smithi044.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T09:11:15.495 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.495 INFO:teuthology.orchestra.run.smithi044.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T09:11:15.495 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T09:11:15.495 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T09:11:15.495 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.495 INFO:teuthology.orchestra.run.smithi044.stdout: # 1m isn't quite enough 2024-09-13T09:11:15.495 INFO:teuthology.orchestra.run.smithi044.stdout: mon_down_mkfs_grace = 2m 2024-09-13T09:11:15.495 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.495 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_filestore_osds = false 2024-09-13T09:11:15.496 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:15.496 INFO:teuthology.orchestra.run.smithi044.stdout:[client] 2024-09-13T09:11:15.496 INFO:teuthology.orchestra.run.smithi044.stdout: rgw cache enabled = true 2024-09-13T09:11:15.496 INFO:teuthology.orchestra.run.smithi044.stdout: rgw enable ops log = true 2024-09-13T09:11:15.496 INFO:teuthology.orchestra.run.smithi044.stdout: rgw enable usage log = true 2024-09-13T09:11:15.496 INFO:teuthology.orchestra.run.smithi044.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T09:11:15.496 INFO:teuthology.orchestra.run.smithi044.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T09:11:15.496 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.a] 2024-09-13T09:11:15.496 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.c] 2024-09-13T09:11:15.496 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.b] 2024-09-13T09:11:15.497 INFO:teuthology.orchestra.run.smithi129.stdout:[global] 2024-09-13T09:11:15.497 INFO:teuthology.orchestra.run.smithi129.stdout: chdir = "" 2024-09-13T09:11:15.497 INFO:teuthology.orchestra.run.smithi129.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T09:11:15.497 INFO:teuthology.orchestra.run.smithi129.stdout: auth supported = cephx 2024-09-13T09:11:15.497 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.497 INFO:teuthology.orchestra.run.smithi129.stdout: filestore xattr use omap = true 2024-09-13T09:11:15.497 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.497 INFO:teuthology.orchestra.run.smithi129.stdout: mon clock drift allowed = 1.000 2024-09-13T09:11:15.497 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.497 INFO:teuthology.orchestra.run.smithi129.stdout: osd crush chooseleaf type = 0 2024-09-13T09:11:15.497 INFO:teuthology.orchestra.run.smithi129.stdout: auth debug = true 2024-09-13T09:11:15.497 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.497 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on old message = true 2024-09-13T09:11:15.498 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on bug = true 2024-09-13T09:11:15.498 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.498 INFO:teuthology.orchestra.run.smithi129.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T09:11:15.498 INFO:teuthology.orchestra.run.smithi129.stdout: mon pg warn max object skew = 0 2024-09-13T09:11:15.498 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.498 INFO:teuthology.orchestra.run.smithi129.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T09:11:15.498 INFO:teuthology.orchestra.run.smithi129.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T09:11:15.498 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.498 INFO:teuthology.orchestra.run.smithi129.stdout: osd pool default size = 2 2024-09-13T09:11:15.498 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.498 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd allow primary affinity = true 2024-09-13T09:11:15.498 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd allow pg remap = true 2024-09-13T09:11:15.498 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on legacy crush tunables = false 2024-09-13T09:11:15.498 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on crush straw calc version zero = false 2024-09-13T09:11:15.499 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on no sortbitwise = false 2024-09-13T09:11:15.499 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on osd down out interval zero = false 2024-09-13T09:11:15.499 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on too few osds = false 2024-09-13T09:11:15.499 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T09:11:15.499 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T09:11:15.499 INFO:teuthology.orchestra.run.smithi129.stdout: mon_allow_pool_size_one = true 2024-09-13T09:11:15.499 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.499 INFO:teuthology.orchestra.run.smithi129.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T09:11:15.499 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.499 INFO:teuthology.orchestra.run.smithi129.stdout: osd default data pool replay window = 5 2024-09-13T09:11:15.499 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.499 INFO:teuthology.orchestra.run.smithi129.stdout: mon allow pool delete = true 2024-09-13T09:11:15.499 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.499 INFO:teuthology.orchestra.run.smithi129.stdout: mon cluster log file level = debug 2024-09-13T09:11:15.500 INFO:teuthology.orchestra.run.smithi129.stdout: debug asserts on shutdown = true 2024-09-13T09:11:15.500 INFO:teuthology.orchestra.run.smithi129.stdout: mon health detail to clog = false 2024-09-13T09:11:15.500 INFO:teuthology.orchestra.run.smithi129.stdout: mon host = "172.21.15.44,[v2:172.21.15.44:3301,v1:172.21.15.44:6790],172.21.15.129" 2024-09-13T09:11:15.500 INFO:teuthology.orchestra.run.smithi129.stdout: mon client directed command retry = 5 2024-09-13T09:11:15.500 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on pool no app = False 2024-09-13T09:11:15.500 INFO:teuthology.orchestra.run.smithi129.stdout: ms inject socket failures = 5000 2024-09-13T09:11:15.500 INFO:teuthology.orchestra.run.smithi129.stdout: fsid = 06cda68b-dd73-4266-b5e6-93ae2b3f7bd5 2024-09-13T09:11:15.500 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.500 INFO:teuthology.orchestra.run.smithi129.stdout:[osd] 2024-09-13T09:11:15.500 INFO:teuthology.orchestra.run.smithi129.stdout: osd journal size = 100 2024-09-13T09:11:15.500 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.500 INFO:teuthology.orchestra.run.smithi129.stdout: osd scrub load threshold = 5.0 2024-09-13T09:11:15.500 INFO:teuthology.orchestra.run.smithi129.stdout: osd scrub max interval = 600 2024-09-13T09:11:15.500 INFO:teuthology.orchestra.run.smithi129.stdout: osd mclock profile = high_recovery_ops 2024-09-13T09:11:15.501 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.501 INFO:teuthology.orchestra.run.smithi129.stdout: osd recover clone overlap = true 2024-09-13T09:11:15.501 INFO:teuthology.orchestra.run.smithi129.stdout: osd recovery max chunk = 1048576 2024-09-13T09:11:15.501 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.501 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug shutdown = true 2024-09-13T09:11:15.501 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug op order = true 2024-09-13T09:11:15.501 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug verify stray on activate = true 2024-09-13T09:11:15.501 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.501 INFO:teuthology.orchestra.run.smithi129.stdout: osd open classes on start = true 2024-09-13T09:11:15.501 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug pg log writeout = true 2024-09-13T09:11:15.501 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.501 INFO:teuthology.orchestra.run.smithi129.stdout: osd deep scrub update digest min age = 30 2024-09-13T09:11:15.501 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.501 INFO:teuthology.orchestra.run.smithi129.stdout: osd map max advance = 10 2024-09-13T09:11:15.501 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.502 INFO:teuthology.orchestra.run.smithi129.stdout: journal zero on create = true 2024-09-13T09:11:15.502 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.502 INFO:teuthology.orchestra.run.smithi129.stdout: filestore ondisk finisher threads = 3 2024-09-13T09:11:15.502 INFO:teuthology.orchestra.run.smithi129.stdout: filestore apply finisher threads = 3 2024-09-13T09:11:15.502 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.502 INFO:teuthology.orchestra.run.smithi129.stdout: bdev debug aio = true 2024-09-13T09:11:15.502 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug misdirected ops = true 2024-09-13T09:11:15.502 INFO:teuthology.orchestra.run.smithi129.stdout: bdev async discard = True 2024-09-13T09:11:15.502 INFO:teuthology.orchestra.run.smithi129.stdout: bdev enable discard = True 2024-09-13T09:11:15.502 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore block size = 96636764160 2024-09-13T09:11:15.502 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore fsck on mount = True 2024-09-13T09:11:15.502 INFO:teuthology.orchestra.run.smithi129.stdout: debug bluefs = 1/20 2024-09-13T09:11:15.502 INFO:teuthology.orchestra.run.smithi129.stdout: debug bluestore = 1/20 2024-09-13T09:11:15.502 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-13T09:11:15.503 INFO:teuthology.orchestra.run.smithi129.stdout: debug osd = 20 2024-09-13T09:11:15.503 INFO:teuthology.orchestra.run.smithi129.stdout: debug rocksdb = 4/10 2024-09-13T09:11:15.503 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T09:11:15.503 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd full ratio = 0.9 2024-09-13T09:11:15.503 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd nearfull ratio = 0.8 2024-09-13T09:11:15.503 INFO:teuthology.orchestra.run.smithi129.stdout: osd blocked scrub grace period = 3600 2024-09-13T09:11:15.503 INFO:teuthology.orchestra.run.smithi129.stdout: osd failsafe full ratio = 0.95 2024-09-13T09:11:15.503 INFO:teuthology.orchestra.run.smithi129.stdout: osd max markdown count = 1000 2024-09-13T09:11:15.503 INFO:teuthology.orchestra.run.smithi129.stdout: osd memory target = 1610612736 2024-09-13T09:11:15.503 INFO:teuthology.orchestra.run.smithi129.stdout: osd objectstore = bluestore 2024-09-13T09:11:15.503 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.503 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr] 2024-09-13T09:11:15.503 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-13T09:11:15.503 INFO:teuthology.orchestra.run.smithi129.stdout: debug mgr = 20 2024-09-13T09:11:15.504 INFO:teuthology.orchestra.run.smithi129.stdout: debug mon = 20 2024-09-13T09:11:15.504 INFO:teuthology.orchestra.run.smithi129.stdout: debug auth = 20 2024-09-13T09:11:15.504 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:11:15.504 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:11:15.504 INFO:teuthology.orchestra.run.smithi129.stdout: mgr/telemetry/nag = false 2024-09-13T09:11:15.504 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.504 INFO:teuthology.orchestra.run.smithi129.stdout:[mon] 2024-09-13T09:11:15.504 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-13T09:11:15.504 INFO:teuthology.orchestra.run.smithi129.stdout: debug mon = 20 2024-09-13T09:11:15.504 INFO:teuthology.orchestra.run.smithi129.stdout: debug paxos = 20 2024-09-13T09:11:15.504 INFO:teuthology.orchestra.run.smithi129.stdout: debug auth = 20 2024-09-13T09:11:15.504 INFO:teuthology.orchestra.run.smithi129.stdout: mon data avail warn = 5 2024-09-13T09:11:15.504 INFO:teuthology.orchestra.run.smithi129.stdout: mon mgr mkfs grace = 240 2024-09-13T09:11:15.504 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:11:15.505 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd reporter subtree level = osd 2024-09-13T09:11:15.505 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd prime pg temp = true 2024-09-13T09:11:15.505 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:11:15.505 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.505 INFO:teuthology.orchestra.run.smithi129.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T09:11:15.505 INFO:teuthology.orchestra.run.smithi129.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T09:11:15.505 INFO:teuthology.orchestra.run.smithi129.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T09:11:15.505 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.505 INFO:teuthology.orchestra.run.smithi129.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T09:11:15.505 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T09:11:15.505 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T09:11:15.505 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.505 INFO:teuthology.orchestra.run.smithi129.stdout: # 1m isn't quite enough 2024-09-13T09:11:15.506 INFO:teuthology.orchestra.run.smithi129.stdout: mon_down_mkfs_grace = 2m 2024-09-13T09:11:15.506 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.506 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_filestore_osds = false 2024-09-13T09:11:15.506 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-13T09:11:15.506 INFO:teuthology.orchestra.run.smithi129.stdout:[client] 2024-09-13T09:11:15.506 INFO:teuthology.orchestra.run.smithi129.stdout: rgw cache enabled = true 2024-09-13T09:11:15.506 INFO:teuthology.orchestra.run.smithi129.stdout: rgw enable ops log = true 2024-09-13T09:11:15.506 INFO:teuthology.orchestra.run.smithi129.stdout: rgw enable usage log = true 2024-09-13T09:11:15.506 INFO:teuthology.orchestra.run.smithi129.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T09:11:15.506 INFO:teuthology.orchestra.run.smithi129.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T09:11:15.507 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.a] 2024-09-13T09:11:15.507 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.c] 2024-09-13T09:11:15.507 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.b] 2024-09-13T09:11:15.508 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2024-09-13T09:11:15.508 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2024-09-13T09:11:15.508 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T09:11:15.508 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2024-09-13T09:11:15.508 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.508 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2024-09-13T09:11:15.508 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.508 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2024-09-13T09:11:15.508 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.508 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2024-09-13T09:11:15.508 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2024-09-13T09:11:15.508 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.509 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2024-09-13T09:11:15.509 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2024-09-13T09:11:15.509 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.509 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T09:11:15.509 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2024-09-13T09:11:15.509 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.509 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T09:11:15.509 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T09:11:15.509 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.509 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2024-09-13T09:11:15.509 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.509 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2024-09-13T09:11:15.509 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2024-09-13T09:11:15.509 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2024-09-13T09:11:15.510 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2024-09-13T09:11:15.510 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2024-09-13T09:11:15.510 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2024-09-13T09:11:15.510 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2024-09-13T09:11:15.510 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T09:11:15.510 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T09:11:15.510 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2024-09-13T09:11:15.510 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.510 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T09:11:15.510 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.510 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2024-09-13T09:11:15.510 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.510 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2024-09-13T09:11:15.511 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.511 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2024-09-13T09:11:15.511 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2024-09-13T09:11:15.511 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2024-09-13T09:11:15.511 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.44,[v2:172.21.15.44:3301,v1:172.21.15.44:6790],172.21.15.129" 2024-09-13T09:11:15.511 INFO:teuthology.orchestra.run.smithi157.stdout: mon client directed command retry = 5 2024-09-13T09:11:15.511 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on pool no app = False 2024-09-13T09:11:15.511 INFO:teuthology.orchestra.run.smithi157.stdout: ms inject socket failures = 5000 2024-09-13T09:11:15.511 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = 06cda68b-dd73-4266-b5e6-93ae2b3f7bd5 2024-09-13T09:11:15.511 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.511 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2024-09-13T09:11:15.511 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2024-09-13T09:11:15.511 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.511 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2024-09-13T09:11:15.512 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2024-09-13T09:11:15.512 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2024-09-13T09:11:15.512 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.512 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2024-09-13T09:11:15.512 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2024-09-13T09:11:15.512 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.512 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2024-09-13T09:11:15.512 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2024-09-13T09:11:15.512 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2024-09-13T09:11:15.512 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.512 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2024-09-13T09:11:15.512 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2024-09-13T09:11:15.512 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.512 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2024-09-13T09:11:15.513 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.513 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2024-09-13T09:11:15.513 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.513 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2024-09-13T09:11:15.513 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.513 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2024-09-13T09:11:15.513 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2024-09-13T09:11:15.513 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.513 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2024-09-13T09:11:15.513 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2024-09-13T09:11:15.513 INFO:teuthology.orchestra.run.smithi157.stdout: bdev async discard = True 2024-09-13T09:11:15.513 INFO:teuthology.orchestra.run.smithi157.stdout: bdev enable discard = True 2024-09-13T09:11:15.513 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2024-09-13T09:11:15.513 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2024-09-13T09:11:15.514 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 1/20 2024-09-13T09:11:15.514 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 1/20 2024-09-13T09:11:15.514 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T09:11:15.514 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2024-09-13T09:11:15.514 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 4/10 2024-09-13T09:11:15.514 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T09:11:15.514 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2024-09-13T09:11:15.514 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2024-09-13T09:11:15.514 INFO:teuthology.orchestra.run.smithi157.stdout: osd blocked scrub grace period = 3600 2024-09-13T09:11:15.514 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2024-09-13T09:11:15.514 INFO:teuthology.orchestra.run.smithi157.stdout: osd max markdown count = 1000 2024-09-13T09:11:15.514 INFO:teuthology.orchestra.run.smithi157.stdout: osd memory target = 1610612736 2024-09-13T09:11:15.514 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2024-09-13T09:11:15.514 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.515 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2024-09-13T09:11:15.515 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T09:11:15.515 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2024-09-13T09:11:15.515 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-09-13T09:11:15.515 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-09-13T09:11:15.515 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:11:15.515 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:11:15.515 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2024-09-13T09:11:15.515 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.515 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2024-09-13T09:11:15.515 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-13T09:11:15.515 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-09-13T09:11:15.515 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2024-09-13T09:11:15.515 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-09-13T09:11:15.516 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2024-09-13T09:11:15.516 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2024-09-13T09:11:15.516 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:11:15.516 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2024-09-13T09:11:15.516 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2024-09-13T09:11:15.516 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:11:15.516 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.516 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T09:11:15.516 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T09:11:15.516 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T09:11:15.516 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.516 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T09:11:15.516 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T09:11:15.516 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T09:11:15.517 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.517 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2024-09-13T09:11:15.517 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2024-09-13T09:11:15.517 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.517 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2024-09-13T09:11:15.517 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-13T09:11:15.517 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2024-09-13T09:11:15.517 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2024-09-13T09:11:15.517 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2024-09-13T09:11:15.517 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2024-09-13T09:11:15.517 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T09:11:15.517 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T09:11:15.517 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2024-09-13T09:11:15.517 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2024-09-13T09:11:15.518 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2024-09-13T09:11:15.519 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T09:11:15.519 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T09:11:15.614 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T09:11:15.614 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:15.614 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T09:11:15.633 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:15.633 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T09:11:15.688 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T09:11:15.688 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:15.689 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T09:11:15.689 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T09:11:15.763 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:15.764 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:11:15.820 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T09:11:15.820 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:11:15.820 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T09:11:15.820 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T09:11:15.860 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:11:15.860 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:11:15.917 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T09:11:15.917 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T09:11:15.917 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T09:11:15.918 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T09:11:15.959 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T09:11:15.959 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:11:16.016 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T09:11:16.016 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T09:11:16.017 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T09:11:16.084 INFO:teuthology.orchestra.run.smithi129.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T09:11:16.087 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T09:11:16.087 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T09:11:16.087 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T09:11:16.139 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T09:11:16.152 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T09:11:16.152 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi129.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-13T09:11:16.152 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T09:11:16.178 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-13T09:11:16.178 INFO:tasks.ceph:role: osd.0 2024-09-13T09:11:16.178 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T09:11:16.178 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T09:11:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:11:16.251 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:11:16.251 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:11:16.251 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T09:11:16.251 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:11:16.251 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:11:16.252 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:11:16.252 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T09:11:16.252 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:11:16.252 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:11:16.755 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-09-13T09:11:16.757 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-09-13T09:11:16.757 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T09:11:16.888 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T09:11:16.921 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T09:11:16.989 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-13T09:11:16.989 INFO:tasks.ceph:role: osd.1 2024-09-13T09:11:16.989 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T09:11:16.989 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T09:11:17.062 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:11:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:11:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:11:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T09:11:17.063 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:11:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:11:17.063 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:11:17.063 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T09:11:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:11:17.063 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:11:17.516 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-09-13T09:11:17.518 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-09-13T09:11:17.518 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T09:11:17.610 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T09:11:17.642 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T09:11:17.709 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-13T09:11:17.709 INFO:tasks.ceph:role: osd.2 2024-09-13T09:11:17.709 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T09:11:17.709 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T09:11:17.782 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:11:17.782 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:11:17.782 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:11:17.783 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T09:11:17.783 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:11:17.783 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:11:17.783 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:11:17.783 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T09:11:17.783 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:11:17.783 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:11:18.266 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-09-13T09:11:18.268 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-09-13T09:11:18.268 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T09:11:18.418 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T09:11:18.450 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:11:18.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:18.551+0000 7f1b1b024640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T09:11:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:18.553+0000 7f1b1b024640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T09:11:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:18.553+0000 7f1b1b024640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T09:11:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:18.553+0000 7f1b1b024640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T09:11:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:18.553+0000 7f1b1b024640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T09:11:18.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:18.560+0000 7f1b1b024640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T09:11:21.418 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T09:11:21.451 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:11:21.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:21.543+0000 7fca4dc0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T09:11:21.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:21.543+0000 7fca4dc0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T09:11:21.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:21.543+0000 7fca4dc0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T09:11:21.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:21.543+0000 7fca4dc0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T09:11:21.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:21.543+0000 7fca4dc0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T09:11:21.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:21.544+0000 7fca4dc0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T09:11:24.393 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T09:11:24.423 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:11:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:24.515+0000 7f7e5812b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T09:11:24.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:24.515+0000 7f7e5812b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T09:11:24.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:24.515+0000 7f7e5812b640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T09:11:24.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:24.515+0000 7f7e5812b640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T09:11:24.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:24.515+0000 7f7e5812b640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T09:11:24.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:24.516+0000 7f7e5812b640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T09:11:27.366 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T09:11:27.395 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T09:11:27.425 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-13T09:11:27.425 INFO:tasks.ceph:role: osd.3 2024-09-13T09:11:27.425 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T09:11:27.425 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T09:11:27.501 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:11:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:11:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:11:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T09:11:27.502 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:11:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:11:27.502 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:11:27.502 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T09:11:27.502 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:11:27.502 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:11:27.999 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-13T09:11:28.001 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-13T09:11:28.002 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-13T09:11:28.153 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T09:11:28.192 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T09:11:28.261 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-13T09:11:28.261 INFO:tasks.ceph:role: osd.4 2024-09-13T09:11:28.261 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T09:11:28.261 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T09:11:28.336 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:11:28.336 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:11:28.336 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:11:28.336 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T09:11:28.336 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:11:28.336 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:11:28.336 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:11:28.336 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T09:11:28.336 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:11:28.336 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:11:28.779 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-13T09:11:28.781 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-13T09:11:28.781 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-13T09:11:28.873 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T09:11:28.901 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T09:11:28.967 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-13T09:11:28.967 INFO:tasks.ceph:role: osd.5 2024-09-13T09:11:28.967 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T09:11:28.967 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T09:11:29.039 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:11:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:11:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:11:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T09:11:29.039 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:11:29.040 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:11:29.040 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:11:29.040 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T09:11:29.040 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:11:29.040 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:11:29.531 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-13T09:11:29.533 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-13T09:11:29.533 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-13T09:11:29.681 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T09:11:29.713 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:11:29.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:29.818+0000 7f3b88bdb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T09:11:29.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:29.820+0000 7f3b88bdb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T09:11:29.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:29.820+0000 7f3b88bdb640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T09:11:29.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:29.820+0000 7f3b88bdb640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T09:11:29.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:29.820+0000 7f3b88bdb640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T09:11:29.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:29.827+0000 7f3b88bdb640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T09:11:32.700 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T09:11:32.729 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:11:32.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:32.817+0000 7f4aabd58640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T09:11:32.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:32.817+0000 7f4aabd58640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T09:11:32.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:32.817+0000 7f4aabd58640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T09:11:32.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:32.817+0000 7f4aabd58640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T09:11:32.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:32.817+0000 7f4aabd58640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T09:11:32.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:32.818+0000 7f4aabd58640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T09:11:35.663 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T09:11:35.692 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:11:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:35.782+0000 7f588a0e2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T09:11:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:35.782+0000 7f588a0e2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T09:11:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:35.783+0000 7f588a0e2640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T09:11:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:35.783+0000 7f588a0e2640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T09:11:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:35.783+0000 7f588a0e2640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T09:11:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-13T09:11:35.783+0000 7f588a0e2640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T09:11:38.635 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T09:11:38.664 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T09:11:38.664 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:38.664 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T09:11:38.696 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:38.696 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T09:11:38.766 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:38.766 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T09:11:38.836 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:11:38.836 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T09:11:38.865 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:11:38.865 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T09:11:38.934 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:11:38.934 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T09:11:39.005 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:11:39.005 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T09:11:39.071 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-13T09:11:39.071 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T09:11:39.091 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T09:11:39.091 DEBUG:teuthology.orchestra.run.smithi044:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T09:11:39.094 DEBUG:teuthology.orchestra.run.smithi129:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T09:11:39.120 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.0] 2024-09-13T09:11:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQC2AeRmsJLxIBAAD9xOWBD9Vy+W0tsmnrOr9Q== 2024-09-13T09:11:39.121 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.1] 2024-09-13T09:11:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQC5AeRmYoJvIBAAns60y+B7vbqz800fJQKpcw== 2024-09-13T09:11:39.121 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.2] 2024-09-13T09:11:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQC8AeRmImfBHhAAiVUYQ73q8qx+S1nuy7505w== 2024-09-13T09:11:39.121 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr.x] 2024-09-13T09:11:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQC0AeRmsegIBRAAcJdU+GQoXLkIwSp26BUkVw== 2024-09-13T09:11:39.121 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.3] 2024-09-13T09:11:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDBAeRmG7rlMBAAM7DVjXAcZQrLllNty8HUXg== 2024-09-13T09:11:39.121 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.4] 2024-09-13T09:11:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDEAeRmqWbOMBAAXvMKizxsq6za+A4ASLL1kw== 2024-09-13T09:11:39.121 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.5] 2024-09-13T09:11:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDHAeRm0wO9LhAAQKq6VnVHezcjyqGJq3f7Iw== 2024-09-13T09:11:39.122 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-09-13T09:11:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQC0AeRm7oNICBAAcqN5w9JstxZPTmI5nTyMgg== 2024-09-13T09:11:39.135 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.0] 2024-09-13T09:11:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQC2AeRmsJLxIBAAD9xOWBD9Vy+W0tsmnrOr9Q== 2024-09-13T09:11:39.135 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.1] 2024-09-13T09:11:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQC5AeRmYoJvIBAAns60y+B7vbqz800fJQKpcw== 2024-09-13T09:11:39.135 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.2] 2024-09-13T09:11:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQC8AeRmImfBHhAAiVUYQ73q8qx+S1nuy7505w== 2024-09-13T09:11:39.135 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr.x] 2024-09-13T09:11:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQC0AeRmsegIBRAAcJdU+GQoXLkIwSp26BUkVw== 2024-09-13T09:11:39.135 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.3] 2024-09-13T09:11:39.135 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDBAeRmG7rlMBAAM7DVjXAcZQrLllNty8HUXg== 2024-09-13T09:11:39.135 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.4] 2024-09-13T09:11:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDEAeRmqWbOMBAAXvMKizxsq6za+A4ASLL1kw== 2024-09-13T09:11:39.136 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.5] 2024-09-13T09:11:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDHAeRm0wO9LhAAQKq6VnVHezcjyqGJq3f7Iw== 2024-09-13T09:11:39.136 INFO:teuthology.orchestra.run.smithi129.stdout:[client.0] 2024-09-13T09:11:39.136 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQC0AeRm7oNICBAAcqN5w9JstxZPTmI5nTyMgg== 2024-09-13T09:11:39.137 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:11:39.165 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:11:39.228 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:11:39.230 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:11:39.284 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:11:39.287 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:11:39.338 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T09:11:39.340 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T09:11:39.391 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:11:39.393 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:11:39.443 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:11:39.445 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:11:39.495 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:11:39.498 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:11:39.551 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T09:11:39.553 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T09:11:39.606 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T09:11:39.607 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T09:11:39.632 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T09:11:39.730 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T09:11:39.755 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T09:11:39.821 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T09:11:39.920 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T09:11:39.946 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T09:11:39.971 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T09:11:40.069 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T09:11:40.093 DEBUG:teuthology.orchestra.run.smithi044:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:11:40.096 DEBUG:teuthology.orchestra.run.smithi129:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:11:40.149 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T09:11:40.149 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T09:11:40.149 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T09:11:40.151 INFO:tasks.ceph.mon.a:Started 2024-09-13T09:11:40.151 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T09:11:40.152 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T09:11:40.153 INFO:tasks.ceph.mon.c:Started 2024-09-13T09:11:40.153 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T09:11:40.154 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T09:11:40.191 INFO:tasks.ceph.mon.b:Started 2024-09-13T09:11:40.191 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T09:11:40.191 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T09:11:40.191 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T09:11:40.193 INFO:tasks.ceph.mgr.x:Started 2024-09-13T09:11:40.194 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T09:11:40.194 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T09:11:45.591 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:45.589+0000 7f6e1d837040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T09:11:45.632 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:45.630+0000 7f6e1d837040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T09:11:45.783 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:45.781+0000 7f6e1d837040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T09:11:45.832 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:45.830+0000 7f6e1d837040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T09:11:45.988 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:45.986+0000 7f6e1d837040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T09:11:46.095 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:46.093+0000 7f6e1d837040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T09:11:46.411 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:46.409+0000 7f6e1d837040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T09:11:46.464 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:46.462+0000 7f6e1d837040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T09:11:46.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:11:46.502+0000 7f40b6ffd640 0 --2- 172.21.15.44:0/1146160283 >> v2:172.21.15.129:3300/0 conn(0x7f40980036e0 0x7f4098005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T09:11:46.582 INFO:teuthology.orchestra.run.smithi044.stderr:adjusted tunables profile to default 2024-09-13T09:11:46.594 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T09:11:46.594 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T09:11:46.595 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:46.595 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T09:11:46.612 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:46.610+0000 7f6e1d837040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T09:11:46.622 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:46.622 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T09:11:46.690 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-09-13T09:11:46.690 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T09:11:46.756 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:11:46.756 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T09:11:46.782 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:11:46.782 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T09:11:46.847 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-13T09:11:46.847 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T09:11:46.911 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new 156c6730-f804-4806-bc2d-3a7d0856c2b2 0 2024-09-13T09:11:47.127 INFO:teuthology.orchestra.run.smithi129.stdout:0 2024-09-13T09:11:47.139 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new 00735025-3bf9-4dd0-8b42-f08f1bd1c421 1 2024-09-13T09:11:47.329 INFO:teuthology.orchestra.run.smithi129.stdout:1 2024-09-13T09:11:47.340 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new 395516ac-33b6-4f43-8333-dcb1f4cded32 2 2024-09-13T09:11:47.522 INFO:teuthology.orchestra.run.smithi129.stdout:2 2024-09-13T09:11:47.533 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new 429e877f-5462-402a-80e5-92a7b17a2eaa 3 2024-09-13T09:11:47.589 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:47.587+0000 7f6e1d837040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T09:11:47.646 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:47.644+0000 7f6e1d837040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T09:11:47.727 INFO:teuthology.orchestra.run.smithi129.stdout:3 2024-09-13T09:11:47.738 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new 9b3be757-15ee-4157-80a8-0537cd1bcf28 4 2024-09-13T09:11:47.918 INFO:teuthology.orchestra.run.smithi129.stdout:4 2024-09-13T09:11:47.929 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new c7601d5e-2164-45e1-ab21-368d9e3af27e 5 2024-09-13T09:11:48.039 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:48.037+0000 7f6e1d837040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T09:11:48.111 INFO:teuthology.orchestra.run.smithi129.stdout:5 2024-09-13T09:11:48.122 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T09:11:48.122 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T09:11:48.125 INFO:tasks.ceph.osd.0:Started 2024-09-13T09:11:48.125 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T09:11:48.126 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T09:11:48.127 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:48.125+0000 7f6e1d837040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T09:11:48.128 INFO:tasks.ceph.osd.1:Started 2024-09-13T09:11:48.128 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T09:11:48.128 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T09:11:48.130 INFO:tasks.ceph.osd.2:Started 2024-09-13T09:11:48.131 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T09:11:48.131 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T09:11:48.133 INFO:tasks.ceph.osd.3:Started 2024-09-13T09:11:48.133 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T09:11:48.133 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T09:11:48.135 INFO:tasks.ceph.osd.4:Started 2024-09-13T09:11:48.135 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T09:11:48.136 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T09:11:48.137 INFO:tasks.ceph.osd.5:Started 2024-09-13T09:11:48.138 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:11:48.246 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:48.244+0000 7f6e1d837040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T09:11:48.292 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:48.290+0000 7f6e1d837040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T09:11:48.324 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:11:48.325 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":8,"fsid":"06cda68b-dd73-4266-b5e6-93ae2b3f7bd5","created":"2024-09-13T09:11:45.458147+0000","modified":"2024-09-13T09:11:48.104369+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T09:11:48.104369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"156c6730-f804-4806-bc2d-3a7d0856c2b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00735025-3bf9-4dd0-8b42-f08f1bd1c421","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"395516ac-33b6-4f43-8333-dcb1f4cded32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"429e877f-5462-402a-80e5-92a7b17a2eaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9b3be757-15ee-4157-80a8-0537cd1bcf28","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c7601d5e-2164-45e1-ab21-368d9e3af27e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:11:48.335 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T09:11:48.335 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T09:11:48.402 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:48.401+0000 7f6e1d837040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T09:11:48.455 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:48.454+0000 7f6e1d837040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T09:11:48.502 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:48.500+0000 7f6e1d837040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T09:11:48.612 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:48.610+0000 7f6e1d837040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T09:11:48.661 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:48.659+0000 7f6e1d837040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T09:11:48.711 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:48.709+0000 7f6e1d837040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T09:11:48.730 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:11:48.728+0000 7fe7a363e640 -1 Falling back to public interface 2024-09-13T09:11:48.731 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:11:48.729+0000 7f2a5cd1e640 -1 Falling back to public interface 2024-09-13T09:11:48.733 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:11:48.732+0000 7f379d7d9640 -1 Falling back to public interface 2024-09-13T09:11:48.734 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:11:48.733+0000 7eff39a0e640 -1 Falling back to public interface 2024-09-13T09:11:48.735 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:11:48.733+0000 7f5858a9c640 -1 Falling back to public interface 2024-09-13T09:11:48.735 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:11:48.733+0000 7fcb44030640 -1 Falling back to public interface 2024-09-13T09:11:48.735 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:11:48.734+0000 7eff38318640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6812/0: (98) Address already in use 2024-09-13T09:11:48.782 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:48.780+0000 7f6e1d837040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T09:11:48.826 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:48.824+0000 7f6e1d837040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T09:11:48.870 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:48.868+0000 7f6e1d837040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T09:11:48.938 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T09:11:49.101 INFO:teuthology.misc.health.smithi044.stdout: 2024-09-13T09:11:49.101 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":8,"fsid":"06cda68b-dd73-4266-b5e6-93ae2b3f7bd5","created":"2024-09-13T09:11:45.458147+0000","modified":"2024-09-13T09:11:48.104369+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T09:11:48.104369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"156c6730-f804-4806-bc2d-3a7d0856c2b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00735025-3bf9-4dd0-8b42-f08f1bd1c421","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"395516ac-33b6-4f43-8333-dcb1f4cded32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"429e877f-5462-402a-80e5-92a7b17a2eaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9b3be757-15ee-4157-80a8-0537cd1bcf28","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c7601d5e-2164-45e1-ab21-368d9e3af27e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:11:49.110 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-13T09:11:49.509 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:49.509+0000 7f6e1d837040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T09:11:50.523 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:50.522+0000 7f6dbda24640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T09:11:50.523 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-13T09:11:50.522+0000 7f6dbda24640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T09:11:52.188 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:11:52.187+0000 7fe7a363e640 -1 osd.4 0 log_to_monitors true 2024-09-13T09:11:52.240 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:11:52.238+0000 7f379d7d9640 -1 osd.1 0 log_to_monitors true 2024-09-13T09:11:52.240 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:11:52.239+0000 7f2a5cd1e640 -1 osd.3 0 log_to_monitors true 2024-09-13T09:11:52.246 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:11:52.245+0000 7eff39a0e640 -1 osd.0 0 log_to_monitors true 2024-09-13T09:11:52.249 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:11:52.249+0000 7fcb44030640 -1 osd.5 0 log_to_monitors true 2024-09-13T09:11:52.254 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:11:52.253+0000 7f5858a9c640 -1 osd.2 0 log_to_monitors true 2024-09-13T09:11:55.264 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:11:55.263+0000 7f2a58488640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T09:11:55.267 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:11:55.266+0000 7fcb3ed4d640 -1 osd.5 0 waiting for initial osdmap 2024-09-13T09:11:55.278 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:11:55.278+0000 7f2a50abb640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:11:55.282 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:11:55.282+0000 7fcb37380640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:11:55.289 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:11:55.288+0000 7f5855a41640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T09:11:55.302 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:11:55.301+0000 7f584c839640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:11:55.328 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:11:55.327+0000 7fe79eda8640 -1 osd.4 0 waiting for initial osdmap 2024-09-13T09:11:55.332 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:11:55.331+0000 7f3798f43640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T09:11:55.346 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:11:55.345+0000 7fe7973db640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:11:55.348 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:11:55.347+0000 7f3791576640 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:11:55.368 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:11:55.367+0000 7eff36314640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T09:11:55.379 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:11:55.378+0000 7eff2d10c640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:11:55.714 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T09:11:55.990 INFO:teuthology.misc.health.smithi044.stdout: 2024-09-13T09:11:55.990 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":10,"fsid":"06cda68b-dd73-4266-b5e6-93ae2b3f7bd5","created":"2024-09-13T09:11:45.458147+0000","modified":"2024-09-13T09:11:53.527253+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T09:11:48.104369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"156c6730-f804-4806-bc2d-3a7d0856c2b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00735025-3bf9-4dd0-8b42-f08f1bd1c421","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"395516ac-33b6-4f43-8333-dcb1f4cded32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"429e877f-5462-402a-80e5-92a7b17a2eaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9b3be757-15ee-4157-80a8-0537cd1bcf28","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c7601d5e-2164-45e1-ab21-368d9e3af27e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:11:56.001 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-13T09:12:02.608 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T09:12:02.894 INFO:teuthology.misc.health.smithi044.stdout: 2024-09-13T09:12:02.894 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":15,"fsid":"06cda68b-dd73-4266-b5e6-93ae2b3f7bd5","created":"2024-09-13T09:11:45.458147+0000","modified":"2024-09-13T09:12:00.286199+0000","last_up_change":"2024-09-13T09:11:56.271910+0000","last_in_change":"2024-09-13T09:11:48.104369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:11:57.562930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"156c6730-f804-4806-bc2d-3a7d0856c2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6810","nonce":2268459392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6815","nonce":2268459392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6823","nonce":2268459392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6817","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6819","nonce":2268459392}]},"public_addr":"172.21.15.44:6810/2268459392","cluster_addr":"172.21.15.44:6815/2268459392","heartbeat_back_addr":"172.21.15.44:6823/2268459392","heartbeat_front_addr":"172.21.15.44:6819/2268459392","state":["exists","up"]},{"osd":1,"uuid":"00735025-3bf9-4dd0-8b42-f08f1bd1c421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6801","nonce":2209612586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6803","nonce":2209612586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6807","nonce":2209612586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6805","nonce":2209612586}]},"public_addr":"172.21.15.44:6801/2209612586","cluster_addr":"172.21.15.44:6803/2209612586","heartbeat_back_addr":"172.21.15.44:6807/2209612586","heartbeat_front_addr":"172.21.15.44:6805/2209612586","state":["exists","up"]},{"osd":2,"uuid":"395516ac-33b6-4f43-8333-dcb1f4cded32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":770926265},{"type":"v1","addr":"172.21.15.44:6811","nonce":770926265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":770926265},{"type":"v1","addr":"172.21.15.44:6814","nonce":770926265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":770926265},{"type":"v1","addr":"172.21.15.44:6822","nonce":770926265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":770926265},{"type":"v1","addr":"172.21.15.44:6818","nonce":770926265}]},"public_addr":"172.21.15.44:6811/770926265","cluster_addr":"172.21.15.44:6814/770926265","heartbeat_back_addr":"172.21.15.44:6822/770926265","heartbeat_front_addr":"172.21.15.44:6818/770926265","state":["exists","up"]},{"osd":3,"uuid":"429e877f-5462-402a-80e5-92a7b17a2eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6809","nonce":2132151186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6811","nonce":2132151186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6815","nonce":2132151186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6813","nonce":2132151186}]},"public_addr":"172.21.15.129:6809/2132151186","cluster_addr":"172.21.15.129:6811/2132151186","heartbeat_back_addr":"172.21.15.129:6815/2132151186","heartbeat_front_addr":"172.21.15.129:6813/2132151186","state":["exists","up"]},{"osd":4,"uuid":"9b3be757-15ee-4157-80a8-0537cd1bcf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393542416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393542416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393542416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393542416}]},"public_addr":"172.21.15.129:6801/3393542416","cluster_addr":"172.21.15.129:6803/3393542416","heartbeat_back_addr":"172.21.15.129:6807/3393542416","heartbeat_front_addr":"172.21.15.129:6805/3393542416","state":["exists","up"]},{"osd":5,"uuid":"c7601d5e-2164-45e1-ab21-368d9e3af27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1803198188},{"type":"v1","addr":"172.21.15.129:6817","nonce":1803198188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1803198188},{"type":"v1","addr":"172.21.15.129:6819","nonce":1803198188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1803198188},{"type":"v1","addr":"172.21.15.129:6823","nonce":1803198188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1803198188},{"type":"v1","addr":"172.21.15.129:6821","nonce":1803198188}]},"public_addr":"172.21.15.129:6817/1803198188","cluster_addr":"172.21.15.129:6819/1803198188","heartbeat_back_addr":"172.21.15.129:6823/1803198188","heartbeat_front_addr":"172.21.15.129:6821/1803198188","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.230474+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.195476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.248729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.195482+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.198746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.279957+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:12:02.905 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-13T09:12:02.905 INFO:tasks.ceph:Creating RBD pool 2024-09-13T09:12:02.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-13T09:12:03.313 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'rbd' created 2024-09-13T09:12:03.326 DEBUG:teuthology.orchestra.run.smithi044:> rbd --cluster ceph pool init rbd 2024-09-13T09:12:06.337 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T09:12:06.338 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T09:12:06.338 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T09:12:06.338 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T09:12:06.339 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T09:12:06.677 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:12:06.688 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":14397013},{"type":"v1","addr":"172.21.15.129:6825","nonce":14397013}]},"active_addr":"172.21.15.129:6825/14397013","active_change":"2024-09-13T09:11:49.511313+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":3980865739}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":1748302019}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":2266791574}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":920882530}]}]} 2024-09-13T09:12:06.690 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T09:12:06.690 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T09:12:06.691 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:12:06.976 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:12:06.977 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":19,"fsid":"06cda68b-dd73-4266-b5e6-93ae2b3f7bd5","created":"2024-09-13T09:11:45.458147+0000","modified":"2024-09-13T09:12:06.321409+0000","last_up_change":"2024-09-13T09:11:56.271910+0000","last_in_change":"2024-09-13T09:11:48.104369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:11:57.562930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T09:12:03.214789+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"156c6730-f804-4806-bc2d-3a7d0856c2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6810","nonce":2268459392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6815","nonce":2268459392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6823","nonce":2268459392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6817","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6819","nonce":2268459392}]},"public_addr":"172.21.15.44:6810/2268459392","cluster_addr":"172.21.15.44:6815/2268459392","heartbeat_back_addr":"172.21.15.44:6823/2268459392","heartbeat_front_addr":"172.21.15.44:6819/2268459392","state":["exists","up"]},{"osd":1,"uuid":"00735025-3bf9-4dd0-8b42-f08f1bd1c421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6801","nonce":2209612586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6803","nonce":2209612586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6807","nonce":2209612586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6805","nonce":2209612586}]},"public_addr":"172.21.15.44:6801/2209612586","cluster_addr":"172.21.15.44:6803/2209612586","heartbeat_back_addr":"172.21.15.44:6807/2209612586","heartbeat_front_addr":"172.21.15.44:6805/2209612586","state":["exists","up"]},{"osd":2,"uuid":"395516ac-33b6-4f43-8333-dcb1f4cded32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":770926265},{"type":"v1","addr":"172.21.15.44:6811","nonce":770926265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":770926265},{"type":"v1","addr":"172.21.15.44:6814","nonce":770926265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":770926265},{"type":"v1","addr":"172.21.15.44:6822","nonce":770926265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":770926265},{"type":"v1","addr":"172.21.15.44:6818","nonce":770926265}]},"public_addr":"172.21.15.44:6811/770926265","cluster_addr":"172.21.15.44:6814/770926265","heartbeat_back_addr":"172.21.15.44:6822/770926265","heartbeat_front_addr":"172.21.15.44:6818/770926265","state":["exists","up"]},{"osd":3,"uuid":"429e877f-5462-402a-80e5-92a7b17a2eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6809","nonce":2132151186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6811","nonce":2132151186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6815","nonce":2132151186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6813","nonce":2132151186}]},"public_addr":"172.21.15.129:6809/2132151186","cluster_addr":"172.21.15.129:6811/2132151186","heartbeat_back_addr":"172.21.15.129:6815/2132151186","heartbeat_front_addr":"172.21.15.129:6813/2132151186","state":["exists","up"]},{"osd":4,"uuid":"9b3be757-15ee-4157-80a8-0537cd1bcf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393542416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393542416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393542416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393542416}]},"public_addr":"172.21.15.129:6801/3393542416","cluster_addr":"172.21.15.129:6803/3393542416","heartbeat_back_addr":"172.21.15.129:6807/3393542416","heartbeat_front_addr":"172.21.15.129:6805/3393542416","state":["exists","up"]},{"osd":5,"uuid":"c7601d5e-2164-45e1-ab21-368d9e3af27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1803198188},{"type":"v1","addr":"172.21.15.129:6817","nonce":1803198188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1803198188},{"type":"v1","addr":"172.21.15.129:6819","nonce":1803198188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1803198188},{"type":"v1","addr":"172.21.15.129:6823","nonce":1803198188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1803198188},{"type":"v1","addr":"172.21.15.129:6821","nonce":1803198188}]},"public_addr":"172.21.15.129:6817/1803198188","cluster_addr":"172.21.15.129:6819/1803198188","heartbeat_back_addr":"172.21.15.129:6823/1803198188","heartbeat_front_addr":"172.21.15.129:6821/1803198188","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.230474+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.195476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.248729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.195482+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.198746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.279957+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:12:06.989 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T09:12:06.989 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:12:07.270 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:12:07.270 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":19,"fsid":"06cda68b-dd73-4266-b5e6-93ae2b3f7bd5","created":"2024-09-13T09:11:45.458147+0000","modified":"2024-09-13T09:12:06.321409+0000","last_up_change":"2024-09-13T09:11:56.271910+0000","last_in_change":"2024-09-13T09:11:48.104369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:11:57.562930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T09:12:03.214789+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"156c6730-f804-4806-bc2d-3a7d0856c2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6810","nonce":2268459392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6815","nonce":2268459392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6823","nonce":2268459392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6817","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6819","nonce":2268459392}]},"public_addr":"172.21.15.44:6810/2268459392","cluster_addr":"172.21.15.44:6815/2268459392","heartbeat_back_addr":"172.21.15.44:6823/2268459392","heartbeat_front_addr":"172.21.15.44:6819/2268459392","state":["exists","up"]},{"osd":1,"uuid":"00735025-3bf9-4dd0-8b42-f08f1bd1c421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6801","nonce":2209612586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6803","nonce":2209612586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6807","nonce":2209612586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6805","nonce":2209612586}]},"public_addr":"172.21.15.44:6801/2209612586","cluster_addr":"172.21.15.44:6803/2209612586","heartbeat_back_addr":"172.21.15.44:6807/2209612586","heartbeat_front_addr":"172.21.15.44:6805/2209612586","state":["exists","up"]},{"osd":2,"uuid":"395516ac-33b6-4f43-8333-dcb1f4cded32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":770926265},{"type":"v1","addr":"172.21.15.44:6811","nonce":770926265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":770926265},{"type":"v1","addr":"172.21.15.44:6814","nonce":770926265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":770926265},{"type":"v1","addr":"172.21.15.44:6822","nonce":770926265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":770926265},{"type":"v1","addr":"172.21.15.44:6818","nonce":770926265}]},"public_addr":"172.21.15.44:6811/770926265","cluster_addr":"172.21.15.44:6814/770926265","heartbeat_back_addr":"172.21.15.44:6822/770926265","heartbeat_front_addr":"172.21.15.44:6818/770926265","state":["exists","up"]},{"osd":3,"uuid":"429e877f-5462-402a-80e5-92a7b17a2eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6809","nonce":2132151186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6811","nonce":2132151186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6815","nonce":2132151186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6813","nonce":2132151186}]},"public_addr":"172.21.15.129:6809/2132151186","cluster_addr":"172.21.15.129:6811/2132151186","heartbeat_back_addr":"172.21.15.129:6815/2132151186","heartbeat_front_addr":"172.21.15.129:6813/2132151186","state":["exists","up"]},{"osd":4,"uuid":"9b3be757-15ee-4157-80a8-0537cd1bcf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393542416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393542416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393542416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393542416}]},"public_addr":"172.21.15.129:6801/3393542416","cluster_addr":"172.21.15.129:6803/3393542416","heartbeat_back_addr":"172.21.15.129:6807/3393542416","heartbeat_front_addr":"172.21.15.129:6805/3393542416","state":["exists","up"]},{"osd":5,"uuid":"c7601d5e-2164-45e1-ab21-368d9e3af27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1803198188},{"type":"v1","addr":"172.21.15.129:6817","nonce":1803198188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1803198188},{"type":"v1","addr":"172.21.15.129:6819","nonce":1803198188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1803198188},{"type":"v1","addr":"172.21.15.129:6823","nonce":1803198188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1803198188},{"type":"v1","addr":"172.21.15.129:6821","nonce":1803198188}]},"public_addr":"172.21.15.129:6817/1803198188","cluster_addr":"172.21.15.129:6819/1803198188","heartbeat_back_addr":"172.21.15.129:6823/1803198188","heartbeat_front_addr":"172.21.15.129:6821/1803198188","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.230474+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.195476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.248729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.195482+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.198746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.279957+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:12:07.283 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T09:12:07.283 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T09:12:07.283 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T09:12:07.284 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T09:12:07.284 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T09:12:07.285 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T09:12:07.443 INFO:teuthology.orchestra.run.smithi044.stdout:47244640262 2024-09-13T09:12:07.444 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:12:07.445 INFO:teuthology.orchestra.run.smithi044.stdout:47244640261 2024-09-13T09:12:07.445 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:12:07.500 INFO:teuthology.orchestra.run.smithi044.stdout:47244640263 2024-09-13T09:12:07.500 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:12:07.504 INFO:teuthology.orchestra.run.smithi044.stdout:47244640262 2024-09-13T09:12:07.505 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:12:07.506 INFO:teuthology.orchestra.run.smithi044.stdout:47244640261 2024-09-13T09:12:07.506 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:12:07.508 INFO:teuthology.orchestra.run.smithi044.stdout:47244640261 2024-09-13T09:12:07.508 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:12:07.815 INFO:teuthology.orchestra.run.smithi044.stdout:47244640260 2024-09-13T09:12:07.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640260 for osd.4 2024-09-13T09:12:07.859 INFO:teuthology.orchestra.run.smithi044.stdout:47244640261 2024-09-13T09:12:07.872 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640263 got 47244640261 for osd.3 2024-09-13T09:12:07.883 INFO:teuthology.orchestra.run.smithi044.stdout:47244640259 2024-09-13T09:12:07.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.0 2024-09-13T09:12:07.953 INFO:teuthology.orchestra.run.smithi044.stdout:47244640259 2024-09-13T09:12:07.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.2 2024-09-13T09:12:07.995 INFO:teuthology.orchestra.run.smithi044.stdout:47244640259 2024-09-13T09:12:08.005 INFO:teuthology.orchestra.run.smithi044.stdout:47244640260 2024-09-13T09:12:08.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.5 2024-09-13T09:12:08.017 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640260 for osd.1 2024-09-13T09:12:08.830 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:12:08.873 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:12:08.897 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:12:08.967 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:12:09.008 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:12:09.018 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:12:09.258 INFO:teuthology.orchestra.run.smithi044.stdout:47244640262 2024-09-13T09:12:09.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.4 2024-09-13T09:12:09.277 DEBUG:teuthology.parallel:result is None 2024-09-13T09:12:09.314 INFO:teuthology.orchestra.run.smithi044.stdout:47244640261 2024-09-13T09:12:09.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.2 2024-09-13T09:12:09.327 DEBUG:teuthology.parallel:result is None 2024-09-13T09:12:09.341 INFO:teuthology.orchestra.run.smithi044.stdout:47244640261 2024-09-13T09:12:09.343 INFO:teuthology.orchestra.run.smithi044.stdout:47244640263 2024-09-13T09:12:09.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.0 2024-09-13T09:12:09.355 DEBUG:teuthology.parallel:result is None 2024-09-13T09:12:09.357 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640263 got 47244640263 for osd.3 2024-09-13T09:12:09.357 DEBUG:teuthology.parallel:result is None 2024-09-13T09:12:09.362 INFO:teuthology.orchestra.run.smithi044.stdout:47244640262 2024-09-13T09:12:09.375 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.1 2024-09-13T09:12:09.375 DEBUG:teuthology.parallel:result is None 2024-09-13T09:12:09.426 INFO:teuthology.orchestra.run.smithi044.stdout:47244640261 2024-09-13T09:12:09.439 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.5 2024-09-13T09:12:09.439 DEBUG:teuthology.parallel:result is None 2024-09-13T09:12:09.439 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T09:12:09.439 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:12:09.720 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:12:09.720 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:12:09.733 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T09:12:09.519296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164356,"kb_used_data":2224,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066684,"statfs":{"total":579820584960,"available":579652284416,"internally_reserved":0,"allocated":2277376,"data_stored":1544134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9563,"internal_metadata":165927589},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001073"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.845582+0000","last_change":"2024-09-13T09:12:06.845781+0000","last_active":"2024-09-13T09:12:06.845582+0000","last_peered":"2024-09-13T09:12:06.845582+0000","last_clean":"2024-09-13T09:12:06.845582+0000","last_became_active":"2024-09-13T09:12:04.446020+0000","last_became_peered":"2024-09-13T09:12:04.446020+0000","last_unstale":"2024-09-13T09:12:06.845582+0000","last_undegraded":"2024-09-13T09:12:06.845582+0000","last_fullsized":"2024-09-13T09:12:06.845582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:19.313225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459779,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.328128+0000","last_change":"2024-09-13T09:12:06.328128+0000","last_active":"2024-09-13T09:12:06.328128+0000","last_peered":"2024-09-13T09:12:06.328128+0000","last_clean":"2024-09-13T09:12:06.328128+0000","last_became_active":"2024-09-13T09:12:04.322523+0000","last_became_peered":"2024-09-13T09:12:04.322523+0000","last_unstale":"2024-09-13T09:12:06.328128+0000","last_undegraded":"2024-09-13T09:12:06.328128+0000","last_fullsized":"2024-09-13T09:12:06.328128+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:29.635790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.845379+0000","last_change":"2024-09-13T09:12:06.845610+0000","last_active":"2024-09-13T09:12:06.845379+0000","last_peered":"2024-09-13T09:12:06.845379+0000","last_clean":"2024-09-13T09:12:06.845379+0000","last_became_active":"2024-09-13T09:12:04.446509+0000","last_became_peered":"2024-09-13T09:12:04.446509+0000","last_unstale":"2024-09-13T09:12:06.845379+0000","last_undegraded":"2024-09-13T09:12:06.845379+0000","last_fullsized":"2024-09-13T09:12:06.845379+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:42:46.166511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042617099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.328091+0000","last_change":"2024-09-13T09:12:06.328091+0000","last_active":"2024-09-13T09:12:06.328091+0000","last_peered":"2024-09-13T09:12:06.328091+0000","last_clean":"2024-09-13T09:12:06.328091+0000","last_became_active":"2024-09-13T09:12:04.446508+0000","last_became_peered":"2024-09-13T09:12:04.446508+0000","last_unstale":"2024-09-13T09:12:06.328091+0000","last_undegraded":"2024-09-13T09:12:06.328091+0000","last_fullsized":"2024-09-13T09:12:06.328091+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:33.410873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042514000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.331246+0000","last_change":"2024-09-13T09:12:06.330649+0000","last_active":"2024-09-13T09:12:06.331246+0000","last_peered":"2024-09-13T09:12:06.331246+0000","last_clean":"2024-09-13T09:12:06.331246+0000","last_became_active":"2024-09-13T09:12:04.325470+0000","last_became_peered":"2024-09-13T09:12:04.325470+0000","last_unstale":"2024-09-13T09:12:06.331246+0000","last_undegraded":"2024-09-13T09:12:06.331246+0000","last_fullsized":"2024-09-13T09:12:06.331246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:37.971615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093634899999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.844901+0000","last_change":"2024-09-13T09:12:06.845125+0000","last_active":"2024-09-13T09:12:06.844901+0000","last_peered":"2024-09-13T09:12:06.844901+0000","last_clean":"2024-09-13T09:12:06.844901+0000","last_became_active":"2024-09-13T09:12:04.321941+0000","last_became_peered":"2024-09-13T09:12:04.321941+0000","last_unstale":"2024-09-13T09:12:06.844901+0000","last_undegraded":"2024-09-13T09:12:06.844901+0000","last_fullsized":"2024-09-13T09:12:06.844901+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:57.706060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031569600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.845327+0000","last_change":"2024-09-13T09:12:06.845534+0000","last_active":"2024-09-13T09:12:06.845327+0000","last_peered":"2024-09-13T09:12:06.845327+0000","last_clean":"2024-09-13T09:12:06.845327+0000","last_became_active":"2024-09-13T09:12:04.322006+0000","last_became_peered":"2024-09-13T09:12:04.322006+0000","last_unstale":"2024-09-13T09:12:06.845327+0000","last_undegraded":"2024-09-13T09:12:06.845327+0000","last_fullsized":"2024-09-13T09:12:06.845327+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:50.637458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038494700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.330140+0000","last_change":"2024-09-13T09:12:06.330140+0000","last_active":"2024-09-13T09:12:06.330140+0000","last_peered":"2024-09-13T09:12:06.330140+0000","last_clean":"2024-09-13T09:12:06.330140+0000","last_became_active":"2024-09-13T09:12:04.324857+0000","last_became_peered":"2024-09-13T09:12:04.324857+0000","last_unstale":"2024-09-13T09:12:06.330140+0000","last_undegraded":"2024-09-13T09:12:06.330140+0000","last_fullsized":"2024-09-13T09:12:06.330140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:57.400286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038911000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T09:12:05.551692+0000","last_change":"2024-09-13T09:12:00.219034+0000","last_active":"2024-09-13T09:12:05.551692+0000","last_peered":"2024-09-13T09:12:05.551692+0000","last_clean":"2024-09-13T09:12:05.551692+0000","last_became_active":"2024-09-13T09:12:00.218065+0000","last_became_peered":"2024-09-13T09:12:00.218065+0000","last_unstale":"2024-09-13T09:12:05.551692+0000","last_undegraded":"2024-09-13T09:12:05.551692+0000","last_fullsized":"2024-09-13T09:12:05.551692+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:43.259310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":60579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":60560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":3,"up_from":11,"seq":47244640263,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":774144,"data_stored":650928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":60560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]}]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":60579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]}]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":650928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:12:09.734 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:12:10.014 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:12:10.014 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:12:10.026 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T09:12:09.519296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164356,"kb_used_data":2224,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066684,"statfs":{"total":579820584960,"available":579652284416,"internally_reserved":0,"allocated":2277376,"data_stored":1544134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9563,"internal_metadata":165927589},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001073"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.845582+0000","last_change":"2024-09-13T09:12:06.845781+0000","last_active":"2024-09-13T09:12:06.845582+0000","last_peered":"2024-09-13T09:12:06.845582+0000","last_clean":"2024-09-13T09:12:06.845582+0000","last_became_active":"2024-09-13T09:12:04.446020+0000","last_became_peered":"2024-09-13T09:12:04.446020+0000","last_unstale":"2024-09-13T09:12:06.845582+0000","last_undegraded":"2024-09-13T09:12:06.845582+0000","last_fullsized":"2024-09-13T09:12:06.845582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:19.313225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459779,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.328128+0000","last_change":"2024-09-13T09:12:06.328128+0000","last_active":"2024-09-13T09:12:06.328128+0000","last_peered":"2024-09-13T09:12:06.328128+0000","last_clean":"2024-09-13T09:12:06.328128+0000","last_became_active":"2024-09-13T09:12:04.322523+0000","last_became_peered":"2024-09-13T09:12:04.322523+0000","last_unstale":"2024-09-13T09:12:06.328128+0000","last_undegraded":"2024-09-13T09:12:06.328128+0000","last_fullsized":"2024-09-13T09:12:06.328128+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:29.635790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.845379+0000","last_change":"2024-09-13T09:12:06.845610+0000","last_active":"2024-09-13T09:12:06.845379+0000","last_peered":"2024-09-13T09:12:06.845379+0000","last_clean":"2024-09-13T09:12:06.845379+0000","last_became_active":"2024-09-13T09:12:04.446509+0000","last_became_peered":"2024-09-13T09:12:04.446509+0000","last_unstale":"2024-09-13T09:12:06.845379+0000","last_undegraded":"2024-09-13T09:12:06.845379+0000","last_fullsized":"2024-09-13T09:12:06.845379+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:42:46.166511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042617099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.328091+0000","last_change":"2024-09-13T09:12:06.328091+0000","last_active":"2024-09-13T09:12:06.328091+0000","last_peered":"2024-09-13T09:12:06.328091+0000","last_clean":"2024-09-13T09:12:06.328091+0000","last_became_active":"2024-09-13T09:12:04.446508+0000","last_became_peered":"2024-09-13T09:12:04.446508+0000","last_unstale":"2024-09-13T09:12:06.328091+0000","last_undegraded":"2024-09-13T09:12:06.328091+0000","last_fullsized":"2024-09-13T09:12:06.328091+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:33.410873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042514000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.331246+0000","last_change":"2024-09-13T09:12:06.330649+0000","last_active":"2024-09-13T09:12:06.331246+0000","last_peered":"2024-09-13T09:12:06.331246+0000","last_clean":"2024-09-13T09:12:06.331246+0000","last_became_active":"2024-09-13T09:12:04.325470+0000","last_became_peered":"2024-09-13T09:12:04.325470+0000","last_unstale":"2024-09-13T09:12:06.331246+0000","last_undegraded":"2024-09-13T09:12:06.331246+0000","last_fullsized":"2024-09-13T09:12:06.331246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:37.971615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093634899999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.844901+0000","last_change":"2024-09-13T09:12:06.845125+0000","last_active":"2024-09-13T09:12:06.844901+0000","last_peered":"2024-09-13T09:12:06.844901+0000","last_clean":"2024-09-13T09:12:06.844901+0000","last_became_active":"2024-09-13T09:12:04.321941+0000","last_became_peered":"2024-09-13T09:12:04.321941+0000","last_unstale":"2024-09-13T09:12:06.844901+0000","last_undegraded":"2024-09-13T09:12:06.844901+0000","last_fullsized":"2024-09-13T09:12:06.844901+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:57.706060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031569600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.845327+0000","last_change":"2024-09-13T09:12:06.845534+0000","last_active":"2024-09-13T09:12:06.845327+0000","last_peered":"2024-09-13T09:12:06.845327+0000","last_clean":"2024-09-13T09:12:06.845327+0000","last_became_active":"2024-09-13T09:12:04.322006+0000","last_became_peered":"2024-09-13T09:12:04.322006+0000","last_unstale":"2024-09-13T09:12:06.845327+0000","last_undegraded":"2024-09-13T09:12:06.845327+0000","last_fullsized":"2024-09-13T09:12:06.845327+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:50.637458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038494700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.330140+0000","last_change":"2024-09-13T09:12:06.330140+0000","last_active":"2024-09-13T09:12:06.330140+0000","last_peered":"2024-09-13T09:12:06.330140+0000","last_clean":"2024-09-13T09:12:06.330140+0000","last_became_active":"2024-09-13T09:12:04.324857+0000","last_became_peered":"2024-09-13T09:12:04.324857+0000","last_unstale":"2024-09-13T09:12:06.330140+0000","last_undegraded":"2024-09-13T09:12:06.330140+0000","last_fullsized":"2024-09-13T09:12:06.330140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:57.400286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038911000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T09:12:05.551692+0000","last_change":"2024-09-13T09:12:00.219034+0000","last_active":"2024-09-13T09:12:05.551692+0000","last_peered":"2024-09-13T09:12:05.551692+0000","last_clean":"2024-09-13T09:12:05.551692+0000","last_became_active":"2024-09-13T09:12:00.218065+0000","last_became_peered":"2024-09-13T09:12:00.218065+0000","last_unstale":"2024-09-13T09:12:05.551692+0000","last_undegraded":"2024-09-13T09:12:05.551692+0000","last_fullsized":"2024-09-13T09:12:05.551692+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:43.259310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":60579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":60560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":3,"up_from":11,"seq":47244640263,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":774144,"data_stored":650928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":60560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]}]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":60579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]}]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":650928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:12:10.027 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T09:12:10.027 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T09:12:10.027 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T09:12:10.028 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T09:12:10.346 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:12:10.346 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T09:12:10.359 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T09:12:10.359 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-13T09:12:10.372 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-13T09:12:10.372 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}} 2024-09-13T09:12:10.372 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-13T09:12:10.373 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-13T09:12:10.674 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-13T09:12:10.674 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T09:12:10.674 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:12:10.956 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:12:10.957 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:12:10.968 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-13T09:12:10.237777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164356,"kb_used_data":2224,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066684,"statfs":{"total":579820584960,"available":579652284416,"internally_reserved":0,"allocated":2277376,"data_stored":1544134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9563,"internal_metadata":165927589},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.921160"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.845582+0000","last_change":"2024-09-13T09:12:06.845781+0000","last_active":"2024-09-13T09:12:06.845582+0000","last_peered":"2024-09-13T09:12:06.845582+0000","last_clean":"2024-09-13T09:12:06.845582+0000","last_became_active":"2024-09-13T09:12:04.446020+0000","last_became_peered":"2024-09-13T09:12:04.446020+0000","last_unstale":"2024-09-13T09:12:06.845582+0000","last_undegraded":"2024-09-13T09:12:06.845582+0000","last_fullsized":"2024-09-13T09:12:06.845582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:19.313225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459779,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.328128+0000","last_change":"2024-09-13T09:12:06.328128+0000","last_active":"2024-09-13T09:12:06.328128+0000","last_peered":"2024-09-13T09:12:06.328128+0000","last_clean":"2024-09-13T09:12:06.328128+0000","last_became_active":"2024-09-13T09:12:04.322523+0000","last_became_peered":"2024-09-13T09:12:04.322523+0000","last_unstale":"2024-09-13T09:12:06.328128+0000","last_undegraded":"2024-09-13T09:12:06.328128+0000","last_fullsized":"2024-09-13T09:12:06.328128+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:29.635790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.845379+0000","last_change":"2024-09-13T09:12:06.845610+0000","last_active":"2024-09-13T09:12:06.845379+0000","last_peered":"2024-09-13T09:12:06.845379+0000","last_clean":"2024-09-13T09:12:06.845379+0000","last_became_active":"2024-09-13T09:12:04.446509+0000","last_became_peered":"2024-09-13T09:12:04.446509+0000","last_unstale":"2024-09-13T09:12:06.845379+0000","last_undegraded":"2024-09-13T09:12:06.845379+0000","last_fullsized":"2024-09-13T09:12:06.845379+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:42:46.166511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042617099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.328091+0000","last_change":"2024-09-13T09:12:06.328091+0000","last_active":"2024-09-13T09:12:06.328091+0000","last_peered":"2024-09-13T09:12:06.328091+0000","last_clean":"2024-09-13T09:12:06.328091+0000","last_became_active":"2024-09-13T09:12:04.446508+0000","last_became_peered":"2024-09-13T09:12:04.446508+0000","last_unstale":"2024-09-13T09:12:06.328091+0000","last_undegraded":"2024-09-13T09:12:06.328091+0000","last_fullsized":"2024-09-13T09:12:06.328091+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:33.410873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042514000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.331246+0000","last_change":"2024-09-13T09:12:06.330649+0000","last_active":"2024-09-13T09:12:06.331246+0000","last_peered":"2024-09-13T09:12:06.331246+0000","last_clean":"2024-09-13T09:12:06.331246+0000","last_became_active":"2024-09-13T09:12:04.325470+0000","last_became_peered":"2024-09-13T09:12:04.325470+0000","last_unstale":"2024-09-13T09:12:06.331246+0000","last_undegraded":"2024-09-13T09:12:06.331246+0000","last_fullsized":"2024-09-13T09:12:06.331246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:37.971615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093634899999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.844901+0000","last_change":"2024-09-13T09:12:06.845125+0000","last_active":"2024-09-13T09:12:06.844901+0000","last_peered":"2024-09-13T09:12:06.844901+0000","last_clean":"2024-09-13T09:12:06.844901+0000","last_became_active":"2024-09-13T09:12:04.321941+0000","last_became_peered":"2024-09-13T09:12:04.321941+0000","last_unstale":"2024-09-13T09:12:06.844901+0000","last_undegraded":"2024-09-13T09:12:06.844901+0000","last_fullsized":"2024-09-13T09:12:06.844901+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:57.706060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031569600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.845327+0000","last_change":"2024-09-13T09:12:06.845534+0000","last_active":"2024-09-13T09:12:06.845327+0000","last_peered":"2024-09-13T09:12:06.845327+0000","last_clean":"2024-09-13T09:12:06.845327+0000","last_became_active":"2024-09-13T09:12:04.322006+0000","last_became_peered":"2024-09-13T09:12:04.322006+0000","last_unstale":"2024-09-13T09:12:06.845327+0000","last_undegraded":"2024-09-13T09:12:06.845327+0000","last_fullsized":"2024-09-13T09:12:06.845327+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:50.637458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038494700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.330140+0000","last_change":"2024-09-13T09:12:06.330140+0000","last_active":"2024-09-13T09:12:06.330140+0000","last_peered":"2024-09-13T09:12:06.330140+0000","last_clean":"2024-09-13T09:12:06.330140+0000","last_became_active":"2024-09-13T09:12:04.324857+0000","last_became_peered":"2024-09-13T09:12:04.324857+0000","last_unstale":"2024-09-13T09:12:06.330140+0000","last_undegraded":"2024-09-13T09:12:06.330140+0000","last_fullsized":"2024-09-13T09:12:06.330140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:57.400286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038911000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T09:12:05.551692+0000","last_change":"2024-09-13T09:12:00.219034+0000","last_active":"2024-09-13T09:12:05.551692+0000","last_peered":"2024-09-13T09:12:05.551692+0000","last_clean":"2024-09-13T09:12:05.551692+0000","last_became_active":"2024-09-13T09:12:00.218065+0000","last_became_peered":"2024-09-13T09:12:00.218065+0000","last_unstale":"2024-09-13T09:12:05.551692+0000","last_undegraded":"2024-09-13T09:12:05.551692+0000","last_fullsized":"2024-09-13T09:12:05.551692+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:43.259310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":60579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":60560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":3,"up_from":11,"seq":47244640263,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":774144,"data_stored":650928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":60560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]}]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":60579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]}]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":650928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:12:10.969 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:12:11.249 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:12:11.250 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:12:11.262 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-13T09:12:10.237777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164356,"kb_used_data":2224,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066684,"statfs":{"total":579820584960,"available":579652284416,"internally_reserved":0,"allocated":2277376,"data_stored":1544134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9563,"internal_metadata":165927589},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.921160"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.845582+0000","last_change":"2024-09-13T09:12:06.845781+0000","last_active":"2024-09-13T09:12:06.845582+0000","last_peered":"2024-09-13T09:12:06.845582+0000","last_clean":"2024-09-13T09:12:06.845582+0000","last_became_active":"2024-09-13T09:12:04.446020+0000","last_became_peered":"2024-09-13T09:12:04.446020+0000","last_unstale":"2024-09-13T09:12:06.845582+0000","last_undegraded":"2024-09-13T09:12:06.845582+0000","last_fullsized":"2024-09-13T09:12:06.845582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:19.313225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459779,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.328128+0000","last_change":"2024-09-13T09:12:06.328128+0000","last_active":"2024-09-13T09:12:06.328128+0000","last_peered":"2024-09-13T09:12:06.328128+0000","last_clean":"2024-09-13T09:12:06.328128+0000","last_became_active":"2024-09-13T09:12:04.322523+0000","last_became_peered":"2024-09-13T09:12:04.322523+0000","last_unstale":"2024-09-13T09:12:06.328128+0000","last_undegraded":"2024-09-13T09:12:06.328128+0000","last_fullsized":"2024-09-13T09:12:06.328128+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:29.635790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.845379+0000","last_change":"2024-09-13T09:12:06.845610+0000","last_active":"2024-09-13T09:12:06.845379+0000","last_peered":"2024-09-13T09:12:06.845379+0000","last_clean":"2024-09-13T09:12:06.845379+0000","last_became_active":"2024-09-13T09:12:04.446509+0000","last_became_peered":"2024-09-13T09:12:04.446509+0000","last_unstale":"2024-09-13T09:12:06.845379+0000","last_undegraded":"2024-09-13T09:12:06.845379+0000","last_fullsized":"2024-09-13T09:12:06.845379+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:42:46.166511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042617099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.328091+0000","last_change":"2024-09-13T09:12:06.328091+0000","last_active":"2024-09-13T09:12:06.328091+0000","last_peered":"2024-09-13T09:12:06.328091+0000","last_clean":"2024-09-13T09:12:06.328091+0000","last_became_active":"2024-09-13T09:12:04.446508+0000","last_became_peered":"2024-09-13T09:12:04.446508+0000","last_unstale":"2024-09-13T09:12:06.328091+0000","last_undegraded":"2024-09-13T09:12:06.328091+0000","last_fullsized":"2024-09-13T09:12:06.328091+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:33.410873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042514000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.331246+0000","last_change":"2024-09-13T09:12:06.330649+0000","last_active":"2024-09-13T09:12:06.331246+0000","last_peered":"2024-09-13T09:12:06.331246+0000","last_clean":"2024-09-13T09:12:06.331246+0000","last_became_active":"2024-09-13T09:12:04.325470+0000","last_became_peered":"2024-09-13T09:12:04.325470+0000","last_unstale":"2024-09-13T09:12:06.331246+0000","last_undegraded":"2024-09-13T09:12:06.331246+0000","last_fullsized":"2024-09-13T09:12:06.331246+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:37.971615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093634899999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.844901+0000","last_change":"2024-09-13T09:12:06.845125+0000","last_active":"2024-09-13T09:12:06.844901+0000","last_peered":"2024-09-13T09:12:06.844901+0000","last_clean":"2024-09-13T09:12:06.844901+0000","last_became_active":"2024-09-13T09:12:04.321941+0000","last_became_peered":"2024-09-13T09:12:04.321941+0000","last_unstale":"2024-09-13T09:12:06.844901+0000","last_undegraded":"2024-09-13T09:12:06.844901+0000","last_fullsized":"2024-09-13T09:12:06.844901+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:57.706060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031569600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.845327+0000","last_change":"2024-09-13T09:12:06.845534+0000","last_active":"2024-09-13T09:12:06.845327+0000","last_peered":"2024-09-13T09:12:06.845327+0000","last_clean":"2024-09-13T09:12:06.845327+0000","last_became_active":"2024-09-13T09:12:04.322006+0000","last_became_peered":"2024-09-13T09:12:04.322006+0000","last_unstale":"2024-09-13T09:12:06.845327+0000","last_undegraded":"2024-09-13T09:12:06.845327+0000","last_fullsized":"2024-09-13T09:12:06.845327+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:50.637458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038494700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:12:06.330140+0000","last_change":"2024-09-13T09:12:06.330140+0000","last_active":"2024-09-13T09:12:06.330140+0000","last_peered":"2024-09-13T09:12:06.330140+0000","last_clean":"2024-09-13T09:12:06.330140+0000","last_became_active":"2024-09-13T09:12:04.324857+0000","last_became_peered":"2024-09-13T09:12:04.324857+0000","last_unstale":"2024-09-13T09:12:06.330140+0000","last_undegraded":"2024-09-13T09:12:06.330140+0000","last_fullsized":"2024-09-13T09:12:06.330140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:57.400286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038911000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T09:12:05.551692+0000","last_change":"2024-09-13T09:12:00.219034+0000","last_active":"2024-09-13T09:12:05.551692+0000","last_peered":"2024-09-13T09:12:05.551692+0000","last_clean":"2024-09-13T09:12:05.551692+0000","last_became_active":"2024-09-13T09:12:00.218065+0000","last_became_peered":"2024-09-13T09:12:00.218065+0000","last_unstale":"2024-09-13T09:12:05.551692+0000","last_undegraded":"2024-09-13T09:12:05.551692+0000","last_fullsized":"2024-09-13T09:12:05.551692+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:43.259310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":60579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":60560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":3,"up_from":11,"seq":47244640263,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":774144,"data_stored":650928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":60560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]}]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":60579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]}]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":650928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:12:11.263 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T09:12:11.263 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-13T09:12:11.548 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 20 2024-09-13T09:12:11.548 INFO:teuthology.orchestra.run.smithi044.stdout:fsid 06cda68b-dd73-4266-b5e6-93ae2b3f7bd5 2024-09-13T09:12:11.548 INFO:teuthology.orchestra.run.smithi044.stdout:created 2024-09-13T09:11:45.458147+0000 2024-09-13T09:12:11.548 INFO:teuthology.orchestra.run.smithi044.stdout:modified 2024-09-13T09:12:10.235771+0000 2024-09-13T09:12:11.548 INFO:teuthology.orchestra.run.smithi044.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-13T09:12:11.548 INFO:teuthology.orchestra.run.smithi044.stdout:crush_version 4 2024-09-13T09:12:11.548 INFO:teuthology.orchestra.run.smithi044.stdout:full_ratio 0.95 2024-09-13T09:12:11.549 INFO:teuthology.orchestra.run.smithi044.stdout:backfillfull_ratio 0.9 2024-09-13T09:12:11.549 INFO:teuthology.orchestra.run.smithi044.stdout:nearfull_ratio 0.85 2024-09-13T09:12:11.549 INFO:teuthology.orchestra.run.smithi044.stdout:require_min_compat_client luminous 2024-09-13T09:12:11.549 INFO:teuthology.orchestra.run.smithi044.stdout:min_compat_client jewel 2024-09-13T09:12:11.549 INFO:teuthology.orchestra.run.smithi044.stdout:require_osd_release reef 2024-09-13T09:12:11.549 INFO:teuthology.orchestra.run.smithi044.stdout:stretch_mode_enabled false 2024-09-13T09:12:11.549 INFO:teuthology.orchestra.run.smithi044.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 15 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 6.06 2024-09-13T09:12:11.549 INFO:teuthology.orchestra.run.smithi044.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 19 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.25 2024-09-13T09:12:11.549 INFO:teuthology.orchestra.run.smithi044.stdout:max_osd 6 2024-09-13T09:12:11.549 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6808/2268459392,v1:172.21.15.44:6810/2268459392] [v2:172.21.15.44:6813/2268459392,v1:172.21.15.44:6815/2268459392] exists,up 156c6730-f804-4806-bc2d-3a7d0856c2b2 2024-09-13T09:12:11.549 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 up in weight 1 up_from 11 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6800/2209612586,v1:172.21.15.44:6801/2209612586] [v2:172.21.15.44:6802/2209612586,v1:172.21.15.44:6803/2209612586] exists,up 00735025-3bf9-4dd0-8b42-f08f1bd1c421 2024-09-13T09:12:11.549 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 up in weight 1 up_from 11 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6809/770926265,v1:172.21.15.44:6811/770926265] [v2:172.21.15.44:6812/770926265,v1:172.21.15.44:6814/770926265] exists,up 395516ac-33b6-4f43-8333-dcb1f4cded32 2024-09-13T09:12:11.549 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 up in weight 1 up_from 11 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6808/2132151186,v1:172.21.15.129:6809/2132151186] [v2:172.21.15.129:6810/2132151186,v1:172.21.15.129:6811/2132151186] exists,up 429e877f-5462-402a-80e5-92a7b17a2eaa 2024-09-13T09:12:11.550 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6800/3393542416,v1:172.21.15.129:6801/3393542416] [v2:172.21.15.129:6802/3393542416,v1:172.21.15.129:6803/3393542416] exists,up 9b3be757-15ee-4157-80a8-0537cd1bcf28 2024-09-13T09:12:11.550 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 up in weight 1 up_from 11 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6816/1803198188,v1:172.21.15.129:6817/1803198188] [v2:172.21.15.129:6818/1803198188,v1:172.21.15.129:6819/1803198188] exists,up c7601d5e-2164-45e1-ab21-368d9e3af27e 2024-09-13T09:12:11.561 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6808/2268459392,v1:172.21.15.44:6810/2268459392] [v2:172.21.15.44:6813/2268459392,v1:172.21.15.44:6815/2268459392] exists,up 156c6730-f804-4806-bc2d-3a7d0856c2b2', 'osd.1 up in weight 1 up_from 11 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6800/2209612586,v1:172.21.15.44:6801/2209612586] [v2:172.21.15.44:6802/2209612586,v1:172.21.15.44:6803/2209612586] exists,up 00735025-3bf9-4dd0-8b42-f08f1bd1c421', 'osd.2 up in weight 1 up_from 11 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6809/770926265,v1:172.21.15.44:6811/770926265] [v2:172.21.15.44:6812/770926265,v1:172.21.15.44:6814/770926265] exists,up 395516ac-33b6-4f43-8333-dcb1f4cded32', 'osd.3 up in weight 1 up_from 11 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6808/2132151186,v1:172.21.15.129:6809/2132151186] [v2:172.21.15.129:6810/2132151186,v1:172.21.15.129:6811/2132151186] exists,up 429e877f-5462-402a-80e5-92a7b17a2eaa', 'osd.4 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6800/3393542416,v1:172.21.15.129:6801/3393542416] [v2:172.21.15.129:6802/3393542416,v1:172.21.15.129:6803/3393542416] exists,up 9b3be757-15ee-4157-80a8-0537cd1bcf28', 'osd.5 up in weight 1 up_from 11 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6816/1803198188,v1:172.21.15.129:6817/1803198188] [v2:172.21.15.129:6818/1803198188,v1:172.21.15.129:6819/1803198188] exists,up c7601d5e-2164-45e1-ab21-368d9e3af27e'] 2024-09-13T09:12:11.561 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-13T09:12:11.662 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:11.662 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mon.a", 2024-09-13T09:12:11.662 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster": "ceph", 2024-09-13T09:12:11.662 INFO:teuthology.orchestra.run.smithi044.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-13T09:12:11.662 INFO:teuthology.orchestra.run.smithi044.stdout: "admin_socket_mode": "", 2024-09-13T09:12:11.662 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-13T09:12:11.663 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_client_required": "cephx, none", 2024-09-13T09:12:11.663 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_cluster_required": "cephx", 2024-09-13T09:12:11.663 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_debug": "true", 2024-09-13T09:12:11.663 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-13T09:12:11.663 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-13T09:12:11.663 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_service_required": "cephx", 2024-09-13T09:12:11.663 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-13T09:12:11.663 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_supported": "cephx", 2024-09-13T09:12:11.663 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_aio": "true", 2024-09-13T09:12:11.663 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-13T09:12:11.663 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_aio_poll_ms": "250", 2024-09-13T09:12:11.663 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_aio_reap_max": "16", 2024-09-13T09:12:11.663 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_async_discard": "false", 2024-09-13T09:12:11.663 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_block_size": "4096", 2024-09-13T09:12:11.664 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_debug_aio": "false", 2024-09-13T09:12:11.664 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-13T09:12:11.664 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-13T09:12:11.664 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_debug_inflight_ios": "false", 2024-09-13T09:12:11.664 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_enable_discard": "false", 2024-09-13T09:12:11.664 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_flock_retry": "3", 2024-09-13T09:12:11.664 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-13T09:12:11.664 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_inject_crash": "0", 2024-09-13T09:12:11.664 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-13T09:12:11.664 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_ioring": "false", 2024-09-13T09:12:11.664 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_ioring_hipri": "false", 2024-09-13T09:12:11.664 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-13T09:12:11.664 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-13T09:12:11.665 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-13T09:12:11.665 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-13T09:12:11.665 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_type": "", 2024-09-13T09:12:11.665 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_alloc_size": "1048576", 2024-09-13T09:12:11.665 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_allocator": "hybrid", 2024-09-13T09:12:11.665 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_buffered_io": "true", 2024-09-13T09:12:11.665 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_check_for_zeros": "false", 2024-09-13T09:12:11.665 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-13T09:12:11.665 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-13T09:12:11.665 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_compact_log_sync": "false", 2024-09-13T09:12:11.665 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-13T09:12:11.665 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-13T09:12:11.665 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-13T09:12:11.666 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-13T09:12:11.666 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_max_log_runway": "4194304", 2024-09-13T09:12:11.666 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_max_prefetch": "1048576", 2024-09-13T09:12:11.666 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_min_flush_size": "524288", 2024-09-13T09:12:11.666 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_min_log_runway": "1048576", 2024-09-13T09:12:11.666 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_replay_recovery": "false", 2024-09-13T09:12:11.666 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-13T09:12:11.666 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-13T09:12:11.666 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_sync_write": "false", 2024-09-13T09:12:11.666 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-13T09:12:11.666 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-13T09:12:11.666 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-13T09:12:11.666 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_allocation_from_file": "true", 2024-09-13T09:12:11.666 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_allocator": "hybrid", 2024-09-13T09:12:11.667 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-13T09:12:11.667 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-13T09:12:11.667 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-13T09:12:11.667 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-13T09:12:11.667 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-13T09:12:11.667 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-13T09:12:11.667 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-13T09:12:11.667 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_create": "true", 2024-09-13T09:12:11.667 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_db_create": "false", 2024-09-13T09:12:11.667 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_db_path": "", 2024-09-13T09:12:11.667 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_db_size": "0", 2024-09-13T09:12:11.667 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_path": "", 2024-09-13T09:12:11.667 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_preallocate_file": "false", 2024-09-13T09:12:11.668 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_size": "107374182400", 2024-09-13T09:12:11.668 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_wal_create": "false", 2024-09-13T09:12:11.668 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_wal_path": "", 2024-09-13T09:12:11.668 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_wal_size": "100663296", 2024-09-13T09:12:11.668 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_bluefs": "true", 2024-09-13T09:12:11.668 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-13T09:12:11.668 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-13T09:12:11.668 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-13T09:12:11.668 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-13T09:12:11.668 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T09:12:11.668 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T09:12:11.668 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-13T09:12:11.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T09:12:11.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_autotune": "true", 2024-09-13T09:12:11.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-13T09:12:11.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-13T09:12:11.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-13T09:12:11.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-13T09:12:11.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_size": "0", 2024-09-13T09:12:11.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-13T09:12:11.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-13T09:12:11.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-13T09:12:11.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-13T09:12:11.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_type": "2q", 2024-09-13T09:12:11.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-13T09:12:11.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_clone_cow": "true", 2024-09-13T09:12:11.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-13T09:12:11.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-13T09:12:11.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-13T09:12:11.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-13T09:12:11.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-13T09:12:11.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-13T09:12:11.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-13T09:12:11.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_mode": "none", 2024-09-13T09:12:11.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-13T09:12:11.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_csum_type": "crc32c", 2024-09-13T09:12:11.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-13T09:12:11.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_freelist": "false", 2024-09-13T09:12:11.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-13T09:12:11.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-13T09:12:11.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-13T09:12:11.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-13T09:12:11.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-13T09:12:11.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-13T09:12:11.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-13T09:12:11.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-13T09:12:11.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-13T09:12:11.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-13T09:12:11.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-13T09:12:11.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-13T09:12:11.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-13T09:12:11.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-13T09:12:11.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_small_allocations": "0", 2024-09-13T09:12:11.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-13T09:12:11.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_default_buffered_read": "true", 2024-09-13T09:12:11.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_default_buffered_write": "false", 2024-09-13T09:12:11.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-13T09:12:11.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-13T09:12:11.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-13T09:12:11.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-13T09:12:11.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-13T09:12:11.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-13T09:12:11.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-13T09:12:11.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-13T09:12:11.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fail_eio": "false", 2024-09-13T09:12:11.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-13T09:12:11.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-13T09:12:11.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-13T09:12:11.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-13T09:12:11.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-13T09:12:11.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-13T09:12:11.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_on_mount": "false", 2024-09-13T09:12:11.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-13T09:12:11.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_on_umount": "false", 2024-09-13T09:12:11.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-13T09:12:11.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-13T09:12:11.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-13T09:12:11.674 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-13T09:12:11.674 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-13T09:12:11.674 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-13T09:12:11.674 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-13T09:12:11.674 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-13T09:12:11.674 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_ignore_data_csum": "false", 2024-09-13T09:12:11.674 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-13T09:12:11.674 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-13T09:12:11.674 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-13T09:12:11.674 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-13T09:12:11.674 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_log_op_age": "5.000000", 2024-09-13T09:12:11.674 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_max_alloc_size": "0", 2024-09-13T09:12:11.674 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_max_blob_size": "0", 2024-09-13T09:12:11.674 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-13T09:12:11.675 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-13T09:12:11.675 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-13T09:12:11.675 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_max_deferred_txc": "32", 2024-09-13T09:12:11.675 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_min_alloc_size": "0", 2024-09-13T09:12:11.675 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-13T09:12:11.675 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-13T09:12:11.675 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_nid_prealloc": "1024", 2024-09-13T09:12:11.675 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-13T09:12:11.675 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-13T09:12:11.675 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-13T09:12:11.675 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-13T09:12:11.675 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_retry_disk_reads": "3", 2024-09-13T09:12:11.675 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_rocksdb_cf": "true", 2024-09-13T09:12:11.675 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-13T09:12:11.676 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-13T09:12:11.676 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-13T09:12:11.676 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-13T09:12:11.676 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-13T09:12:11.676 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-13T09:12:11.676 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_spdk_mem": "512", 2024-09-13T09:12:11.676 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-13T09:12:11.676 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-13T09:12:11.676 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-13T09:12:11.676 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-13T09:12:11.676 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-13T09:12:11.676 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-13T09:12:11.676 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-13T09:12:11.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_tracing": "false", 2024-09-13T09:12:11.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-13T09:12:11.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-13T09:12:11.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-13T09:12:11.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-13T09:12:11.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-13T09:12:11.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-13T09:12:11.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-13T09:12:11.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-13T09:12:11.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-13T09:12:11.677 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_zero_block_detection": "false", 2024-09-13T09:12:11.677 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-13T09:12:11.677 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-13T09:12:11.678 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-13T09:12:11.678 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-13T09:12:11.678 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-13T09:12:11.678 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-13T09:12:11.678 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-13T09:12:11.678 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-13T09:12:11.678 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-13T09:12:11.678 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-13T09:12:11.678 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-13T09:12:11.678 INFO:teuthology.orchestra.run.smithi044.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-13T09:12:11.678 INFO:teuthology.orchestra.run.smithi044.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-13T09:12:11.678 INFO:teuthology.orchestra.run.smithi044.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-13T09:12:11.678 INFO:teuthology.orchestra.run.smithi044.stdout: "cephx_cluster_require_signatures": "false", 2024-09-13T09:12:11.679 INFO:teuthology.orchestra.run.smithi044.stdout: "cephx_cluster_require_version": "2", 2024-09-13T09:12:11.679 INFO:teuthology.orchestra.run.smithi044.stdout: "cephx_require_signatures": "false", 2024-09-13T09:12:11.679 INFO:teuthology.orchestra.run.smithi044.stdout: "cephx_require_version": "2", 2024-09-13T09:12:11.679 INFO:teuthology.orchestra.run.smithi044.stdout: "cephx_service_require_signatures": "false", 2024-09-13T09:12:11.679 INFO:teuthology.orchestra.run.smithi044.stdout: "cephx_service_require_version": "2", 2024-09-13T09:12:11.679 INFO:teuthology.orchestra.run.smithi044.stdout: "cephx_sign_messages": "true", 2024-09-13T09:12:11.679 INFO:teuthology.orchestra.run.smithi044.stdout: "chdir": "", 2024-09-13T09:12:11.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_acl_type": "", 2024-09-13T09:12:11.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_asio_thread_count": "2", 2024-09-13T09:12:11.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_cache_mid": "0.750000", 2024-09-13T09:12:11.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_cache_size": "16384", 2024-09-13T09:12:11.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_caps_release_delay": "5", 2024-09-13T09:12:11.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_check_pool_perm": "true", 2024-09-13T09:12:11.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-13T09:12:11.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_debug_force_sync_read": "false", 2024-09-13T09:12:11.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_debug_getattr_caps": "false", 2024-09-13T09:12:11.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_debug_inject_features": "", 2024-09-13T09:12:11.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_debug_inject_tick_delay": "0", 2024-09-13T09:12:11.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-13T09:12:11.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_die_on_failed_remount": "false", 2024-09-13T09:12:11.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_dirsize_rbytes": "true", 2024-09-13T09:12:11.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_force_lazyio": "false", 2024-09-13T09:12:11.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_fs": "", 2024-09-13T09:12:11.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-13T09:12:11.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_inject_release_failure": "false", 2024-09-13T09:12:11.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_max_inline_size": "4096", 2024-09-13T09:12:11.681 INFO:teuthology.orchestra.run.smithi044.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-13T09:12:11.681 INFO:teuthology.orchestra.run.smithi044.stdout: "client_mds_namespace": "", 2024-09-13T09:12:11.681 INFO:teuthology.orchestra.run.smithi044.stdout: "client_metadata": "", 2024-09-13T09:12:11.681 INFO:teuthology.orchestra.run.smithi044.stdout: "client_mount_gid": "-1", 2024-09-13T09:12:11.681 INFO:teuthology.orchestra.run.smithi044.stdout: "client_mount_timeout": "300", 2024-09-13T09:12:11.681 INFO:teuthology.orchestra.run.smithi044.stdout: "client_mount_uid": "-1", 2024-09-13T09:12:11.681 INFO:teuthology.orchestra.run.smithi044.stdout: "client_mountpoint": "/", 2024-09-13T09:12:11.681 INFO:teuthology.orchestra.run.smithi044.stdout: "client_notify_timeout": "10", 2024-09-13T09:12:11.681 INFO:teuthology.orchestra.run.smithi044.stdout: "client_oc": "true", 2024-09-13T09:12:11.681 INFO:teuthology.orchestra.run.smithi044.stdout: "client_oc_max_dirty": "104857600", 2024-09-13T09:12:11.681 INFO:teuthology.orchestra.run.smithi044.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-13T09:12:11.681 INFO:teuthology.orchestra.run.smithi044.stdout: "client_oc_max_objects": "1000", 2024-09-13T09:12:11.681 INFO:teuthology.orchestra.run.smithi044.stdout: "client_oc_size": "209715200", 2024-09-13T09:12:11.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client_oc_target_dirty": "8388608", 2024-09-13T09:12:11.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client_permissions": "true", 2024-09-13T09:12:11.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client_quota": "true", 2024-09-13T09:12:11.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client_quota_df": "true", 2024-09-13T09:12:11.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client_readahead_max_bytes": "0", 2024-09-13T09:12:11.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client_readahead_max_periods": "4", 2024-09-13T09:12:11.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client_readahead_min": "131072", 2024-09-13T09:12:11.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client_reconnect_stale": "false", 2024-09-13T09:12:11.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client_shutdown_timeout": "30", 2024-09-13T09:12:11.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client_snapdir": ".snap", 2024-09-13T09:12:11.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client_tick_interval": "1", 2024-09-13T09:12:11.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client_trace": "", 2024-09-13T09:12:11.683 INFO:teuthology.orchestra.run.smithi044.stdout: "client_try_dentry_invalidate": "false", 2024-09-13T09:12:11.683 INFO:teuthology.orchestra.run.smithi044.stdout: "client_use_faked_inos": "false", 2024-09-13T09:12:11.683 INFO:teuthology.orchestra.run.smithi044.stdout: "client_use_random_mds": "false", 2024-09-13T09:12:11.683 INFO:teuthology.orchestra.run.smithi044.stdout: "clog_to_graylog": "false", 2024-09-13T09:12:11.683 INFO:teuthology.orchestra.run.smithi044.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-13T09:12:11.683 INFO:teuthology.orchestra.run.smithi044.stdout: "clog_to_graylog_port": "12201", 2024-09-13T09:12:11.683 INFO:teuthology.orchestra.run.smithi044.stdout: "clog_to_monitors": "default=true", 2024-09-13T09:12:11.683 INFO:teuthology.orchestra.run.smithi044.stdout: "clog_to_syslog": "false", 2024-09-13T09:12:11.683 INFO:teuthology.orchestra.run.smithi044.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-13T09:12:11.683 INFO:teuthology.orchestra.run.smithi044.stdout: "clog_to_syslog_level": "info", 2024-09-13T09:12:11.683 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "-", 2024-09-13T09:12:11.683 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_network": "", 2024-09-13T09:12:11.683 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_network_interface": "", 2024-09-13T09:12:11.683 INFO:teuthology.orchestra.run.smithi044.stdout: "compressor_zlib_isal": "false", 2024-09-13T09:12:11.684 INFO:teuthology.orchestra.run.smithi044.stdout: "compressor_zlib_level": "5", 2024-09-13T09:12:11.684 INFO:teuthology.orchestra.run.smithi044.stdout: "compressor_zlib_winsize": "-15", 2024-09-13T09:12:11.684 INFO:teuthology.orchestra.run.smithi044.stdout: "compressor_zstd_level": "1", 2024-09-13T09:12:11.684 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-13T09:12:11.684 INFO:teuthology.orchestra.run.smithi044.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-13T09:12:11.684 INFO:teuthology.orchestra.run.smithi044.stdout: "crimson_alien_op_num_threads": "6", 2024-09-13T09:12:11.684 INFO:teuthology.orchestra.run.smithi044.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-13T09:12:11.684 INFO:teuthology.orchestra.run.smithi044.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-13T09:12:11.684 INFO:teuthology.orchestra.run.smithi044.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-13T09:12:11.684 INFO:teuthology.orchestra.run.smithi044.stdout: "crimson_seastar_smp": "1", 2024-09-13T09:12:11.684 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "", 2024-09-13T09:12:11.684 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location_hook": "", 2024-09-13T09:12:11.684 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location_hook_timeout": "10", 2024-09-13T09:12:11.685 INFO:teuthology.orchestra.run.smithi044.stdout: "daemonize": "false", 2024-09-13T09:12:11.685 INFO:teuthology.orchestra.run.smithi044.stdout: "daos_pool": "tank", 2024-09-13T09:12:11.685 INFO:teuthology.orchestra.run.smithi044.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-13T09:12:11.685 INFO:teuthology.orchestra.run.smithi044.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-13T09:12:11.685 INFO:teuthology.orchestra.run.smithi044.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-13T09:12:11.685 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_alienstore": "0/5", 2024-09-13T09:12:11.685 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_allow_any_pool_priority": "false", 2024-09-13T09:12:11.685 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_asok": "1/5", 2024-09-13T09:12:11.685 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_asok_assert_abort": "false", 2024-09-13T09:12:11.685 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_asserts_on_shutdown": "true", 2024-09-13T09:12:11.685 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_auth": "20/20", 2024-09-13T09:12:11.685 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_bdev": "1/3", 2024-09-13T09:12:11.685 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_bluefs": "1/5", 2024-09-13T09:12:11.685 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_bluestore": "1/5", 2024-09-13T09:12:11.686 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_buffer": "0/1", 2024-09-13T09:12:11.686 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_ceph_exporter": "1/5", 2024-09-13T09:12:11.686 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_cephfs_mirror": "0/5", 2024-09-13T09:12:11.686 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_cephsqlite": "0/5", 2024-09-13T09:12:11.686 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_client": "0/5", 2024-09-13T09:12:11.686 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_compressor": "1/5", 2024-09-13T09:12:11.686 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_context": "0/1", 2024-09-13T09:12:11.686 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_crush": "1/1", 2024-09-13T09:12:11.686 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_crypto": "1/5", 2024-09-13T09:12:11.686 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_cyanstore": "0/5", 2024-09-13T09:12:11.686 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_deliberately_leak_memory": "false", 2024-09-13T09:12:11.686 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_disable_randomized_ping": "false", 2024-09-13T09:12:11.686 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_dpdk": "1/5", 2024-09-13T09:12:11.687 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_eventtrace": "1/5", 2024-09-13T09:12:11.687 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_filer": "0/1", 2024-09-13T09:12:11.687 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_filestore": "1/3", 2024-09-13T09:12:11.687 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_finisher": "1/1", 2024-09-13T09:12:11.687 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_fuse": "1/5", 2024-09-13T09:12:11.687 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_heartbeat_testing_span": "0", 2024-09-13T09:12:11.687 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_heartbeatmap": "1/5", 2024-09-13T09:12:11.687 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-13T09:12:11.687 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_javaclient": "1/5", 2024-09-13T09:12:11.687 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_journal": "1/3", 2024-09-13T09:12:11.687 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_journaler": "0/5", 2024-09-13T09:12:11.687 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_kstore": "1/5", 2024-09-13T09:12:11.687 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_leveldb": "4/5", 2024-09-13T09:12:11.687 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_lockdep": "0/1", 2024-09-13T09:12:11.688 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mclock": "1/5", 2024-09-13T09:12:11.688 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mds": "1/5", 2024-09-13T09:12:11.688 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mds_balancer": "1/5", 2024-09-13T09:12:11.688 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mds_locker": "1/5", 2024-09-13T09:12:11.688 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mds_log": "1/5", 2024-09-13T09:12:11.688 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mds_log_expire": "1/5", 2024-09-13T09:12:11.688 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mds_migrator": "1/5", 2024-09-13T09:12:11.688 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_memstore": "1/5", 2024-09-13T09:12:11.688 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mgr": "2/5", 2024-09-13T09:12:11.688 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mgrc": "1/5", 2024-09-13T09:12:11.688 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mon": "20/20", 2024-09-13T09:12:11.688 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_monc": "0/10", 2024-09-13T09:12:11.688 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_ms": "1/1", 2024-09-13T09:12:11.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_none": "0/5", 2024-09-13T09:12:11.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_objclass": "0/5", 2024-09-13T09:12:11.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_objectcacher": "0/5", 2024-09-13T09:12:11.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_objecter": "0/1", 2024-09-13T09:12:11.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_optracker": "0/5", 2024-09-13T09:12:11.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_osd": "1/5", 2024-09-13T09:12:11.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_paxos": "20/20", 2024-09-13T09:12:11.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_perfcounter": "1/5", 2024-09-13T09:12:11.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_prioritycache": "1/5", 2024-09-13T09:12:11.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rados": "0/5", 2024-09-13T09:12:11.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rbd": "0/5", 2024-09-13T09:12:11.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rbd_mirror": "0/5", 2024-09-13T09:12:11.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rbd_pwl": "0/5", 2024-09-13T09:12:11.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rbd_replay": "0/5", 2024-09-13T09:12:11.690 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_refs": "0/0", 2024-09-13T09:12:11.690 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_reserver": "1/1", 2024-09-13T09:12:11.690 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rgw": "1/5", 2024-09-13T09:12:11.690 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rgw_access": "1/5", 2024-09-13T09:12:11.690 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rgw_datacache": "1/5", 2024-09-13T09:12:11.690 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rgw_dbstore": "1/5", 2024-09-13T09:12:11.690 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rgw_flight": "1/5", 2024-09-13T09:12:11.690 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rgw_sync": "1/5", 2024-09-13T09:12:11.690 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rocksdb": "4/5", 2024-09-13T09:12:11.690 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore": "0/5", 2024-09-13T09:12:11.690 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_backref": "0/5", 2024-09-13T09:12:11.690 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_cache": "0/5", 2024-09-13T09:12:11.690 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_cleaner": "0/5", 2024-09-13T09:12:11.690 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_device": "0/5", 2024-09-13T09:12:11.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_epm": "0/5", 2024-09-13T09:12:11.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-13T09:12:11.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_journal": "0/5", 2024-09-13T09:12:11.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_lba": "0/5", 2024-09-13T09:12:11.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_odata": "0/5", 2024-09-13T09:12:11.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_omap": "0/5", 2024-09-13T09:12:11.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_onode": "0/5", 2024-09-13T09:12:11.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_t": "0/5", 2024-09-13T09:12:11.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_tm": "0/5", 2024-09-13T09:12:11.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_striper": "0/1", 2024-09-13T09:12:11.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_test": "0/5", 2024-09-13T09:12:11.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_throttle": "1/1", 2024-09-13T09:12:11.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_timer": "0/1", 2024-09-13T09:12:11.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_tp": "0/5", 2024-09-13T09:12:11.692 INFO:teuthology.orchestra.run.smithi044.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-13T09:12:11.692 INFO:teuthology.orchestra.run.smithi044.stdout: "device_failure_prediction_mode": "none", 2024-09-13T09:12:11.692 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-13T09:12:11.692 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-13T09:12:11.692 INFO:teuthology.orchestra.run.smithi044.stdout: "err_to_graylog": "false", 2024-09-13T09:12:11.692 INFO:teuthology.orchestra.run.smithi044.stdout: "err_to_journald": "false", 2024-09-13T09:12:11.692 INFO:teuthology.orchestra.run.smithi044.stdout: "err_to_stderr": "true", 2024-09-13T09:12:11.692 INFO:teuthology.orchestra.run.smithi044.stdout: "err_to_syslog": "false", 2024-09-13T09:12:11.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event_tracing": "false", 2024-09-13T09:12:11.692 INFO:teuthology.orchestra.run.smithi044.stdout: "exporter_addr": "0.0.0.0", 2024-09-13T09:12:11.692 INFO:teuthology.orchestra.run.smithi044.stdout: "exporter_http_port": "9926", 2024-09-13T09:12:11.692 INFO:teuthology.orchestra.run.smithi044.stdout: "exporter_prio_limit": "5", 2024-09-13T09:12:11.693 INFO:teuthology.orchestra.run.smithi044.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-13T09:12:11.693 INFO:teuthology.orchestra.run.smithi044.stdout: "exporter_sort_metrics": "true", 2024-09-13T09:12:11.693 INFO:teuthology.orchestra.run.smithi044.stdout: "exporter_stats_period": "5", 2024-09-13T09:12:11.693 INFO:teuthology.orchestra.run.smithi044.stdout: "fake_statfs_for_testing": "0", 2024-09-13T09:12:11.693 INFO:teuthology.orchestra.run.smithi044.stdout: "fatal_signal_handlers": "true", 2024-09-13T09:12:11.693 INFO:teuthology.orchestra.run.smithi044.stdout: "filer_max_purge_ops": "10", 2024-09-13T09:12:11.693 INFO:teuthology.orchestra.run.smithi044.stdout: "filer_max_truncate_ops": "128", 2024-09-13T09:12:11.693 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_apply_finisher_threads": "1", 2024-09-13T09:12:11.693 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_blackhole": "false", 2024-09-13T09:12:11.693 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_btrfs_clone_range": "true", 2024-09-13T09:12:11.693 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_btrfs_snap": "true", 2024-09-13T09:12:11.693 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_caller_concurrency": "10", 2024-09-13T09:12:11.693 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_collect_device_partition_information": "true", 2024-09-13T09:12:11.693 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_commit_timeout": "600.000000", 2024-09-13T09:12:11.694 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_debug_inject_read_err": "false", 2024-09-13T09:12:11.694 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_debug_omap_check": "false", 2024-09-13T09:12:11.694 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-13T09:12:11.694 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_debug_verify_split": "false", 2024-09-13T09:12:11.694 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_dump_file": "", 2024-09-13T09:12:11.694 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-13T09:12:11.694 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-13T09:12:11.694 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_fadvise": "true", 2024-09-13T09:12:11.694 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_fail_eio": "true", 2024-09-13T09:12:11.694 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_fd_cache_shards": "16", 2024-09-13T09:12:11.694 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_fd_cache_size": "128", 2024-09-13T09:12:11.694 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_fiemap": "false", 2024-09-13T09:12:11.695 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_fiemap_threshold": "4096", 2024-09-13T09:12:11.695 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-13T09:12:11.695 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-13T09:12:11.695 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_inject_stall": "0", 2024-09-13T09:12:11.695 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_journal_parallel": "false", 2024-09-13T09:12:11.695 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_journal_trailing": "false", 2024-09-13T09:12:11.695 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_journal_writeahead": "false", 2024-09-13T09:12:11.695 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_kill_at": "0", 2024-09-13T09:12:11.695 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-13T09:12:11.695 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-13T09:12:11.695 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-13T09:12:11.695 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-13T09:12:11.695 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-13T09:12:11.695 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattrs": "0", 2024-09-13T09:12:11.695 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-13T09:12:11.696 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-13T09:12:11.696 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-13T09:12:11.696 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-13T09:12:11.696 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_xattr_value_size": "0", 2024-09-13T09:12:11.696 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-13T09:12:11.696 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-13T09:12:11.696 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-13T09:12:11.696 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_merge_threshold": "-10", 2024-09-13T09:12:11.696 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-13T09:12:11.696 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_odsync_write": "false", 2024-09-13T09:12:11.696 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_omap_backend": "rocksdb", 2024-09-13T09:12:11.696 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_omap_backend_path": "", 2024-09-13T09:12:11.697 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-13T09:12:11.697 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-13T09:12:11.697 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-13T09:12:11.697 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_op_thread_timeout": "60", 2024-09-13T09:12:11.697 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_op_threads": "2", 2024-09-13T09:12:11.697 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_punch_hole": "false", 2024-09-13T09:12:11.697 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-13T09:12:11.697 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-13T09:12:11.697 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-13T09:12:11.697 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-13T09:12:11.697 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-13T09:12:11.697 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-13T09:12:11.697 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-13T09:12:11.697 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-13T09:12:11.698 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-13T09:12:11.698 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_max_ops": "50", 2024-09-13T09:12:11.698 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-13T09:12:11.698 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_seek_data_hole": "false", 2024-09-13T09:12:11.698 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_sloppy_crc": "false", 2024-09-13T09:12:11.698 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-13T09:12:11.698 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_splice": "false", 2024-09-13T09:12:11.698 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_split_multiple": "2", 2024-09-13T09:12:11.698 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_split_rand_factor": "20", 2024-09-13T09:12:11.698 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_update_to": "1000", 2024-09-13T09:12:11.698 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-13T09:12:11.698 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-13T09:12:11.698 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-13T09:12:11.699 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-13T09:12:11.699 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-13T09:12:11.699 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-13T09:12:11.699 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_enable": "true", 2024-09-13T09:12:11.699 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-13T09:12:11.699 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-13T09:12:11.699 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-13T09:12:11.699 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-13T09:12:11.699 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-13T09:12:11.699 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-13T09:12:11.699 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_xfs_extsize": "false", 2024-09-13T09:12:11.699 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_zfs_snap": "false", 2024-09-13T09:12:11.699 INFO:teuthology.orchestra.run.smithi044.stdout: "fio_dir": "/tmp/fio", 2024-09-13T09:12:11.699 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "06cda68b-dd73-4266-b5e6-93ae2b3f7bd5", 2024-09-13T09:12:11.700 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_allow_other": "true", 2024-09-13T09:12:11.700 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_atomic_o_trunc": "true", 2024-09-13T09:12:11.700 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_big_writes": "true", 2024-09-13T09:12:11.700 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_debug": "false", 2024-09-13T09:12:11.700 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_default_permissions": "false", 2024-09-13T09:12:11.700 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_disable_pagecache": "false", 2024-09-13T09:12:11.700 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_max_write": "0", 2024-09-13T09:12:11.700 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_multithreaded": "true", 2024-09-13T09:12:11.700 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_require_active_mds": "true", 2024-09-13T09:12:11.700 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_set_user_groups": "true", 2024-09-13T09:12:11.700 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_splice_move": "true", 2024-09-13T09:12:11.700 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_splice_read": "true", 2024-09-13T09:12:11.700 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_splice_write": "true", 2024-09-13T09:12:11.701 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-13T09:12:11.701 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_use_invalidate_cb": "true", 2024-09-13T09:12:11.701 INFO:teuthology.orchestra.run.smithi044.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-13T09:12:11.701 INFO:teuthology.orchestra.run.smithi044.stdout: "gss_target_name": "ceph", 2024-09-13T09:12:11.701 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_file": "", 2024-09-13T09:12:11.701 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_inject_failure": "0", 2024-09-13T09:12:11.701 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_interval": "5", 2024-09-13T09:12:11.701 INFO:teuthology.orchestra.run.smithi044.stdout: "host": "smithi044", 2024-09-13T09:12:11.701 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-13T09:12:11.701 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-13T09:12:11.701 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-13T09:12:11.701 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-13T09:12:11.701 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-13T09:12:11.702 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-13T09:12:11.702 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-13T09:12:11.702 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-13T09:12:11.702 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-13T09:12:11.702 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-13T09:12:11.702 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-13T09:12:11.702 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-13T09:12:11.702 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-13T09:12:11.702 INFO:teuthology.orchestra.run.smithi044.stdout: "inject_early_sigterm": "false", 2024-09-13T09:12:11.702 INFO:teuthology.orchestra.run.smithi044.stdout: "jaeger_agent_port": "6799", 2024-09-13T09:12:11.702 INFO:teuthology.orchestra.run.smithi044.stdout: "jaeger_tracing_enable": "false", 2024-09-13T09:12:11.702 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_aio": "true", 2024-09-13T09:12:11.702 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_align_min_size": "65536", 2024-09-13T09:12:11.702 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_block_align": "true", 2024-09-13T09:12:11.703 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_block_size": "4096", 2024-09-13T09:12:11.703 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_dio": "true", 2024-09-13T09:12:11.703 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_discard": "false", 2024-09-13T09:12:11.703 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_force_aio": "false", 2024-09-13T09:12:11.703 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_ignore_corruption": "false", 2024-09-13T09:12:11.703 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_max_write_bytes": "10485760", 2024-09-13T09:12:11.703 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_max_write_entries": "100", 2024-09-13T09:12:11.703 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_replay_from": "0", 2024-09-13T09:12:11.703 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-13T09:12:11.703 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-13T09:12:11.703 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-13T09:12:11.703 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-13T09:12:11.703 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_write_header_frequency": "0", 2024-09-13T09:12:11.703 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_zero_on_create": "false", 2024-09-13T09:12:11.704 INFO:teuthology.orchestra.run.smithi044.stdout: "journaler_prefetch_periods": "10", 2024-09-13T09:12:11.704 INFO:teuthology.orchestra.run.smithi044.stdout: "journaler_prezero_periods": "5", 2024-09-13T09:12:11.704 INFO:teuthology.orchestra.run.smithi044.stdout: "journaler_write_head_interval": "15", 2024-09-13T09:12:11.704 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "", 2024-09-13T09:12:11.704 INFO:teuthology.orchestra.run.smithi044.stdout: "keyfile": "", 2024-09-13T09:12:11.704 INFO:teuthology.orchestra.run.smithi044.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-13T09:12:11.704 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_backend": "rocksdb", 2024-09-13T09:12:11.704 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_default_stripe_size": "65536", 2024-09-13T09:12:11.704 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_fsck_on_mount": "false", 2024-09-13T09:12:11.704 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-13T09:12:11.704 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_max_bytes": "67108864", 2024-09-13T09:12:11.704 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_max_ops": "512", 2024-09-13T09:12:11.704 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_nid_prealloc": "1024", 2024-09-13T09:12:11.705 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_onode_map_size": "1024", 2024-09-13T09:12:11.705 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-13T09:12:11.705 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_sync_submit_transaction": "false", 2024-09-13T09:12:11.705 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_sync_transaction": "false", 2024-09-13T09:12:11.705 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_block_size": "65536", 2024-09-13T09:12:11.705 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_bloom_size": "0", 2024-09-13T09:12:11.705 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_cache_size": "536870912", 2024-09-13T09:12:11.705 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_compact_on_mount": "false", 2024-09-13T09:12:11.705 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_compression": "false", 2024-09-13T09:12:11.705 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_log": "", 2024-09-13T09:12:11.705 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-13T09:12:11.705 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_max_open_files": "0", 2024-09-13T09:12:11.705 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_paranoid": "false", 2024-09-13T09:12:11.706 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-13T09:12:11.706 INFO:teuthology.orchestra.run.smithi044.stdout: "librados_thread_count": "2", 2024-09-13T09:12:11.706 INFO:teuthology.orchestra.run.smithi044.stdout: "lockdep": "false", 2024-09-13T09:12:11.706 INFO:teuthology.orchestra.run.smithi044.stdout: "lockdep_force_backtrace": "false", 2024-09-13T09:12:11.706 INFO:teuthology.orchestra.run.smithi044.stdout: "log_coarse_timestamps": "true", 2024-09-13T09:12:11.706 INFO:teuthology.orchestra.run.smithi044.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-13T09:12:11.706 INFO:teuthology.orchestra.run.smithi044.stdout: "log_flush_on_exit": "false", 2024-09-13T09:12:11.706 INFO:teuthology.orchestra.run.smithi044.stdout: "log_graylog_host": "127.0.0.1", 2024-09-13T09:12:11.706 INFO:teuthology.orchestra.run.smithi044.stdout: "log_graylog_port": "12201", 2024-09-13T09:12:11.706 INFO:teuthology.orchestra.run.smithi044.stdout: "log_max_new": "1000", 2024-09-13T09:12:11.706 INFO:teuthology.orchestra.run.smithi044.stdout: "log_max_recent": "10000", 2024-09-13T09:12:11.706 INFO:teuthology.orchestra.run.smithi044.stdout: "log_stderr_prefix": "", 2024-09-13T09:12:11.706 INFO:teuthology.orchestra.run.smithi044.stdout: "log_stop_at_utilization": "0.970000", 2024-09-13T09:12:11.707 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": "true", 2024-09-13T09:12:11.707 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_graylog": "false", 2024-09-13T09:12:11.707 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_journald": "false", 2024-09-13T09:12:11.707 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_stderr": "false", 2024-09-13T09:12:11.707 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_syslog": "false", 2024-09-13T09:12:11.707 INFO:teuthology.orchestra.run.smithi044.stdout: "max_rotating_auth_attempts": "10", 2024-09-13T09:12:11.707 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-13T09:12:11.707 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_action_on_write_error": "1", 2024-09-13T09:12:11.707 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_alternate_name_max": "8192", 2024-09-13T09:12:11.707 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_asio_thread_count": "2", 2024-09-13T09:12:11.707 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_export_pin": "true", 2024-09-13T09:12:11.707 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_fragment_dirs": "true", 2024-09-13T09:12:11.707 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-13T09:12:11.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_fragment_interval": "5", 2024-09-13T09:12:11.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-13T09:12:11.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-13T09:12:11.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_interval": "10", 2024-09-13T09:12:11.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_max": "-1", 2024-09-13T09:12:11.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_max_until": "-1", 2024-09-13T09:12:11.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_merge_size": "50", 2024-09-13T09:12:11.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_midchunk": "0.300000", 2024-09-13T09:12:11.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-13T09:12:11.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_min_start": "0.200000", 2024-09-13T09:12:11.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_minchunk": "0.001000", 2024-09-13T09:12:11.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_mode": "0", 2024-09-13T09:12:11.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_need_max": "1.200000", 2024-09-13T09:12:11.708 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_need_min": "0.800000", 2024-09-13T09:12:11.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-13T09:12:11.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-13T09:12:11.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_split_bits": "3", 2024-09-13T09:12:11.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-13T09:12:11.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_split_size": "10000", 2024-09-13T09:12:11.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-13T09:12:11.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_target_decay": "10.000000", 2024-09-13T09:12:11.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-13T09:12:11.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_beacon_grace": "15.000000", 2024-09-13T09:12:11.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_beacon_interval": "4.000000", 2024-09-13T09:12:11.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-13T09:12:11.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-13T09:12:11.709 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache_mid": "0.700000", 2024-09-13T09:12:11.710 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache_release_free_interval": "10", 2024-09-13T09:12:11.710 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache_reservation": "0.050000", 2024-09-13T09:12:11.710 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-13T09:12:11.710 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache_trim_interval": "1", 2024-09-13T09:12:11.710 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache_trim_threshold": "262144", 2024-09-13T09:12:11.710 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-13T09:12:11.710 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-13T09:12:11.710 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-13T09:12:11.710 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_client_prealloc_inos": "1000", 2024-09-13T09:12:11.710 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-13T09:12:11.710 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_connect_bootstrapping": "false", 2024-09-13T09:12:11.710 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_damage_table_max_entries": "10000", 2024-09-13T09:12:11.710 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-13T09:12:11.711 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_debug_auth_pins": "false", 2024-09-13T09:12:11.711 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_debug_frag": "false", 2024-09-13T09:12:11.711 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_debug_scatterstat": "false", 2024-09-13T09:12:11.711 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_debug_subtrees": "false", 2024-09-13T09:12:11.711 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_decay_halflife": "5.000000", 2024-09-13T09:12:11.711 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_default_dir_hash": "2", 2024-09-13T09:12:11.711 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_defer_session_stale": "true", 2024-09-13T09:12:11.711 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_deny_all_reconnect": "false", 2024-09-13T09:12:11.711 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dir_keys_per_op": "16384", 2024-09-13T09:12:11.711 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dir_max_commit_size": "10", 2024-09-13T09:12:11.711 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dir_max_entries": "0", 2024-09-13T09:12:11.711 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dir_prefetch": "true", 2024-09-13T09:12:11.711 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-13T09:12:11.711 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-13T09:12:11.712 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dump_cache_on_map": "false", 2024-09-13T09:12:11.712 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-13T09:12:11.712 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-13T09:12:11.712 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_early_reply": "true", 2024-09-13T09:12:11.712 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_enable_op_tracker": "true", 2024-09-13T09:12:11.712 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_enforce_unique_name": "true", 2024-09-13T09:12:11.712 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-13T09:12:11.712 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-13T09:12:11.712 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_export_ephemeral_random": "true", 2024-09-13T09:12:11.712 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-13T09:12:11.712 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-13T09:12:11.712 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-13T09:12:11.712 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-13T09:12:11.713 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-13T09:12:11.713 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-13T09:12:11.713 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-13T09:12:11.713 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-13T09:12:11.713 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_health_summarize_threshold": "10", 2024-09-13T09:12:11.713 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-13T09:12:11.713 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-13T09:12:11.713 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_inject_health_dummy": "false", 2024-09-13T09:12:11.713 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-13T09:12:11.713 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_inject_migrator_session_race": "false", 2024-09-13T09:12:11.713 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-13T09:12:11.713 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-13T09:12:11.713 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-13T09:12:11.714 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_join_fs": "", 2024-09-13T09:12:11.714 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_journal_format": "1", 2024-09-13T09:12:11.714 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-13T09:12:11.714 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_create_at": "0", 2024-09-13T09:12:11.714 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_export_at": "0", 2024-09-13T09:12:11.714 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_import_at": "0", 2024-09-13T09:12:11.714 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_journal_at": "0", 2024-09-13T09:12:11.714 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_journal_expire_at": "0", 2024-09-13T09:12:11.714 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_journal_replay_at": "0", 2024-09-13T09:12:11.714 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_link_at": "0", 2024-09-13T09:12:11.714 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_mdstable_at": "0", 2024-09-13T09:12:11.714 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_openc_at": "0", 2024-09-13T09:12:11.714 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_rename_at": "0", 2024-09-13T09:12:11.714 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log_events_per_segment": "1024", 2024-09-13T09:12:11.715 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log_max_events": "-1", 2024-09-13T09:12:11.715 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log_max_segments": "128", 2024-09-13T09:12:11.715 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log_pause": "false", 2024-09-13T09:12:11.715 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log_segment_size": "0", 2024-09-13T09:12:11.715 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-13T09:12:11.715 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log_warn_factor": "2.000000", 2024-09-13T09:12:11.715 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_caps_per_client": "1048576", 2024-09-13T09:12:11.715 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_completed_flushes": "100000", 2024-09-13T09:12:11.715 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_completed_requests": "100000", 2024-09-13T09:12:11.715 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_export_size": "20971520", 2024-09-13T09:12:11.715 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_file_recover": "32", 2024-09-13T09:12:11.715 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_purge_files": "64", 2024-09-13T09:12:11.715 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_purge_ops": "8192", 2024-09-13T09:12:11.716 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-13T09:12:11.716 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-13T09:12:11.716 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_snaps_per_dir": "100", 2024-09-13T09:12:11.716 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_metrics_update_interval": "2", 2024-09-13T09:12:11.716 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_min_caps_per_client": "100", 2024-09-13T09:12:11.716 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_min_caps_working_set": "10000", 2024-09-13T09:12:11.716 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-13T09:12:11.716 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_numa_node": "-1", 2024-09-13T09:12:11.716 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-13T09:12:11.716 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_op_complaint_time": "30.000000", 2024-09-13T09:12:11.716 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_op_history_duration": "600", 2024-09-13T09:12:11.716 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_op_history_size": "20", 2024-09-13T09:12:11.716 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_op_history_slow_op_size": "20", 2024-09-13T09:12:11.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-13T09:12:11.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_op_log_threshold": "5", 2024-09-13T09:12:11.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_ping_grace": "15", 2024-09-13T09:12:11.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_ping_interval": "5", 2024-09-13T09:12:11.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-13T09:12:11.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-13T09:12:11.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_recall_max_caps": "30000", 2024-09-13T09:12:11.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-13T09:12:11.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-13T09:12:11.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-13T09:12:11.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_recall_warning_threshold": "262144", 2024-09-13T09:12:11.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-13T09:12:11.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_replay_interval": "1.000000", 2024-09-13T09:12:11.717 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-13T09:12:11.718 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-13T09:12:11.718 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_root_ino_gid": "0", 2024-09-13T09:12:11.718 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_root_ino_uid": "0", 2024-09-13T09:12:11.718 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-13T09:12:11.718 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-13T09:12:11.718 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-13T09:12:11.718 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-13T09:12:11.718 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-13T09:12:11.718 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-13T09:12:11.718 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-13T09:12:11.718 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-13T09:12:11.718 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-13T09:12:11.718 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-13T09:12:11.719 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_shutdown_check": "0", 2024-09-13T09:12:11.719 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_skip_ino": "0", 2024-09-13T09:12:11.719 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-13T09:12:11.719 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_snap_max_uid": "4294967294", 2024-09-13T09:12:11.719 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_snap_min_uid": "0", 2024-09-13T09:12:11.719 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_snap_rstat": "false", 2024-09-13T09:12:11.719 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_standby_replay_damaged": "false", 2024-09-13T09:12:11.719 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_symlink_recovery": "true", 2024-09-13T09:12:11.719 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-13T09:12:11.719 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_thrash_exports": "0", 2024-09-13T09:12:11.720 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_thrash_fragments": "0", 2024-09-13T09:12:11.720 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_tick_interval": "5.000000", 2024-09-13T09:12:11.720 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_valgrind_exit": "false", 2024-09-13T09:12:11.720 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_verify_backtrace": "1", 2024-09-13T09:12:11.720 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_verify_scatter": "false", 2024-09-13T09:12:11.720 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-13T09:12:11.720 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_wipe_sessions": "false", 2024-09-13T09:12:11.720 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool_debug": "false", 2024-09-13T09:12:11.720 INFO:teuthology.orchestra.run.smithi044.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-13T09:12:11.720 INFO:teuthology.orchestra.run.smithi044.stdout: "memstore_device_bytes": "1073741824", 2024-09-13T09:12:11.720 INFO:teuthology.orchestra.run.smithi044.stdout: "memstore_page_set": "false", 2024-09-13T09:12:11.721 INFO:teuthology.orchestra.run.smithi044.stdout: "memstore_page_size": "65536", 2024-09-13T09:12:11.721 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_client_bytes": "134217728", 2024-09-13T09:12:11.721 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_client_messages": "512", 2024-09-13T09:12:11.721 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-13T09:12:11.721 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-13T09:12:11.721 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-13T09:12:11.721 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-13T09:12:11.721 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_disabled_modules": "", 2024-09-13T09:12:11.721 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-13T09:12:11.721 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_max_pg_num_change": "128", 2024-09-13T09:12:11.721 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_mds_bytes": "134217728", 2024-09-13T09:12:11.721 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_mds_messages": "128", 2024-09-13T09:12:11.721 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-13T09:12:11.722 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_mon_bytes": "134217728", 2024-09-13T09:12:11.722 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_mon_messages": "128", 2024-09-13T09:12:11.722 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_osd_bytes": "536870912", 2024-09-13T09:12:11.722 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_osd_messages": "8192", 2024-09-13T09:12:11.722 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_pool": "true", 2024-09-13T09:12:11.722 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-13T09:12:11.722 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_standby_modules": "true", 2024-09-13T09:12:11.722 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_stats_period": "5", 2024-09-13T09:12:11.722 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_stats_threshold": "5", 2024-09-13T09:12:11.722 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_tick_period": "2", 2024-09-13T09:12:11.722 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-13T09:12:11.722 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-13T09:12:11.722 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_allow_pool_delete": "true", 2024-09-13T09:12:11.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_allow_pool_size_one": "true", 2024-09-13T09:12:11.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_auth_validate_all_caps": "true", 2024-09-13T09:12:11.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-13T09:12:11.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-13T09:12:11.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_bytes": "104857600", 2024-09-13T09:12:11.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_directed_command_retry": "5", 2024-09-13T09:12:11.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-13T09:12:11.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-13T09:12:11.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-13T09:12:11.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-13T09:12:11.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_hunt_parallel": "3", 2024-09-13T09:12:11.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_log_interval": "1.000000", 2024-09-13T09:12:11.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-13T09:12:11.723 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_ping_interval": "10.000000", 2024-09-13T09:12:11.724 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-13T09:12:11.724 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-13T09:12:11.724 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-13T09:12:11.724 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-13T09:12:11.724 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_file_level": "debug", 2024-09-13T09:12:11.724 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_file": "true", 2024-09-13T09:12:11.724 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-13T09:12:11.724 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-13T09:12:11.724 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-13T09:12:11.724 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_journald": "false", 2024-09-13T09:12:11.724 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-13T09:12:11.724 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-13T09:12:11.724 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-13T09:12:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-13T09:12:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_compact_on_bootstrap": "false", 2024-09-13T09:12:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_compact_on_start": "false", 2024-09-13T09:12:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_compact_on_trim": "true", 2024-09-13T09:12:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-13T09:12:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-13T09:12:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-13T09:12:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cpu_threads": "4", 2024-09-13T09:12:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_crush_min_required_version": "hammer", 2024-09-13T09:12:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_daemon_bytes": "419430400", 2024-09-13T09:12:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-13T09:12:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_data_avail_crit": "5", 2024-09-13T09:12:11.725 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_data_avail_warn": "5", 2024-09-13T09:12:11.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_data_size_warn": "16106127360", 2024-09-13T09:12:11.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-13T09:12:11.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-13T09:12:11.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_dump_json": "false", 2024-09-13T09:12:11.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-13T09:12:11.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_dump_transactions": "false", 2024-09-13T09:12:11.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_extra_checks": "false", 2024-09-13T09:12:11.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-13T09:12:11.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-13T09:12:11.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_no_require_quincy": "false", 2024-09-13T09:12:11.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_no_require_reef": "false", 2024-09-13T09:12:11.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-13T09:12:11.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-13T09:12:11.726 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-13T09:12:11.727 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_down_mkfs_grace": "120", 2024-09-13T09:12:11.727 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_election_default_strategy": "1", 2024-09-13T09:12:11.727 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_election_timeout": "5.000000", 2024-09-13T09:12:11.727 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-13T09:12:11.727 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_elector_ping_divisor": "2", 2024-09-13T09:12:11.727 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-13T09:12:11.727 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_enable_op_tracker": "true", 2024-09-13T09:12:11.727 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_fake_pool_delete": "false", 2024-09-13T09:12:11.727 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_force_quorum_join": "false", 2024-09-13T09:12:11.727 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_globalid_prealloc": "10000", 2024-09-13T09:12:11.727 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_health_detail_to_clog": "false", 2024-09-13T09:12:11.727 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_health_log_update_period": "5", 2024-09-13T09:12:11.727 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_health_max_detail": "50", 2024-09-13T09:12:11.728 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_health_to_clog": "true", 2024-09-13T09:12:11.728 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_health_to_clog_interval": "600", 2024-09-13T09:12:11.728 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-13T09:12:11.728 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_host": "172.21.15.44,[v2:172.21.15.44:3301,v1:172.21.15.44:6790],172.21.15.129", 2024-09-13T09:12:11.728 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_host_override": "", 2024-09-13T09:12:11.728 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_initial_members": "", 2024-09-13T09:12:11.728 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-13T09:12:11.728 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-13T09:12:11.728 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-13T09:12:11.728 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-13T09:12:11.728 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-13T09:12:11.728 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_lease": "5.000000", 2024-09-13T09:12:11.728 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-13T09:12:11.728 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-13T09:12:11.729 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_log_full_interval": "50", 2024-09-13T09:12:11.729 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_log_max": "10000", 2024-09-13T09:12:11.729 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_log_max_summary": "50", 2024-09-13T09:12:11.729 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-13T09:12:11.729 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_log_epochs": "500", 2024-09-13T09:12:11.729 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-13T09:12:11.729 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-13T09:12:11.729 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_osd": "10000", 2024-09-13T09:12:11.729 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_pg_per_osd": "10000", 2024-09-13T09:12:11.729 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_pool_pg_num": "65536", 2024-09-13T09:12:11.729 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-13T09:12:11.729 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-13T09:12:11.729 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mds_force_trim_to": "0", 2024-09-13T09:12:11.730 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mds_skip_sanity": "false", 2024-09-13T09:12:11.730 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_memory_autotune": "true", 2024-09-13T09:12:11.730 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_memory_target": "2147483648", 2024-09-13T09:12:11.730 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mgr_beacon_grace": "30", 2024-09-13T09:12:11.730 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-13T09:12:11.730 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mgr_digest_period": "5", 2024-09-13T09:12:11.730 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mgr_inactive_grace": "60", 2024-09-13T09:12:11.730 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-13T09:12:11.730 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-13T09:12:11.730 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_min_osdmap_epochs": "500", 2024-09-13T09:12:11.730 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_op_complaint_time": "30", 2024-09-13T09:12:11.730 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_op_history_duration": "600", 2024-09-13T09:12:11.730 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_op_history_size": "20", 2024-09-13T09:12:11.730 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_op_history_slow_op_size": "20", 2024-09-13T09:12:11.731 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-13T09:12:11.731 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_op_log_threshold": "5", 2024-09-13T09:12:11.731 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-13T09:12:11.731 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-13T09:12:11.731 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-13T09:12:11.731 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_auto_mark_in": "false", 2024-09-13T09:12:11.731 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-13T09:12:11.731 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-13T09:12:11.731 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-13T09:12:11.731 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_cache_size": "500", 2024-09-13T09:12:11.731 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-13T09:12:11.731 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-13T09:12:11.731 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-13T09:12:11.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_down_out_interval": "600", 2024-09-13T09:12:11.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-13T09:12:11.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-13T09:12:11.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_force_trim_to": "0", 2024-09-13T09:12:11.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-13T09:12:11.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-13T09:12:11.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-13T09:12:11.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-13T09:12:11.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-13T09:12:11.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-13T09:12:11.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-13T09:12:11.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-13T09:12:11.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_min_down_reporters": "2", 2024-09-13T09:12:11.732 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-13T09:12:11.733 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-13T09:12:11.733 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-13T09:12:11.733 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-13T09:12:11.733 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-13T09:12:11.733 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-13T09:12:11.733 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_report_timeout": "900", 2024-09-13T09:12:11.733 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-13T09:12:11.733 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-13T09:12:11.733 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-13T09:12:11.733 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-13T09:12:11.733 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-13T09:12:11.733 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-13T09:12:11.733 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-13T09:12:11.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-13T09:12:11.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-13T09:12:11.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pg_stuck_threshold": "60", 2024-09-13T09:12:11.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-13T09:12:11.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-13T09:12:11.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-13T09:12:11.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-13T09:12:11.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-13T09:12:11.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-13T09:12:11.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_probe_timeout": "2.000000", 2024-09-13T09:12:11.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_reweight_max_change": "0.050000", 2024-09-13T09:12:11.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_reweight_max_osds": "4", 2024-09-13T09:12:11.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-13T09:12:11.734 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-13T09:12:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-13T09:12:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-13T09:12:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-13T09:12:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_scrub_interval": "86400", 2024-09-13T09:12:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_scrub_max_keys": "100", 2024-09-13T09:12:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_scrub_timeout": "300", 2024-09-13T09:12:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_session_timeout": "300", 2024-09-13T09:12:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_smart_report_timeout": "5", 2024-09-13T09:12:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_stat_smooth_intervals": "6", 2024-09-13T09:12:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-13T09:12:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_stretch_pool_min_size": "2", 2024-09-13T09:12:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_stretch_pool_size": "4", 2024-09-13T09:12:11.735 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-13T09:12:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-13T09:12:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_sync_debug": "false", 2024-09-13T09:12:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-13T09:12:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-13T09:12:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_sync_provider_kill_at": "0", 2024-09-13T09:12:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_sync_requester_kill_at": "0", 2024-09-13T09:12:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_sync_timeout": "60.000000", 2024-09-13T09:12:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_target_pg_per_osd": "100", 2024-09-13T09:12:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_tick_interval": "5", 2024-09-13T09:12:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_timecheck_interval": "300.000000", 2024-09-13T09:12:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-13T09:12:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_use_min_delay_socket": "false", 2024-09-13T09:12:11.736 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_older_version_delay": "604800", 2024-09-13T09:12:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-13T09:12:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-13T09:12:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-13T09:12:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-13T09:12:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-13T09:12:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-13T09:12:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-13T09:12:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_misplaced": "false", 2024-09-13T09:12:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-13T09:12:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_older_version": "true", 2024-09-13T09:12:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-13T09:12:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-13T09:12:11.737 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-13T09:12:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-13T09:12:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-13T09:12:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-13T09:12:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-13T09:12:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-13T09:12:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-13T09:12:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": "", 2024-09-13T09:12:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T09:12:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-13T09:12:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-13T09:12:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T09:12:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-13T09:12:11.738 INFO:teuthology.orchestra.run.smithi044.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-13T09:12:11.739 INFO:teuthology.orchestra.run.smithi044.stdout: "motr_tracing_enabled": "false", 2024-09-13T09:12:11.739 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_op_threads": "3", 2024-09-13T09:12:11.739 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-13T09:12:11.739 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_cm": "false", 2024-09-13T09:12:11.739 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_device_name": "", 2024-09-13T09:12:11.739 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_dscp": "96", 2024-09-13T09:12:11.739 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-13T09:12:11.739 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-13T09:12:11.739 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_local_gid": "", 2024-09-13T09:12:11.739 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-13T09:12:11.739 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_port_num": "1", 2024-09-13T09:12:11.739 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-13T09:12:11.739 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-13T09:12:11.739 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-13T09:12:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-13T09:12:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_sl": "3", 2024-09-13T09:12:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_support_srq": "true", 2024-09-13T09:12:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_type": "ib", 2024-09-13T09:12:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_reap_threshold": "5", 2024-09-13T09:12:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_before_connect": "false", 2024-09-13T09:12:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_ipv4": "true", 2024-09-13T09:12:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_ipv6": "false", 2024-09-13T09:12:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_msgr1": "true", 2024-09-13T09:12:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_msgr2": "true", 2024-09-13T09:12:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_port_max": "7568", 2024-09-13T09:12:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_port_min": "6800", 2024-09-13T09:12:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-13T09:12:11.740 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_retry_count": "3", 2024-09-13T09:12:11.741 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_retry_delay": "5", 2024-09-13T09:12:11.741 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_blackhole_client": "false", 2024-09-13T09:12:11.741 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_blackhole_mds": "false", 2024-09-13T09:12:11.741 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_blackhole_mgr": "false", 2024-09-13T09:12:11.741 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_blackhole_mon": "false", 2024-09-13T09:12:11.741 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_blackhole_osd": "false", 2024-09-13T09:12:11.741 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_client_mode": "crc secure", 2024-09-13T09:12:11.741 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_cluster_mode": "crc secure", 2024-09-13T09:12:11.741 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_cluster_type": "", 2024-09-13T09:12:11.741 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_compress_secure": "false", 2024-09-13T09:12:11.741 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_connection_idle_timeout": "900", 2024-09-13T09:12:11.741 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_connection_ready_timeout": "10", 2024-09-13T09:12:11.741 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_crc_data": "true", 2024-09-13T09:12:11.742 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_crc_header": "true", 2024-09-13T09:12:11.742 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_die_on_bad_msg": "false", 2024-09-13T09:12:11.742 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_die_on_bug": "true", 2024-09-13T09:12:11.742 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_die_on_old_message": "true", 2024-09-13T09:12:11.742 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_die_on_skipped_message": "false", 2024-09-13T09:12:11.742 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-13T09:12:11.742 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-13T09:12:11.742 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_coremask": "0xF", 2024-09-13T09:12:11.742 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-13T09:12:11.742 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-13T09:12:11.742 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_enable_tso": "true", 2024-09-13T09:12:11.742 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-13T09:12:11.742 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-13T09:12:11.743 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_hugepages": "", 2024-09-13T09:12:11.743 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-13T09:12:11.743 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-13T09:12:11.743 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_lro": "true", 2024-09-13T09:12:11.743 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_memory_channel": "4", 2024-09-13T09:12:11.743 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-13T09:12:11.743 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_pmd": "", 2024-09-13T09:12:11.743 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_port_id": "0", 2024-09-13T09:12:11.743 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-13T09:12:11.743 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-13T09:12:11.743 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dump_on_send": "false", 2024-09-13T09:12:11.743 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_initial_backoff": "0.200000", 2024-09-13T09:12:11.743 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_inject_delay_max": "1.000000", 2024-09-13T09:12:11.744 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-13T09:12:11.744 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_inject_delay_type": "", 2024-09-13T09:12:11.744 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-13T09:12:11.744 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_inject_network_congestion": "0", 2024-09-13T09:12:11.744 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_inject_socket_failures": "5000", 2024-09-13T09:12:11.744 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_learn_addr_from_peer": "true", 2024-09-13T09:12:11.744 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_max_accept_failures": "4", 2024-09-13T09:12:11.744 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_max_backoff": "15.000000", 2024-09-13T09:12:11.744 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_mon_client_mode": "secure crc", 2024-09-13T09:12:11.744 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-13T09:12:11.744 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_mon_service_mode": "secure crc", 2024-09-13T09:12:11.744 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_osd_compress_min_size": "1024", 2024-09-13T09:12:11.744 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_osd_compress_mode": "none", 2024-09-13T09:12:11.744 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-13T09:12:11.745 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-13T09:12:11.745 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_pq_min_cost": "65536", 2024-09-13T09:12:11.745 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_public_type": "", 2024-09-13T09:12:11.745 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_service_mode": "crc secure", 2024-09-13T09:12:11.745 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_tcp_listen_backlog": "512", 2024-09-13T09:12:11.745 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_tcp_nodelay": "true", 2024-09-13T09:12:11.745 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-13T09:12:11.745 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_tcp_rcvbuf": "0", 2024-09-13T09:12:11.745 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_type": "async+posix", 2024-09-13T09:12:11.745 INFO:teuthology.orchestra.run.smithi044.stdout: "no_config_file": "false", 2024-09-13T09:12:11.745 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_completion_locks_per_session": "32", 2024-09-13T09:12:11.745 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-13T09:12:11.746 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-13T09:12:11.746 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_inflight_ops": "1024", 2024-09-13T09:12:11.746 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-13T09:12:11.746 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-13T09:12:11.746 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_tick_interval": "5.000000", 2024-09-13T09:12:11.746 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_timeout": "10.000000", 2024-09-13T09:12:11.746 INFO:teuthology.orchestra.run.smithi044.stdout: "objectstore_blackhole": "false", 2024-09-13T09:12:11.746 INFO:teuthology.orchestra.run.smithi044.stdout: "openssl_engine_opts": "", 2024-09-13T09:12:11.746 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_agent_delay_time": "5.000000", 2024-09-13T09:12:11.746 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_agent_hist_halflife": "1000", 2024-09-13T09:12:11.746 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_agent_max_low_ops": "2", 2024-09-13T09:12:11.746 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_agent_max_ops": "4", 2024-09-13T09:12:11.746 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-13T09:12:11.746 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-13T09:12:11.747 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_agent_slop": "0.020000", 2024-09-13T09:12:11.747 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-13T09:12:11.747 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-13T09:12:11.747 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_asio_thread_count": "2", 2024-09-13T09:12:11.747 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_async_recovery_min_cost": "100", 2024-09-13T09:12:11.747 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-13T09:12:11.747 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-13T09:12:11.747 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_backfill_scan_max": "512", 2024-09-13T09:12:11.747 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_backfill_scan_min": "64", 2024-09-13T09:12:11.747 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_backoff_on_degraded": "false", 2024-09-13T09:12:11.747 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_backoff_on_peering": "false", 2024-09-13T09:12:11.747 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_backoff_on_unfound": "true", 2024-09-13T09:12:11.748 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_beacon_report_interval": "300", 2024-09-13T09:12:11.748 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bench_duration": "30", 2024-09-13T09:12:11.748 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-13T09:12:11.748 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bench_max_block_size": "67108864", 2024-09-13T09:12:11.748 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-13T09:12:11.748 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_blkin_trace_all": "false", 2024-09-13T09:12:11.748 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-13T09:12:11.748 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-13T09:12:11.748 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-13T09:12:11.748 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-13T09:12:11.749 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_check_for_log_corruption": "false", 2024-09-13T09:12:11.749 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-13T09:12:11.749 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T09:12:11.749 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-13T09:12:11.749 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T09:12:11.749 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_class_update_on_start": "true", 2024-09-13T09:12:11.749 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_client_message_cap": "256", 2024-09-13T09:12:11.749 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_client_message_size_cap": "524288000", 2024-09-13T09:12:11.749 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_client_op_priority": "63", 2024-09-13T09:12:11.749 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_client_watch_timeout": "30", 2024-09-13T09:12:11.749 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_command_max_records": "256", 2024-09-13T09:12:11.749 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_compact_on_start": "false", 2024-09-13T09:12:11.749 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-13T09:12:11.750 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-13T09:12:11.750 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-13T09:12:11.750 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_crush_update_on_start": "true", 2024-09-13T09:12:11.750 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_crush_update_weight_set": "true", 2024-09-13T09:12:11.750 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-13T09:12:11.750 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-13T09:12:11.750 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-13T09:12:11.750 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-13T09:12:11.750 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-13T09:12:11.750 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_feed_pullee": "-1", 2024-09-13T09:12:11.750 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-13T09:12:11.750 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-13T09:12:11.750 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-13T09:12:11.750 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_misdirected_ops": "false", 2024-09-13T09:12:11.751 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_no_acting_change": "false", 2024-09-13T09:12:11.751 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_no_purge_strays": "false", 2024-09-13T09:12:11.751 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_op_order": "false", 2024-09-13T09:12:11.751 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-13T09:12:11.751 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-13T09:12:11.751 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-13T09:12:11.751 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-13T09:12:11.751 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_shutdown": "false", 2024-09-13T09:12:11.751 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-13T09:12:11.751 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-13T09:12:11.751 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-13T09:12:11.751 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-13T09:12:11.751 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_verify_snaps": "false", 2024-09-13T09:12:11.752 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-13T09:12:11.752 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-13T09:12:11.752 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_deep_scrub_keys": "1024", 2024-09-13T09:12:11.752 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-13T09:12:11.752 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-13T09:12:11.752 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-13T09:12:11.752 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_deep_scrub_stride": "524288", 2024-09-13T09:12:11.752 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-13T09:12:11.752 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-13T09:12:11.752 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_default_notify_timeout": "30", 2024-09-13T09:12:11.752 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_delete_sleep": "0.000000", 2024-09-13T09:12:11.752 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-13T09:12:11.752 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-13T09:12:11.753 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-13T09:12:11.753 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_discard_disconnected_ops": "true", 2024-09-13T09:12:11.753 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_enable_op_tracker": "true", 2024-09-13T09:12:11.753 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-13T09:12:11.753 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-13T09:12:11.753 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-13T09:12:11.753 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-13T09:12:11.753 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_fast_info": "true", 2024-09-13T09:12:11.753 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_fast_shutdown": "true", 2024-09-13T09:12:11.753 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-13T09:12:11.753 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-13T09:12:11.753 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-13T09:12:11.753 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-13T09:12:11.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-13T09:12:11.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_function_tracing": "false", 2024-09-13T09:12:11.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_heartbeat_grace": "20", 2024-09-13T09:12:11.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_heartbeat_interval": "6", 2024-09-13T09:12:11.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-13T09:12:11.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_heartbeat_min_peers": "10", 2024-09-13T09:12:11.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_heartbeat_min_size": "2000", 2024-09-13T09:12:11.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_heartbeat_stale": "600", 2024-09-13T09:12:11.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-13T09:12:11.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_hit_set_max_size": "100000", 2024-09-13T09:12:11.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_hit_set_min_size": "1000", 2024-09-13T09:12:11.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-13T09:12:11.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-13T09:12:11.754 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-13T09:12:11.755 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-13T09:12:11.755 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-13T09:12:11.755 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-13T09:12:11.755 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_journal_size": "5120", 2024-09-13T09:12:11.755 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_kill_backfill_at": "0", 2024-09-13T09:12:11.755 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-13T09:12:11.755 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_map_cache_size": "50", 2024-09-13T09:12:11.755 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_map_dedup": "true", 2024-09-13T09:12:11.755 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_map_message_max": "40", 2024-09-13T09:12:11.755 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-13T09:12:11.755 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_map_share_max_epochs": "40", 2024-09-13T09:12:11.755 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_attr_name_len": "100", 2024-09-13T09:12:11.755 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_attr_size": "0", 2024-09-13T09:12:11.756 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_backfills": "1", 2024-09-13T09:12:11.756 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_markdown_count": "5", 2024-09-13T09:12:11.756 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_markdown_period": "600", 2024-09-13T09:12:11.756 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_object_name_len": "2048", 2024-09-13T09:12:11.756 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_object_namespace_len": "256", 2024-09-13T09:12:11.756 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_object_size": "134217728", 2024-09-13T09:12:11.756 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-13T09:12:11.756 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-13T09:12:11.756 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_pg_blocked_by": "16", 2024-09-13T09:12:11.756 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_pg_log_entries": "10000", 2024-09-13T09:12:11.756 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-13T09:12:11.756 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_pgls": "1024", 2024-09-13T09:12:11.756 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_push_cost": "8388608", 2024-09-13T09:12:11.757 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_push_objects": "10", 2024-09-13T09:12:11.757 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_scrubs": "3", 2024-09-13T09:12:11.757 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-13T09:12:11.757 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_trimming_pgs": "2", 2024-09-13T09:12:11.757 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_write_op_reply_len": "64", 2024-09-13T09:12:11.757 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_write_size": "90", 2024-09-13T09:12:11.757 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-13T09:12:11.757 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-13T09:12:11.757 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-13T09:12:11.757 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-13T09:12:11.757 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-13T09:12:11.757 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-13T09:12:11.757 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-13T09:12:11.757 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-13T09:12:11.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_profile": "balanced", 2024-09-13T09:12:11.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-13T09:12:11.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-13T09:12:11.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-13T09:12:11.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-13T09:12:11.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-13T09:12:11.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-13T09:12:11.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-13T09:12:11.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-13T09:12:11.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-13T09:12:11.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-13T09:12:11.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-13T09:12:11.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_memory_base": "805306368", 2024-09-13T09:12:11.758 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_memory_cache_min": "134217728", 2024-09-13T09:12:11.759 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-13T09:12:11.759 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-13T09:12:11.759 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_memory_target": "4294967296", 2024-09-13T09:12:11.759 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_memory_target_autotune": "false", 2024-09-13T09:12:11.759 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-13T09:12:11.759 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_min_pg_log_entries": "250", 2024-09-13T09:12:11.759 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_min_recovery_priority": "0", 2024-09-13T09:12:11.759 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-13T09:12:11.759 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-13T09:12:11.759 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mon_report_interval": "5", 2024-09-13T09:12:11.759 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-13T09:12:11.759 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-13T09:12:11.759 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_num_cache_shards": "32", 2024-09-13T09:12:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_num_op_tracker_shard": "32", 2024-09-13T09:12:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_numa_auto_affinity": "true", 2024-09-13T09:12:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_numa_node": "-1", 2024-09-13T09:12:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_numa_prefer_iface": "true", 2024-09-13T09:12:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-13T09:12:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_objecter_finishers": "1", 2024-09-13T09:12:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_objectstore": "bluestore", 2024-09-13T09:12:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_objectstore_fuse": "false", 2024-09-13T09:12:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_objectstore_tracing": "false", 2024-09-13T09:12:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_complaint_time": "30.000000", 2024-09-13T09:12:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_history_duration": "600", 2024-09-13T09:12:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_history_size": "20", 2024-09-13T09:12:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_history_slow_op_size": "20", 2024-09-13T09:12:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-13T09:12:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_log_threshold": "5", 2024-09-13T09:12:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_num_shards": "0", 2024-09-13T09:12:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_num_shards_hdd": "5", 2024-09-13T09:12:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_num_shards_ssd": "8", 2024-09-13T09:12:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-13T09:12:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-13T09:12:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-13T09:12:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-13T09:12:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_pq_min_cost": "65536", 2024-09-13T09:12:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-13T09:12:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_queue_cut_off": "high", 2024-09-13T09:12:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-13T09:12:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_thread_timeout": "15", 2024-09-13T09:12:11.762 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_open_classes_on_start": "true", 2024-09-13T09:12:11.762 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_os_flags": "0", 2024-09-13T09:12:11.762 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_peering_op_priority": "255", 2024-09-13T09:12:11.762 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_delete_cost": "1048576", 2024-09-13T09:12:11.762 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_delete_priority": "5", 2024-09-13T09:12:11.762 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-13T09:12:11.762 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-13T09:12:11.762 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-13T09:12:11.762 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_log_trim_max": "10000", 2024-09-13T09:12:11.762 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_log_trim_min": "100", 2024-09-13T09:12:11.762 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-13T09:12:11.762 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-13T09:12:11.762 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-13T09:12:11.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-13T09:12:11.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-13T09:12:11.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-13T09:12:11.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-13T09:12:11.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-13T09:12:11.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-13T09:12:11.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_crimson": "false", 2024-09-13T09:12:11.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-13T09:12:11.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-13T09:12:11.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-13T09:12:11.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-13T09:12:11.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-13T09:12:11.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-13T09:12:11.763 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-13T09:12:11.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-13T09:12:11.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_flags": "0", 2024-09-13T09:12:11.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-13T09:12:11.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_min_size": "0", 2024-09-13T09:12:11.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-13T09:12:11.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_pg_num": "32", 2024-09-13T09:12:11.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_pgp_num": "0", 2024-09-13T09:12:11.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-13T09:12:11.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_size": "2", 2024-09-13T09:12:11.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_type": "replicated", 2024-09-13T09:12:11.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-13T09:12:11.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-13T09:12:11.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_push_per_object_cost": "1000", 2024-09-13T09:12:11.765 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-13T09:12:11.765 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recover_clone_overlap": "true", 2024-09-13T09:12:11.765 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-13T09:12:11.765 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_cost": "20971520", 2024-09-13T09:12:11.765 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-13T09:12:11.765 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_max_active": "0", 2024-09-13T09:12:11.765 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-13T09:12:11.765 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-13T09:12:11.765 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-13T09:12:11.765 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-13T09:12:11.765 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_max_single_start": "1", 2024-09-13T09:12:11.765 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_op_priority": "3", 2024-09-13T09:12:11.765 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-13T09:12:11.766 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_priority": "5", 2024-09-13T09:12:11.766 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-13T09:12:11.766 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_sleep": "0.000000", 2024-09-13T09:12:11.766 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-13T09:12:11.766 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-13T09:12:11.766 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-13T09:12:11.766 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_repair_during_recovery": "false", 2024-09-13T09:12:11.766 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_requested_scrub_priority": "120", 2024-09-13T09:12:11.766 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-13T09:12:11.766 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-13T09:12:11.766 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_auto_repair": "false", 2024-09-13T09:12:11.766 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-13T09:12:11.766 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-13T09:12:11.767 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_begin_hour": "0", 2024-09-13T09:12:11.767 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_begin_week_day": "0", 2024-09-13T09:12:11.767 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_chunk_max": "25", 2024-09-13T09:12:11.767 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_chunk_min": "5", 2024-09-13T09:12:11.767 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_cost": "52428800", 2024-09-13T09:12:11.767 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_during_recovery": "false", 2024-09-13T09:12:11.767 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_end_hour": "0", 2024-09-13T09:12:11.767 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_end_week_day": "0", 2024-09-13T09:12:11.767 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_event_cost": "4096", 2024-09-13T09:12:11.767 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-13T09:12:11.767 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-13T09:12:11.767 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_invalid_stats": "true", 2024-09-13T09:12:11.767 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-13T09:12:11.768 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-13T09:12:11.768 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_max_preemptions": "5", 2024-09-13T09:12:11.768 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-13T09:12:11.768 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_priority": "5", 2024-09-13T09:12:11.768 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-13T09:12:11.768 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_sleep": "0.000000", 2024-09-13T09:12:11.768 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-13T09:12:11.768 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-13T09:12:11.768 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-13T09:12:11.768 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-13T09:12:11.768 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_skip_data_digest": "false", 2024-09-13T09:12:11.768 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_smart_report_timeout": "5", 2024-09-13T09:12:11.768 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_snap_trim_cost": "1048576", 2024-09-13T09:12:11.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_snap_trim_priority": "5", 2024-09-13T09:12:11.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-13T09:12:11.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-13T09:12:11.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-13T09:12:11.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-13T09:12:11.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-13T09:12:11.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-13T09:12:11.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-13T09:12:11.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_target_transaction_size": "30", 2024-09-13T09:12:11.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-13T09:12:11.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-13T09:12:11.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-13T09:12:11.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-13T09:12:11.769 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-13T09:12:11.770 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-13T09:12:11.770 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-13T09:12:11.770 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-13T09:12:11.770 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-13T09:12:11.770 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-13T09:12:11.770 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tracing": "false", 2024-09-13T09:12:11.770 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_use_stale_snap": "false", 2024-09-13T09:12:11.770 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-13T09:12:11.770 INFO:teuthology.orchestra.run.smithi044.stdout: "osdc_blkin_trace_all": "false", 2024-09-13T09:12:11.770 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_kill_at": "0", 2024-09-13T09:12:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_max_join_drift": "10", 2024-09-13T09:12:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_min": "500", 2024-09-13T09:12:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_min_wait": "0.050000", 2024-09-13T09:12:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_propose_interval": "1.000000", 2024-09-13T09:12:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_service_trim_max": "500", 2024-09-13T09:12:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-13T09:12:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_service_trim_min": "250", 2024-09-13T09:12:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_stash_full_interval": "25", 2024-09-13T09:12:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_trim_max": "500", 2024-09-13T09:12:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_trim_min": "250", 2024-09-13T09:12:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "perf": "true", 2024-09-13T09:12:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-13T09:12:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-13T09:12:11.772 INFO:teuthology.orchestra.run.smithi044.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-13T09:12:11.772 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "-", 2024-09-13T09:12:11.772 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrv": "", 2024-09-13T09:12:11.772 INFO:teuthology.orchestra.run.smithi044.stdout: "public_bind_addr": "-", 2024-09-13T09:12:11.772 INFO:teuthology.orchestra.run.smithi044.stdout: "public_network": "", 2024-09-13T09:12:11.772 INFO:teuthology.orchestra.run.smithi044.stdout: "public_network_interface": "", 2024-09-13T09:12:11.772 INFO:teuthology.orchestra.run.smithi044.stdout: "qat_compressor_enabled": "false", 2024-09-13T09:12:11.772 INFO:teuthology.orchestra.run.smithi044.stdout: "qat_compressor_session_max_number": "256", 2024-09-13T09:12:11.772 INFO:teuthology.orchestra.run.smithi044.stdout: "rados_mon_op_timeout": "0", 2024-09-13T09:12:11.772 INFO:teuthology.orchestra.run.smithi044.stdout: "rados_osd_op_timeout": "0", 2024-09-13T09:12:11.773 INFO:teuthology.orchestra.run.smithi044.stdout: "rados_tracing": "false", 2024-09-13T09:12:11.773 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_atime_update_interval": "60", 2024-09-13T09:12:11.773 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-13T09:12:11.773 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_balance_parent_reads": "false", 2024-09-13T09:12:11.773 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_balance_snap_reads": "false", 2024-09-13T09:12:11.773 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_blkin_trace_all": "false", 2024-09-13T09:12:11.773 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-13T09:12:11.773 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-13T09:12:11.773 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache": "true", 2024-09-13T09:12:11.773 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-13T09:12:11.773 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-13T09:12:11.773 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-13T09:12:11.773 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-13T09:12:11.774 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_policy": "writearound", 2024-09-13T09:12:11.774 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_size": "33554432", 2024-09-13T09:12:11.774 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-13T09:12:11.774 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-13T09:12:11.774 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_clone_copy_on_read": "false", 2024-09-13T09:12:11.774 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_compression_hint": "none", 2024-09-13T09:12:11.774 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_concurrent_management_ops": "10", 2024-09-13T09:12:11.774 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-13T09:12:11.774 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_clone_format": "auto", 2024-09-13T09:12:11.774 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_data_pool": "", 2024-09-13T09:12:11.774 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_features": "61", 2024-09-13T09:12:11.774 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_format": "2", 2024-09-13T09:12:11.774 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_map_options": "", 2024-09-13T09:12:11.774 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_order": "22", 2024-09-13T09:12:11.775 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_pool": "rbd", 2024-09-13T09:12:11.775 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-13T09:12:11.775 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_stripe_count": "0", 2024-09-13T09:12:11.775 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_stripe_unit": "0", 2024-09-13T09:12:11.775 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-13T09:12:11.775 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-13T09:12:11.775 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-13T09:12:11.775 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_enable_alloc_hint": "true", 2024-09-13T09:12:11.775 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-13T09:12:11.775 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_io_scheduler": "simple", 2024-09-13T09:12:11.775 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-13T09:12:11.775 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-13T09:12:11.775 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-13T09:12:11.776 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-13T09:12:11.776 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-13T09:12:11.776 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-13T09:12:11.776 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-13T09:12:11.776 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-13T09:12:11.776 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-13T09:12:11.776 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_order": "24", 2024-09-13T09:12:11.776 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_pool": "", 2024-09-13T09:12:11.776 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_splay_width": "4", 2024-09-13T09:12:11.776 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_localize_parent_reads": "false", 2024-09-13T09:12:11.776 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_localize_snap_reads": "false", 2024-09-13T09:12:11.776 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-13T09:12:11.776 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-13T09:12:11.776 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-13T09:12:11.777 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-13T09:12:11.777 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-13T09:12:11.777 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-13T09:12:11.777 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-13T09:12:11.777 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-13T09:12:11.777 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-13T09:12:11.777 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-13T09:12:11.777 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-13T09:12:11.777 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-13T09:12:11.777 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-13T09:12:11.777 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-13T09:12:11.777 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-13T09:12:11.777 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-13T09:12:11.777 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-13T09:12:11.778 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-13T09:12:11.778 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-13T09:12:11.778 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-13T09:12:11.778 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-13T09:12:11.778 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-13T09:12:11.778 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-13T09:12:11.778 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-13T09:12:11.778 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-13T09:12:11.778 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-13T09:12:11.778 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-13T09:12:11.778 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-13T09:12:11.778 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-13T09:12:11.778 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-13T09:12:11.779 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-13T09:12:11.779 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mtime_update_interval": "60", 2024-09-13T09:12:11.779 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_non_blocking_aio": "true", 2024-09-13T09:12:11.779 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_op_thread_timeout": "60", 2024-09-13T09:12:11.779 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_op_threads": "1", 2024-09-13T09:12:11.779 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_parent_cache_enabled": "false", 2024-09-13T09:12:11.779 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-13T09:12:11.779 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-13T09:12:11.779 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-13T09:12:11.779 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_plugins": "", 2024-09-13T09:12:11.779 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_bps_burst": "0", 2024-09-13T09:12:11.779 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-13T09:12:11.779 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_bps_limit": "0", 2024-09-13T09:12:11.779 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_exclude_ops": "0", 2024-09-13T09:12:11.780 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_iops_burst": "0", 2024-09-13T09:12:11.780 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-13T09:12:11.780 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_iops_limit": "0", 2024-09-13T09:12:11.780 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-13T09:12:11.780 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-13T09:12:11.780 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-13T09:12:11.780 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-13T09:12:11.780 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-13T09:12:11.780 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-13T09:12:11.780 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-13T09:12:11.780 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-13T09:12:11.780 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-13T09:12:11.781 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-13T09:12:11.781 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-13T09:12:11.781 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-13T09:12:11.781 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-13T09:12:11.781 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-13T09:12:11.781 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_read_from_replica_policy": "default", 2024-09-13T09:12:11.781 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-13T09:12:11.781 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-13T09:12:11.781 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-13T09:12:11.781 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-13T09:12:11.781 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_skip_partial_discard": "true", 2024-09-13T09:12:11.781 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-13T09:12:11.781 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_tracing": "false", 2024-09-13T09:12:11.781 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_validate_names": "true", 2024-09-13T09:12:11.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_validate_pool": "true", 2024-09-13T09:12:11.782 INFO:teuthology.orchestra.run.smithi044.stdout: "restapi_base_url": "", 2024-09-13T09:12:11.782 INFO:teuthology.orchestra.run.smithi044.stdout: "restapi_log_level": "", 2024-09-13T09:12:11.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_acl_grants_max_num": "100", 2024-09-13T09:12:11.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_admin_entry": "admin", 2024-09-13T09:12:11.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-13T09:12:11.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_backend_store": "rados", 2024-09-13T09:12:11.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_barbican_url": "", 2024-09-13T09:12:11.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_beast_enable_async": "true", 2024-09-13T09:12:11.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-13T09:12:11.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-13T09:12:11.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-13T09:12:11.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-13T09:12:11.782 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-13T09:12:11.783 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-13T09:12:11.783 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-13T09:12:11.783 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_cache_enabled": "true", 2024-09-13T09:12:11.783 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_cache_expiry_interval": "900", 2024-09-13T09:12:11.783 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_cache_lru_size": "10000", 2024-09-13T09:12:11.783 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_config_store": "rados", 2024-09-13T09:12:11.783 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_content_length_compat": "false", 2024-09-13T09:12:11.783 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_copy_obj_progress": "true", 2024-09-13T09:12:11.783 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-13T09:12:11.783 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_cors_rules_max_num": "100", 2024-09-13T09:12:11.783 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_cross_domain_policy": "", 2024-09-13T09:12:11.783 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-13T09:12:11.783 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_addr": "", 2024-09-13T09:12:11.783 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-13T09:12:11.784 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-13T09:12:11.784 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-13T09:12:11.784 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-13T09:12:11.784 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_password": "", 2024-09-13T09:12:11.784 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-13T09:12:11.784 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_username": "", 2024-09-13T09:12:11.784 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_require_ssl": "true", 2024-09-13T09:12:11.784 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-13T09:12:11.784 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-13T09:12:11.784 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-13T09:12:11.784 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-13T09:12:11.784 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-13T09:12:11.784 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-13T09:12:11.785 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-13T09:12:11.785 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-13T09:12:11.788 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-13T09:12:11.788 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-13T09:12:11.788 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-13T09:12:11.788 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-13T09:12:11.788 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-13T09:12:11.788 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-13T09:12:11.788 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-13T09:12:11.788 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_addr": "", 2024-09-13T09:12:11.788 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_auth": "token", 2024-09-13T09:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_namespace": "", 2024-09-13T09:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_prefix": "", 2024-09-13T09:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-13T09:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-13T09:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-13T09:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-13T09:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_token_file": "", 2024-09-13T09:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-13T09:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_curl_buffersize": "524288", 2024-09-13T09:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-13T09:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_curl_low_speed_time": "300", 2024-09-13T09:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-13T09:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-13T09:12:11.789 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-13T09:12:11.790 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-13T09:12:11.790 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-13T09:12:11.790 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-13T09:12:11.790 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-13T09:12:11.790 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-13T09:12:11.790 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-13T09:12:11.790 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-13T09:12:11.790 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-13T09:12:11.790 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data_log_changes_size": "1000", 2024-09-13T09:12:11.790 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data_log_num_shards": "128", 2024-09-13T09:12:11.790 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-13T09:12:11.790 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data_log_window": "30", 2024-09-13T09:12:11.790 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-13T09:12:11.790 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-13T09:12:11.791 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-13T09:12:11.791 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-13T09:12:11.791 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-13T09:12:11.791 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-13T09:12:11.791 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-13T09:12:11.791 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-13T09:12:11.791 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-13T09:12:11.791 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-13T09:12:11.791 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-13T09:12:11.791 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-13T09:12:11.791 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-13T09:12:11.791 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-13T09:12:11.791 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-13T09:12:11.791 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-13T09:12:11.792 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-13T09:12:11.792 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-13T09:12:11.792 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-13T09:12:11.792 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-13T09:12:11.792 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-13T09:12:11.792 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-13T09:12:11.792 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-13T09:12:11.792 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-13T09:12:11.792 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dns_name": "", 2024-09-13T09:12:11.792 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dns_s3website_name": "", 2024-09-13T09:12:11.792 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dynamic_resharding": "true", 2024-09-13T09:12:11.792 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-13T09:12:11.792 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enable_gc_threads": "true", 2024-09-13T09:12:11.793 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enable_lc_threads": "true", 2024-09-13T09:12:11.793 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enable_ops_log": "false", 2024-09-13T09:12:11.793 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enable_quota_threads": "true", 2024-09-13T09:12:11.793 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enable_static_website": "false", 2024-09-13T09:12:11.793 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enable_usage_log": "false", 2024-09-13T09:12:11.793 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enforce_swift_acls": "true", 2024-09-13T09:12:11.793 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_exit_timeout_secs": "120", 2024-09-13T09:12:11.793 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_expose_bucket": "false", 2024-09-13T09:12:11.793 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_extended_http_attrs": "", 2024-09-13T09:12:11.793 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_filter": "none", 2024-09-13T09:12:11.793 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-13T09:12:11.793 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_frontends": "beast port=7480", 2024-09-13T09:12:11.793 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-13T09:12:11.793 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_max_deferred": "50", 2024-09-13T09:12:11.794 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-13T09:12:11.794 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_max_objs": "32", 2024-09-13T09:12:11.794 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-13T09:12:11.794 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-13T09:12:11.794 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-13T09:12:11.794 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-13T09:12:11.794 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_processor_period": "3600", 2024-09-13T09:12:11.794 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-13T09:12:11.794 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-13T09:12:11.794 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-13T09:12:11.794 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-13T09:12:11.794 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_init_timeout": "300", 2024-09-13T09:12:11.794 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-13T09:12:11.794 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-13T09:12:11.795 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-13T09:12:11.795 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-13T09:12:11.795 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_domain": "", 2024-09-13T09:12:11.795 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_password": "", 2024-09-13T09:12:11.795 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_password_path": "", 2024-09-13T09:12:11.795 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_project": "", 2024-09-13T09:12:11.795 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_tenant": "", 2024-09-13T09:12:11.795 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_token": "", 2024-09-13T09:12:11.795 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_token_path": "", 2024-09-13T09:12:11.795 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_user": "", 2024-09-13T09:12:11.795 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_api_version": "2", 2024-09-13T09:12:11.795 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_barbican_domain": "", 2024-09-13T09:12:11.795 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_barbican_password": "", 2024-09-13T09:12:11.795 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_barbican_project": "", 2024-09-13T09:12:11.796 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-13T09:12:11.796 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_barbican_user": "", 2024-09-13T09:12:11.796 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-13T09:12:11.796 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-13T09:12:11.796 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-13T09:12:11.796 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-13T09:12:11.796 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-13T09:12:11.796 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_url": "", 2024-09-13T09:12:11.796 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-13T09:12:11.796 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lc_debug_interval": "-1", 2024-09-13T09:12:11.796 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lc_lock_max_time": "90", 2024-09-13T09:12:11.796 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lc_max_objs": "32", 2024-09-13T09:12:11.796 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lc_max_rules": "1000", 2024-09-13T09:12:11.797 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lc_max_worker": "3", 2024-09-13T09:12:11.797 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-13T09:12:11.797 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lc_thread_delay": "0", 2024-09-13T09:12:11.797 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-13T09:12:11.797 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ldap_dnattr": "uid", 2024-09-13T09:12:11.797 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-13T09:12:11.797 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ldap_searchfilter": "", 2024-09-13T09:12:11.797 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-13T09:12:11.797 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-13T09:12:11.797 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-13T09:12:11.797 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-13T09:12:11.797 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-13T09:12:11.797 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_log_http_headers": "", 2024-09-13T09:12:11.797 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-13T09:12:11.798 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-13T09:12:11.798 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_log_object_name_utc": "false", 2024-09-13T09:12:11.798 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-13T09:12:11.798 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_attr_name_len": "0", 2024-09-13T09:12:11.798 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_attr_size": "0", 2024-09-13T09:12:11.798 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-13T09:12:11.798 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_chunk_size": "4194304", 2024-09-13T09:12:11.798 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-13T09:12:11.798 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-13T09:12:11.798 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-13T09:12:11.798 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_listing_results": "1000", 2024-09-13T09:12:11.798 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_notify_retries": "10", 2024-09-13T09:12:11.798 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-13T09:12:11.799 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_put_param_size": "1048576", 2024-09-13T09:12:11.799 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_put_size": "5368709120", 2024-09-13T09:12:11.799 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_slo_entries": "1000", 2024-09-13T09:12:11.799 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_md_log_max_shards": "64", 2024-09-13T09:12:11.799 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-13T09:12:11.799 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-13T09:12:11.799 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-13T09:12:11.799 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-13T09:12:11.799 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_mp_lock_max_time": "600", 2024-09-13T09:12:11.799 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-13T09:12:11.799 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-13T09:12:11.799 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-13T09:12:11.799 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-13T09:12:11.799 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-13T09:12:11.800 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-13T09:12:11.800 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-13T09:12:11.800 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-13T09:12:11.800 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_max_gc": "300", 2024-09-13T09:12:11.800 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-13T09:12:11.800 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-13T09:12:11.800 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-13T09:12:11.800 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-13T09:12:11.800 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-13T09:12:11.800 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-13T09:12:11.800 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-13T09:12:11.800 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_num_async_rados_threads": "32", 2024-09-13T09:12:11.800 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_num_control_oids": "8", 2024-09-13T09:12:11.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_numa_node": "-1", 2024-09-13T09:12:11.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-13T09:12:11.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-13T09:12:11.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_objexp_chunk_size": "100", 2024-09-13T09:12:11.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_objexp_gc_interval": "600", 2024-09-13T09:12:11.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-13T09:12:11.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-13T09:12:11.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-13T09:12:11.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_op_thread_timeout": "600", 2024-09-13T09:12:11.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_op_tracing": "false", 2024-09-13T09:12:11.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_opa_token": "", 2024-09-13T09:12:11.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_opa_url": "", 2024-09-13T09:12:11.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_opa_verify_ssl": "true", 2024-09-13T09:12:11.801 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-13T09:12:11.802 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-13T09:12:11.802 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ops_log_rados": "false", 2024-09-13T09:12:11.802 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ops_log_socket_path": "", 2024-09-13T09:12:11.802 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-13T09:12:11.802 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-13T09:12:11.802 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-13T09:12:11.802 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_period_push_interval": "2.000000", 2024-09-13T09:12:11.802 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-13T09:12:11.802 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-13T09:12:11.802 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-13T09:12:11.802 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_print_continue": "true", 2024-09-13T09:12:11.802 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-13T09:12:11.802 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-13T09:12:11.802 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-13T09:12:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-13T09:12:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-13T09:12:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_rados_tracing": "false", 2024-09-13T09:12:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_realm": "", 2024-09-13T09:12:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_realm_id": "", 2024-09-13T09:12:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-13T09:12:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_region": "", 2024-09-13T09:12:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-13T09:12:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-13T09:12:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-13T09:12:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-13T09:12:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_request_uri": "", 2024-09-13T09:12:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_reshard_batch_size": "64", 2024-09-13T09:12:11.803 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-13T09:12:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_reshard_max_aio": "128", 2024-09-13T09:12:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_reshard_num_logs": "16", 2024-09-13T09:12:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_reshard_thread_interval": "600", 2024-09-13T09:12:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_resolve_cname": "false", 2024-09-13T09:12:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-13T09:12:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_run_sync_thread": "true", 2024-09-13T09:12:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-13T09:12:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-13T09:12:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-13T09:12:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-13T09:12:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-13T09:12:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-13T09:12:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-13T09:12:11.804 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-13T09:12:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_scheduler_type": "throttler", 2024-09-13T09:12:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_script_uri": "", 2024-09-13T09:12:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_service_provider_name": "", 2024-09-13T09:12:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-13T09:12:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sts_client_id": "", 2024-09-13T09:12:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sts_client_secret": "", 2024-09-13T09:12:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sts_entry": "sts", 2024-09-13T09:12:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sts_key": "sts", 2024-09-13T09:12:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-13T09:12:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sts_min_session_duration": "900", 2024-09-13T09:12:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sts_token_introspection_url": "", 2024-09-13T09:12:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_account_in_url": "false", 2024-09-13T09:12:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_auth_entry": "auth", 2024-09-13T09:12:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_auth_url": "", 2024-09-13T09:12:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_custom_header": "", 2024-09-13T09:12:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-13T09:12:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_need_stats": "true", 2024-09-13T09:12:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_tenant_name": "", 2024-09-13T09:12:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_token_expiration": "86400", 2024-09-13T09:12:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_url": "", 2024-09-13T09:12:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_url_prefix": "swift", 2024-09-13T09:12:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-13T09:12:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-13T09:12:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-13T09:12:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_lease_period": "120", 2024-09-13T09:12:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-13T09:12:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-13T09:12:11.806 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-13T09:12:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-13T09:12:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-13T09:12:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-13T09:12:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-13T09:12:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-13T09:12:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-13T09:12:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_thread_pool_size": "512", 2024-09-13T09:12:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_torrent_comment": "", 2024-09-13T09:12:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_torrent_createby": "", 2024-09-13T09:12:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_torrent_encoding": "", 2024-09-13T09:12:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_torrent_flag": "false", 2024-09-13T09:12:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_torrent_origin": "", 2024-09-13T09:12:11.807 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-13T09:12:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_torrent_tracker": "", 2024-09-13T09:12:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_trust_forwarded_https": "false", 2024-09-13T09:12:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-13T09:12:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-13T09:12:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_usage_max_shards": "32", 2024-09-13T09:12:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_usage_max_user_shards": "1", 2024-09-13T09:12:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_use_opa_authz": "false", 2024-09-13T09:12:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-13T09:12:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-13T09:12:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_max_buckets": "1000", 2024-09-13T09:12:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_policies_max_num": "100", 2024-09-13T09:12:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-13T09:12:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-13T09:12:11.808 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-13T09:12:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-13T09:12:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_unique_email": "true", 2024-09-13T09:12:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_verify_ssl": "true", 2024-09-13T09:12:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-13T09:12:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_zone": "", 2024-09-13T09:12:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_zone_id": "", 2024-09-13T09:12:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-13T09:12:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_zonegroup": "", 2024-09-13T09:12:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_zonegroup_id": "", 2024-09-13T09:12:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-13T09:12:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "rgwlc_auto_session_clear": "true", 2024-09-13T09:12:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-13T09:12:11.809 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_block_size": "4096", 2024-09-13T09:12:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-13T09:12:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-13T09:12:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-13T09:12:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-13T09:12:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-13T09:12:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cache_size": "536870912", 2024-09-13T09:12:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-13T09:12:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-13T09:12:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-13T09:12:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-13T09:12:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-13T09:12:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-13T09:12:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-13T09:12:11.810 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-13T09:12:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_index_type": "binary_search", 2024-09-13T09:12:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-13T09:12:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-13T09:12:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_partition_filters": "false", 2024-09-13T09:12:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_perf": "false", 2024-09-13T09:12:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-13T09:12:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-13T09:12:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-13T09:12:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "run_dir": "/var/run/ceph", 2024-09-13T09:12:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_block_create": "true", 2024-09-13T09:12:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_cache_lru_size": "67108864", 2024-09-13T09:12:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-13T09:12:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_default_max_object_size": "16777216", 2024-09-13T09:12:11.811 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-13T09:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_device_size": "53687091200", 2024-09-13T09:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_journal_batch_capacity": "16", 2024-09-13T09:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-13T09:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-13T09:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-13T09:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_main_device_type": "SSD", 2024-09-13T09:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-13T09:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-13T09:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-13T09:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-13T09:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-13T09:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_segment_size": "67108864", 2024-09-13T09:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "set_keepcaps": "false", 2024-09-13T09:12:11.812 INFO:teuthology.orchestra.run.smithi044.stdout: "setgroup": "", 2024-09-13T09:12:11.813 INFO:teuthology.orchestra.run.smithi044.stdout: "setuser": "", 2024-09-13T09:12:11.813 INFO:teuthology.orchestra.run.smithi044.stdout: "setuser_match_path": "", 2024-09-13T09:12:11.813 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-13T09:12:11.813 INFO:teuthology.orchestra.run.smithi044.stdout: "thp": "false", 2024-09-13T09:12:11.813 INFO:teuthology.orchestra.run.smithi044.stdout: "threadpool_default_timeout": "60", 2024-09-13T09:12:11.813 INFO:teuthology.orchestra.run.smithi044.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-13T09:12:11.813 INFO:teuthology.orchestra.run.smithi044.stdout: "throttler_perf_counter": "true" 2024-09-13T09:12:11.813 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:11.815 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-13T09:12:11.931 INFO:teuthology.orchestra.run.smithi044.stdout:mon.a: {} 2024-09-13T09:12:11.931 INFO:teuthology.orchestra.run.smithi044.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:12:11.942 INFO:teuthology.orchestra.run.smithi044.stdout:mon.b: {} 2024-09-13T09:12:11.942 INFO:teuthology.orchestra.run.smithi044.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:12:11.950 INFO:teuthology.orchestra.run.smithi044.stdout:mon.c: {} 2024-09-13T09:12:11.951 INFO:teuthology.orchestra.run.smithi044.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:12:11.961 DEBUG:teuthology.orchestra.run.smithi129:> type ceph-objectstore-tool 2024-09-13T09:12:11.980 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T09:12:11.980 DEBUG:teuthology.orchestra.run.smithi044:> type ceph-objectstore-tool 2024-09-13T09:12:11.995 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T09:12:11.996 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-13T09:12:12.005 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-13T09:12:12.006 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-13T09:12:12.006 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-13T09:12:12.006 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:12:12.006 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 2, 3, 4, 5] 2024-09-13T09:12:12.007 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-13T09:12:12.007 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-13T09:12:12.007 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-13T09:12:12.007 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:12:12.008 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-13T09:12:12.008 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:12.008 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-13T09:12:12.008 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:12:12.009 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0 2024-09-13T09:12:12.021 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:12.019+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:12.069 INFO:teuthology.orchestra.run.smithi157.stdout:mapped writes DISABLED 2024-09-13T09:12:12.069 INFO:teuthology.orchestra.run.smithi157.stdout:mapped reads DISABLED 2024-09-13T09:12:12.069 INFO:teuthology.orchestra.run.smithi157.stdout:Seed set to 7073 2024-09-13T09:12:12.069 INFO:teuthology.orchestra.run.smithi157.stdout:rbd-nbd mode enabled 2024-09-13T09:12:12.109 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:12.107+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:12.151 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:12.152 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:12.152 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:12.152 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:12.165 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:12.168 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:12.168 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:12.190 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:12.190 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:12.207 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:12.207 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:12.209 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:12.207+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:12.224 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:12.224 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:12.241 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:12.241 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:12.259 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:12.259 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:12.277 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:12.277 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:12.286 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:12.310 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:12.308+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:12.358 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-09-13T09:12:12.357+0000 7f251c3a0640 -1 mon.a@0(leader).osd e21 definitely_dead 0 2024-09-13T09:12:12.411 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:12.409+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:12.439 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:12.439 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:12.448 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:12.532 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:12.531+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:12.595 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:12.595 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:12.605 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:12.611 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:12.610+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:12.712 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:12.711+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:12.751 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:12.751 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:12.760 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:12.813 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:12.812+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:12.867 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:12.867 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:12.875 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:12.917 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:12.916+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:13.015 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:13.014+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:13.023 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:13.023 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:13.032 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:13.116 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:13.115+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:13.138 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:13.139 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:13.147 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:13.217 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:13.216+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:13.254 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:13.254 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:13.264 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:13.318 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:13.317+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:13.362 INFO:teuthology.orchestra.run.smithi044.stderr:marked out osd.3. 2024-09-13T09:12:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-09-13T09:12:13.385 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:13.385 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:13.395 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:13.423 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:13.422+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:13.499 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:13.499 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:13.507 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:13.551 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:13.549+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:13.609 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:13.610 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:13.618 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:13.660 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:13.658+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:13.722 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:13.720+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:13.762 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:13.770 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:13.855 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:13.853+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:13.912 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:13.912 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:13.920 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:13.924 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:13.923+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:14.025 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:14.023+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:14.064 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:14.064 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:14.073 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:14.157 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:14.155+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:14.216 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:14.216 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:14.224 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:14.227 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:14.226+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:14.271 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:12:14.328 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:14.326+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:14.370 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:14.370 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:14.381 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:14.412 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:14.413 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:14.429 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:14.429 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:14.445 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:14.445 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:14.462 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:14.462 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:14.467 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:14.466+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:14.478 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:14.478 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:14.495 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:14.496 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:14.529 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:14.529 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:14.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:14.529 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:14.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.529 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4221.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:12:14.529 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:04.320224+0000", 2024-09-13T09:12:14.529 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.207244351, 2024-09-13T09:12:14.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010683929, 2024-09-13T09:12:14.529 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:14.530 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:14.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4221", 2024-09-13T09:12:14.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.44:0/558773684", 2024-09-13T09:12:14.530 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1 2024-09-13T09:12:14.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.530 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:14.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:14.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320224+0000", 2024-09-13T09:12:14.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:14.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320224+0000", 2024-09-13T09:12:14.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:14.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320220+0000", 2024-09-13T09:12:14.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999967 2024-09-13T09:12:14.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:14.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320230+0000", 2024-09-13T09:12:14.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0275000000000001e-05 2024-09-13T09:12:14.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:14.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320241+0000", 2024-09-13T09:12:14.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0154e-05 2024-09-13T09:12:14.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320256+0000", 2024-09-13T09:12:14.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5068999999999999e-05 2024-09-13T09:12:14.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:14.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320328+0000", 2024-09-13T09:12:14.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2507000000000002e-05 2024-09-13T09:12:14.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:14.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320336+0000", 2024-09-13T09:12:14.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:12:14.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:14.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.322475+0000", 2024-09-13T09:12:14.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021399430000000001 2024-09-13T09:12:14.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:14.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.322484+0000", 2024-09-13T09:12:14.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5779999999999999e-06 2024-09-13T09:12:14.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:14.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.325065+0000", 2024-09-13T09:12:14.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002581323 2024-09-13T09:12:14.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:14.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.325211+0000", 2024-09-13T09:12:14.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014534899999999999 2024-09-13T09:12:14.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:14.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.325374+0000", 2024-09-13T09:12:14.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000162898 2024-09-13T09:12:14.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:14.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.326528+0000", 2024-09-13T09:12:14.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001154415 2024-09-13T09:12:14.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:14.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.330811+0000", 2024-09-13T09:12:14.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042825090000000003 2024-09-13T09:12:14.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:14.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.330862+0000", 2024-09-13T09:12:14.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1430000000000001e-05 2024-09-13T09:12:14.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:14.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.330908+0000", 2024-09-13T09:12:14.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5636000000000001e-05 2024-09-13T09:12:14.536 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.536 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:14.536 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.536 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop_reply(client.4221.0:1 2.3 e17/16 ondisk, result = 0)", 2024-09-13T09:12:14.537 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:04.326465+0000", 2024-09-13T09:12:14.537 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.201002947999999, 2024-09-13T09:12:14.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5619999999999999e-05, 2024-09-13T09:12:14.537 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:14.537 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:14.537 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:14.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:14.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.326465+0000", 2024-09-13T09:12:14.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:14.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.326465+0000", 2024-09-13T09:12:14.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:14.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.326464+0000", 2024-09-13T09:12:14.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:14.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:14.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.326469+0000", 2024-09-13T09:12:14.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9830000000000004e-06 2024-09-13T09:12:14.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:14.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.326480+0000", 2024-09-13T09:12:14.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0705e-05 2024-09-13T09:12:14.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.326486+0000", 2024-09-13T09:12:14.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1210000000000003e-06 2024-09-13T09:12:14.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:14.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.326519+0000", 2024-09-13T09:12:14.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2951999999999999e-05 2024-09-13T09:12:14.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:14.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.326525+0000", 2024-09-13T09:12:14.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5659999999999997e-06 2024-09-13T09:12:14.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:14.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.326531+0000", 2024-09-13T09:12:14.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1020000000000002e-06 2024-09-13T09:12:14.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.540 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:14.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.540 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4221.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:12:14.541 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:04.333551+0000", 2024-09-13T09:12:14.541 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.19391677, 2024-09-13T09:12:14.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00041999799999999999, 2024-09-13T09:12:14.541 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:14.541 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:14.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:14.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4221", 2024-09-13T09:12:14.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.44:0/558773684", 2024-09-13T09:12:14.541 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2 2024-09-13T09:12:14.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.541 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:14.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:14.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.333551+0000", 2024-09-13T09:12:14.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:14.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.333551+0000", 2024-09-13T09:12:14.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:14.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.333549+0000", 2024-09-13T09:12:14.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:12:14.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:14.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.333563+0000", 2024-09-13T09:12:14.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3609e-05 2024-09-13T09:12:14.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:14.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.333593+0000", 2024-09-13T09:12:14.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0546999999999997e-05 2024-09-13T09:12:14.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.333613+0000", 2024-09-13T09:12:14.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9942e-05 2024-09-13T09:12:14.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:14.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.333757+0000", 2024-09-13T09:12:14.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014353099999999999 2024-09-13T09:12:14.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:14.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.333971+0000", 2024-09-13T09:12:14.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021462200000000001 2024-09-13T09:12:14.544 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.544 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:14.544 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.545 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4221.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:12:14.545 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:04.334214+0000", 2024-09-13T09:12:14.545 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.193253669000001, 2024-09-13T09:12:14.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000126792, 2024-09-13T09:12:14.545 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:14.545 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:14.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:14.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4221", 2024-09-13T09:12:14.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.44:0/558773684", 2024-09-13T09:12:14.545 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3 2024-09-13T09:12:14.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.545 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:14.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:14.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.334214+0000", 2024-09-13T09:12:14.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:14.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.334214+0000", 2024-09-13T09:12:14.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:14.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.334213+0000", 2024-09-13T09:12:14.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:14.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:14.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.334222+0000", 2024-09-13T09:12:14.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.098e-06 2024-09-13T09:12:14.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:14.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.334237+0000", 2024-09-13T09:12:14.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4771e-05 2024-09-13T09:12:14.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.334243+0000", 2024-09-13T09:12:14.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1249999999999998e-06 2024-09-13T09:12:14.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:14.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.334279+0000", 2024-09-13T09:12:14.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5985999999999997e-05 2024-09-13T09:12:14.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:14.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.334341+0000", 2024-09-13T09:12:14.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1897999999999996e-05 2024-09-13T09:12:14.548 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.548 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:14.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.549 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4221.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:12:14.549 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:05.322972+0000", 2024-09-13T09:12:14.549 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.2044962049999999, 2024-09-13T09:12:14.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0041565500000000002, 2024-09-13T09:12:14.549 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:14.549 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.549 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:14.549 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4221", 2024-09-13T09:12:14.549 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.44:0/558773684", 2024-09-13T09:12:14.549 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5 2024-09-13T09:12:14.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.550 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:14.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:14.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.322972+0000", 2024-09-13T09:12:14.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:14.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.322972+0000", 2024-09-13T09:12:14.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:14.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.322970+0000", 2024-09-13T09:12:14.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:12:14.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:14.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.322980+0000", 2024-09-13T09:12:14.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0866999999999999e-05 2024-09-13T09:12:14.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:14.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.322999+0000", 2024-09-13T09:12:14.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8349999999999999e-05 2024-09-13T09:12:14.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.323016+0000", 2024-09-13T09:12:14.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6988999999999999e-05 2024-09-13T09:12:14.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:14.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.323090+0000", 2024-09-13T09:12:14.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4369000000000006e-05 2024-09-13T09:12:14.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:14.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.323098+0000", 2024-09-13T09:12:14.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3760000000000002e-06 2024-09-13T09:12:14.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:14.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.325767+0000", 2024-09-13T09:12:14.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026697729999999999 2024-09-13T09:12:14.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:14.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.325881+0000", 2024-09-13T09:12:14.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011373499999999999 2024-09-13T09:12:14.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:14.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.325985+0000", 2024-09-13T09:12:14.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103627 2024-09-13T09:12:14.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:14.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.326752+0000", 2024-09-13T09:12:14.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00076745899999999996 2024-09-13T09:12:14.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:14.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.327067+0000", 2024-09-13T09:12:14.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000314369 2024-09-13T09:12:14.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.327104+0000", 2024-09-13T09:12:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7320000000000002e-05 2024-09-13T09:12:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.327128+0000", 2024-09-13T09:12:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4527000000000001e-05 2024-09-13T09:12:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop_reply(client.4221.0:5 2.2 e18/16 ondisk, result = 0)", 2024-09-13T09:12:14.555 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:05.327001+0000", 2024-09-13T09:12:14.556 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.2004669309999993, 2024-09-13T09:12:14.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013511999999999999, 2024-09-13T09:12:14.556 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:14.556 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:14.556 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:14.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:14.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.327001+0000", 2024-09-13T09:12:14.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:14.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.327001+0000", 2024-09-13T09:12:14.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:14.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.327000+0000", 2024-09-13T09:12:14.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:14.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:14.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.327007+0000", 2024-09-13T09:12:14.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:12:14.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:14.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.327017+0000", 2024-09-13T09:12:14.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:12:14.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.327024+0000", 2024-09-13T09:12:14.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7170000000000002e-06 2024-09-13T09:12:14.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:14.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.327057+0000", 2024-09-13T09:12:14.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3343000000000002e-05 2024-09-13T09:12:14.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:14.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.327063+0000", 2024-09-13T09:12:14.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.857e-06 2024-09-13T09:12:14.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:14.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.327136+0000", 2024-09-13T09:12:14.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3319999999999996e-05 2024-09-13T09:12:14.559 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.559 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:14.559 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.559 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4221.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:12:14.559 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:06.327956+0000", 2024-09-13T09:12:14.559 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.1995119209999991, 2024-09-13T09:12:14.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033160099999999999, 2024-09-13T09:12:14.560 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:14.560 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.560 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:14.560 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4221", 2024-09-13T09:12:14.560 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.44:0/558773684", 2024-09-13T09:12:14.560 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 7 2024-09-13T09:12:14.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.560 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:14.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:14.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.327956+0000", 2024-09-13T09:12:14.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:14.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.327956+0000", 2024-09-13T09:12:14.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:14.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.327954+0000", 2024-09-13T09:12:14.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:12:14.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:14.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.327965+0000", 2024-09-13T09:12:14.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0861e-05 2024-09-13T09:12:14.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:14.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.327983+0000", 2024-09-13T09:12:14.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8189000000000002e-05 2024-09-13T09:12:14.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.327995+0000", 2024-09-13T09:12:14.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1943000000000001e-05 2024-09-13T09:12:14.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:14.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.328059+0000", 2024-09-13T09:12:14.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3905000000000006e-05 2024-09-13T09:12:14.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:14.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.328065+0000", 2024-09-13T09:12:14.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3450000000000002e-06 2024-09-13T09:12:14.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:14.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.329859+0000", 2024-09-13T09:12:14.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0017938209999999999 2024-09-13T09:12:14.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:14.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.329928+0000", 2024-09-13T09:12:14.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8823000000000002e-05 2024-09-13T09:12:14.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:14.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.330025+0000", 2024-09-13T09:12:14.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7536999999999998e-05 2024-09-13T09:12:14.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:14.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.330915+0000", 2024-09-13T09:12:14.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00088944299999999996 2024-09-13T09:12:14.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:14.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.331211+0000", 2024-09-13T09:12:14.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029682500000000002 2024-09-13T09:12:14.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:14.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.331245+0000", 2024-09-13T09:12:14.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3454999999999997e-05 2024-09-13T09:12:14.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:14.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.331272+0000", 2024-09-13T09:12:14.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7341000000000001e-05 2024-09-13T09:12:14.565 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.566 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:14.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.566 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop_reply(client.4221.0:7 2.2 e19/16 ondisk, result = 0)", 2024-09-13T09:12:14.566 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:06.331133+0000", 2024-09-13T09:12:14.566 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.1963347239999997, 2024-09-13T09:12:14.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014417400000000001, 2024-09-13T09:12:14.566 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:14.566 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:14.566 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:14.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:14.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.331133+0000", 2024-09-13T09:12:14.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:14.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.331133+0000", 2024-09-13T09:12:14.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:14.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:14.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.331132+0000", 2024-09-13T09:12:14.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:14.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:14.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.331147+0000", 2024-09-13T09:12:14.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5051999999999999e-05 2024-09-13T09:12:14.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:14.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.331161+0000", 2024-09-13T09:12:14.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4037e-05 2024-09-13T09:12:14.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.331169+0000", 2024-09-13T09:12:14.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2649999999999999e-06 2024-09-13T09:12:14.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:14.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.331200+0000", 2024-09-13T09:12:14.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1615000000000001e-05 2024-09-13T09:12:14.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:14.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.331208+0000", 2024-09-13T09:12:14.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6140000000000003e-06 2024-09-13T09:12:14.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:14.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:14.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:14.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.331277+0000", 2024-09-13T09:12:14.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9596000000000002e-05 2024-09-13T09:12:14.569 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.569 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:14.570 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.570 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:14.570 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:14.570 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:14.570 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:14.527+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:14.571 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:14.630 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:14.628+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:14.678 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:14.678 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:14.678 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:14.687 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:14.731 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:14.730+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:14.793 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:14.793 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:14.793 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:14.802 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:14.845 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:14.844+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:14.910 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:1 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'1, mlcod=0'0)", 2024-09-13T09:12:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.218895+0000", 2024-09-13T09:12:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.689061325000001, 2024-09-13T09:12:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0043868170000000003, 2024-09-13T09:12:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.218895+0000", 2024-09-13T09:12:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.218895+0000", 2024-09-13T09:12:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.218894+0000", 2024-09-13T09:12:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.218920+0000", 2024-09-13T09:12:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6044e-05 2024-09-13T09:12:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.218955+0000", 2024-09-13T09:12:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4555999999999999e-05 2024-09-13T09:12:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.218976+0000", 2024-09-13T09:12:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1103000000000001e-05 2024-09-13T09:12:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.219081+0000", 2024-09-13T09:12:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105015 2024-09-13T09:12:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.219174+0000", 2024-09-13T09:12:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2917999999999994e-05 2024-09-13T09:12:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.223249+0000", 2024-09-13T09:12:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0040751870000000001 2024-09-13T09:12:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.223281+0000", 2024-09-13T09:12:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2691000000000003e-05 2024-09-13T09:12:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:55 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'33, mlcod=14'29)", 2024-09-13T09:12:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.276361+0000", 2024-09-13T09:12:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.631594807999999, 2024-09-13T09:12:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0036939859999999998, 2024-09-13T09:12:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276361+0000", 2024-09-13T09:12:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276361+0000", 2024-09-13T09:12:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276360+0000", 2024-09-13T09:12:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276408+0000", 2024-09-13T09:12:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7936999999999999e-05 2024-09-13T09:12:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276426+0000", 2024-09-13T09:12:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7611000000000001e-05 2024-09-13T09:12:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276449+0000", 2024-09-13T09:12:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3363999999999999e-05 2024-09-13T09:12:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276543+0000", 2024-09-13T09:12:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3764999999999994e-05 2024-09-13T09:12:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276570+0000", 2024-09-13T09:12:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7430999999999999e-05 2024-09-13T09:12:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.280018+0000", 2024-09-13T09:12:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034480050000000001 2024-09-13T09:12:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.280055+0000", 2024-09-13T09:12:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6931999999999998e-05 2024-09-13T09:12:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:56 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'34, mlcod=14'29)", 2024-09-13T09:12:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.276889+0000", 2024-09-13T09:12:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.631067279, 2024-09-13T09:12:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0060300759999999997, 2024-09-13T09:12:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276889+0000", 2024-09-13T09:12:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276889+0000", 2024-09-13T09:12:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276888+0000", 2024-09-13T09:12:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276895+0000", 2024-09-13T09:12:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:12:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276908+0000", 2024-09-13T09:12:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3074999999999999e-05 2024-09-13T09:12:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276914+0000", 2024-09-13T09:12:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4739999999999996e-06 2024-09-13T09:12:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.277225+0000", 2024-09-13T09:12:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00031188100000000002 2024-09-13T09:12:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.277242+0000", 2024-09-13T09:12:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7079e-05 2024-09-13T09:12:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.282885+0000", 2024-09-13T09:12:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0056424370000000001 2024-09-13T09:12:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.282919+0000", 2024-09-13T09:12:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3807999999999999e-05 2024-09-13T09:12:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.924 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:61 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'39, mlcod=14'37)", 2024-09-13T09:12:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.291730+0000", 2024-09-13T09:12:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.616225492, 2024-09-13T09:12:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019352029999999999, 2024-09-13T09:12:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.291730+0000", 2024-09-13T09:12:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.291730+0000", 2024-09-13T09:12:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.291730+0000", 2024-09-13T09:12:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.291740+0000", 2024-09-13T09:12:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0482999999999999e-05 2024-09-13T09:12:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.291752+0000", 2024-09-13T09:12:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1816e-05 2024-09-13T09:12:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.291768+0000", 2024-09-13T09:12:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6098000000000001e-05 2024-09-13T09:12:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.291840+0000", 2024-09-13T09:12:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1530000000000001e-05 2024-09-13T09:12:14.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.291856+0000", 2024-09-13T09:12:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6835999999999999e-05 2024-09-13T09:12:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.293625+0000", 2024-09-13T09:12:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0017685979999999999 2024-09-13T09:12:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.293666+0000", 2024-09-13T09:12:14.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0567e-05 2024-09-13T09:12:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:62 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'40, mlcod=14'38)", 2024-09-13T09:12:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.294329+0000", 2024-09-13T09:12:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.613627016000001, 2024-09-13T09:12:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00203922, 2024-09-13T09:12:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.294329+0000", 2024-09-13T09:12:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.294329+0000", 2024-09-13T09:12:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.294328+0000", 2024-09-13T09:12:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.294334+0000", 2024-09-13T09:12:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0979999999999999e-06 2024-09-13T09:12:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.294346+0000", 2024-09-13T09:12:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1677e-05 2024-09-13T09:12:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.294351+0000", 2024-09-13T09:12:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0930000000000002e-06 2024-09-13T09:12:14.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.294394+0000", 2024-09-13T09:12:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3010000000000003e-05 2024-09-13T09:12:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.294410+0000", 2024-09-13T09:12:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6384000000000001e-05 2024-09-13T09:12:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.296333+0000", 2024-09-13T09:12:14.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019224660000000001 2024-09-13T09:12:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.296368+0000", 2024-09-13T09:12:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5269999999999999e-05 2024-09-13T09:12:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:68 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'42, mlcod=14'39)", 2024-09-13T09:12:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.298866+0000", 2024-09-13T09:12:14.933 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.609089954, 2024-09-13T09:12:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028304459999999999, 2024-09-13T09:12:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.298866+0000", 2024-09-13T09:12:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.298866+0000", 2024-09-13T09:12:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.298865+0000", 2024-09-13T09:12:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.298872+0000", 2024-09-13T09:12:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1389999999999998e-06 2024-09-13T09:12:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.298887+0000", 2024-09-13T09:12:14.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5101e-05 2024-09-13T09:12:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.298894+0000", 2024-09-13T09:12:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7889999999999997e-06 2024-09-13T09:12:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.298950+0000", 2024-09-13T09:12:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5739000000000003e-05 2024-09-13T09:12:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.298966+0000", 2024-09-13T09:12:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6120000000000002e-05 2024-09-13T09:12:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.301654+0000", 2024-09-13T09:12:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026881639999999998 2024-09-13T09:12:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.301696+0000", 2024-09-13T09:12:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2215000000000002e-05 2024-09-13T09:12:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.937 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:69 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'43, mlcod=14'39)", 2024-09-13T09:12:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.299371+0000", 2024-09-13T09:12:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.608584909999999, 2024-09-13T09:12:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049236979999999998, 2024-09-13T09:12:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299371+0000", 2024-09-13T09:12:14.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299371+0000", 2024-09-13T09:12:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299370+0000", 2024-09-13T09:12:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299424+0000", 2024-09-13T09:12:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3879999999999999e-05 2024-09-13T09:12:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299436+0000", 2024-09-13T09:12:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2150000000000001e-05 2024-09-13T09:12:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299441+0000", 2024-09-13T09:12:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1789999999999999e-06 2024-09-13T09:12:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299481+0000", 2024-09-13T09:12:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9249999999999999e-05 2024-09-13T09:12:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299500+0000", 2024-09-13T09:12:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9188999999999999e-05 2024-09-13T09:12:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.304259+0000", 2024-09-13T09:12:14.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047587269999999999 2024-09-13T09:12:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.304295+0000", 2024-09-13T09:12:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6137000000000003e-05 2024-09-13T09:12:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:70 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'44, mlcod=14'39)", 2024-09-13T09:12:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.300075+0000", 2024-09-13T09:12:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.607880499, 2024-09-13T09:12:14.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042356549999999996, 2024-09-13T09:12:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300075+0000", 2024-09-13T09:12:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300075+0000", 2024-09-13T09:12:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300075+0000", 2024-09-13T09:12:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300080+0000", 2024-09-13T09:12:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7010000000000004e-06 2024-09-13T09:12:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300088+0000", 2024-09-13T09:12:14.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0160000000000008e-06 2024-09-13T09:12:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300091+0000", 2024-09-13T09:12:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8489999999999999e-06 2024-09-13T09:12:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300127+0000", 2024-09-13T09:12:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5212000000000001e-05 2024-09-13T09:12:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300141+0000", 2024-09-13T09:12:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4048e-05 2024-09-13T09:12:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.304297+0000", 2024-09-13T09:12:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004156234 2024-09-13T09:12:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.304311+0000", 2024-09-13T09:12:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4126e-05 2024-09-13T09:12:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.946 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:119 1.0 e15/13 1:30306672:devicehealth::main.db.0000000000000000:head v 15'81, mlcod=15'78)", 2024-09-13T09:12:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.481101+0000", 2024-09-13T09:12:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.426854553, 2024-09-13T09:12:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027645769999999998, 2024-09-13T09:12:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481101+0000", 2024-09-13T09:12:14.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481101+0000", 2024-09-13T09:12:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481101+0000", 2024-09-13T09:12:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481163+0000", 2024-09-13T09:12:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1878999999999995e-05 2024-09-13T09:12:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481170+0000", 2024-09-13T09:12:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1679999999999998e-06 2024-09-13T09:12:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481173+0000", 2024-09-13T09:12:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3519999999999998e-06 2024-09-13T09:12:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481207+0000", 2024-09-13T09:12:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4155000000000001e-05 2024-09-13T09:12:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481220+0000", 2024-09-13T09:12:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2860000000000001e-05 2024-09-13T09:12:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.483841+0000", 2024-09-13T09:12:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026206210000000001 2024-09-13T09:12:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.483866+0000", 2024-09-13T09:12:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5029999999999999e-05 2024-09-13T09:12:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:120 1.0 e15/13 1:30306672:devicehealth::main.db.0000000000000000:head v 15'82, mlcod=15'78)", 2024-09-13T09:12:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.481511+0000", 2024-09-13T09:12:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.426444591999999, 2024-09-13T09:12:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026571950000000002, 2024-09-13T09:12:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481511+0000", 2024-09-13T09:12:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481511+0000", 2024-09-13T09:12:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481510+0000", 2024-09-13T09:12:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481567+0000", 2024-09-13T09:12:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6177000000000001e-05 2024-09-13T09:12:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481577+0000", 2024-09-13T09:12:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0752e-05 2024-09-13T09:12:14.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481591+0000", 2024-09-13T09:12:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3219e-05 2024-09-13T09:12:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481616+0000", 2024-09-13T09:12:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5282999999999998e-05 2024-09-13T09:12:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481627+0000", 2024-09-13T09:12:14.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1653e-05 2024-09-13T09:12:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484152+0000", 2024-09-13T09:12:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025244180000000001 2024-09-13T09:12:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484169+0000", 2024-09-13T09:12:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6639000000000001e-05 2024-09-13T09:12:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.955 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:121 1.0 e15/13 1:30306672:devicehealth::main.db.0000000000000000:head v 15'83, mlcod=15'78)", 2024-09-13T09:12:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.481882+0000", 2024-09-13T09:12:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.426074105, 2024-09-13T09:12:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00230926, 2024-09-13T09:12:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481882+0000", 2024-09-13T09:12:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481882+0000", 2024-09-13T09:12:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481881+0000", 2024-09-13T09:12:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481937+0000", 2024-09-13T09:12:14.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6131000000000002e-05 2024-09-13T09:12:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481945+0000", 2024-09-13T09:12:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4209999999999998e-06 2024-09-13T09:12:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481948+0000", 2024-09-13T09:12:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4599999999999999e-06 2024-09-13T09:12:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481971+0000", 2024-09-13T09:12:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.251e-05 2024-09-13T09:12:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.481983+0000", 2024-09-13T09:12:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1877e-05 2024-09-13T09:12:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484170+0000", 2024-09-13T09:12:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002187732 2024-09-13T09:12:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484191+0000", 2024-09-13T09:12:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0608999999999999e-05 2024-09-13T09:12:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:122 1.0 e15/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'84, mlcod=15'82)", 2024-09-13T09:12:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.484733+0000", 2024-09-13T09:12:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.423223429, 2024-09-13T09:12:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042987140000000004, 2024-09-13T09:12:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.960 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484733+0000", 2024-09-13T09:12:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484733+0000", 2024-09-13T09:12:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484732+0000", 2024-09-13T09:12:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484736+0000", 2024-09-13T09:12:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0620000000000002e-06 2024-09-13T09:12:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484744+0000", 2024-09-13T09:12:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1859999999999992e-06 2024-09-13T09:12:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484751+0000", 2024-09-13T09:12:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.985e-06 2024-09-13T09:12:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484778+0000", 2024-09-13T09:12:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.69e-05 2024-09-13T09:12:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484792+0000", 2024-09-13T09:12:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3329000000000001e-05 2024-09-13T09:12:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.489000+0000", 2024-09-13T09:12:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042089040000000003 2024-09-13T09:12:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.489031+0000", 2024-09-13T09:12:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0806000000000003e-05 2024-09-13T09:12:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.964 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:144 1.0 e15/13 1:30306672:devicehealth::main.db.0000000000000000:head v 15'102, mlcod=15'98)", 2024-09-13T09:12:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.860483+0000", 2024-09-13T09:12:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.047473067, 2024-09-13T09:12:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029436359999999999, 2024-09-13T09:12:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.860483+0000", 2024-09-13T09:12:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.860483+0000", 2024-09-13T09:12:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.860482+0000", 2024-09-13T09:12:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.860572+0000", 2024-09-13T09:12:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9153999999999994e-05 2024-09-13T09:12:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.860578+0000", 2024-09-13T09:12:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8680000000000001e-06 2024-09-13T09:12:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.860582+0000", 2024-09-13T09:12:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5449999999999999e-06 2024-09-13T09:12:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.860617+0000", 2024-09-13T09:12:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4717e-05 2024-09-13T09:12:14.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.860635+0000", 2024-09-13T09:12:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8702000000000002e-05 2024-09-13T09:12:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.863400+0000", 2024-09-13T09:12:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027650359999999998 2024-09-13T09:12:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.863427+0000", 2024-09-13T09:12:14.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6103000000000001e-05 2024-09-13T09:12:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:145 1.0 e15/13 1:30306672:devicehealth::main.db.0000000000000000:head v 15'103, mlcod=15'98)", 2024-09-13T09:12:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.860867+0000", 2024-09-13T09:12:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.047088433000001, 2024-09-13T09:12:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002802695, 2024-09-13T09:12:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.860867+0000", 2024-09-13T09:12:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.860867+0000", 2024-09-13T09:12:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.860867+0000", 2024-09-13T09:12:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.860935+0000", 2024-09-13T09:12:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7880999999999996e-05 2024-09-13T09:12:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.860943+0000", 2024-09-13T09:12:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4479999999999994e-06 2024-09-13T09:12:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.860949+0000", 2024-09-13T09:12:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:12:14.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.860987+0000", 2024-09-13T09:12:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7929000000000002e-05 2024-09-13T09:12:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.861000+0000", 2024-09-13T09:12:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3835000000000001e-05 2024-09-13T09:12:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.863648+0000", 2024-09-13T09:12:14.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026478090000000001 2024-09-13T09:12:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.863670+0000", 2024-09-13T09:12:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1888000000000001e-05 2024-09-13T09:12:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:146 1.0 e15/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'104, mlcod=15'102)", 2024-09-13T09:12:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.864136+0000", 2024-09-13T09:12:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.043819992, 2024-09-13T09:12:14.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023255580000000001, 2024-09-13T09:12:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.864136+0000", 2024-09-13T09:12:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.864136+0000", 2024-09-13T09:12:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.864135+0000", 2024-09-13T09:12:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.864140+0000", 2024-09-13T09:12:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:12:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.864150+0000", 2024-09-13T09:12:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0358e-05 2024-09-13T09:12:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.864155+0000", 2024-09-13T09:12:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4909999999999999e-06 2024-09-13T09:12:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.864190+0000", 2024-09-13T09:12:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4570999999999998e-05 2024-09-13T09:12:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.864203+0000", 2024-09-13T09:12:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3583e-05 2024-09-13T09:12:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.866434+0000", 2024-09-13T09:12:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002230962 2024-09-13T09:12:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.866461+0000", 2024-09-13T09:12:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7393e-05 2024-09-13T09:12:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:147 1.0 e15/13 1:30306672:devicehealth::main.db.0000000000000000:head v 15'105, mlcod=15'103)", 2024-09-13T09:12:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.867154+0000", 2024-09-13T09:12:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.040802344999999, 2024-09-13T09:12:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00206705, 2024-09-13T09:12:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.867154+0000", 2024-09-13T09:12:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.867154+0000", 2024-09-13T09:12:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.867153+0000", 2024-09-13T09:12:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.867158+0000", 2024-09-13T09:12:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2870000000000004e-06 2024-09-13T09:12:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.867169+0000", 2024-09-13T09:12:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0525e-05 2024-09-13T09:12:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.867173+0000", 2024-09-13T09:12:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5990000000000004e-06 2024-09-13T09:12:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.867209+0000", 2024-09-13T09:12:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5880000000000002e-05 2024-09-13T09:12:14.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.867223+0000", 2024-09-13T09:12:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3900000000000001e-05 2024-09-13T09:12:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.869192+0000", 2024-09-13T09:12:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019687760000000002 2024-09-13T09:12:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.869221+0000", 2024-09-13T09:12:14.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8793e-05 2024-09-13T09:12:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:156 1.0 e15/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'110, mlcod=15'105)", 2024-09-13T09:12:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:01.281545+0000", 2024-09-13T09:12:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.626411299000001, 2024-09-13T09:12:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002642265, 2024-09-13T09:12:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281545+0000", 2024-09-13T09:12:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281545+0000", 2024-09-13T09:12:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281544+0000", 2024-09-13T09:12:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281550+0000", 2024-09-13T09:12:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.891e-06 2024-09-13T09:12:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281557+0000", 2024-09-13T09:12:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3599999999999998e-06 2024-09-13T09:12:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281562+0000", 2024-09-13T09:12:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2370000000000003e-06 2024-09-13T09:12:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281584+0000", 2024-09-13T09:12:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2510999999999998e-05 2024-09-13T09:12:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281600+0000", 2024-09-13T09:12:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:12:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.284161+0000", 2024-09-13T09:12:14.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025613950000000002 2024-09-13T09:12:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.284187+0000", 2024-09-13T09:12:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5919e-05 2024-09-13T09:12:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:157 1.0 e15/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'111, mlcod=15'105)", 2024-09-13T09:12:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:01.281882+0000", 2024-09-13T09:12:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.62607375, 2024-09-13T09:12:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023298220000000001, 2024-09-13T09:12:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281882+0000", 2024-09-13T09:12:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281882+0000", 2024-09-13T09:12:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281882+0000", 2024-09-13T09:12:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281885+0000", 2024-09-13T09:12:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.703e-06 2024-09-13T09:12:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281893+0000", 2024-09-13T09:12:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2479999999999999e-06 2024-09-13T09:12:14.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281896+0000", 2024-09-13T09:12:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3110000000000001e-06 2024-09-13T09:12:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281921+0000", 2024-09-13T09:12:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5219e-05 2024-09-13T09:12:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.281934+0000", 2024-09-13T09:12:14.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2632999999999999e-05 2024-09-13T09:12:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.284190+0000", 2024-09-13T09:12:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022566349999999999 2024-09-13T09:12:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.284212+0000", 2024-09-13T09:12:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509e-05 2024-09-13T09:12:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:158 1.0 e15/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'112, mlcod=15'107)", 2024-09-13T09:12:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:01.282486+0000", 2024-09-13T09:12:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.625470191, 2024-09-13T09:12:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002170824, 2024-09-13T09:12:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.282486+0000", 2024-09-13T09:12:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.282486+0000", 2024-09-13T09:12:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.282485+0000", 2024-09-13T09:12:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.282555+0000", 2024-09-13T09:12:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9828999999999998e-05 2024-09-13T09:12:14.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.282565+0000", 2024-09-13T09:12:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0457e-05 2024-09-13T09:12:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.282572+0000", 2024-09-13T09:12:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5119999999999997e-06 2024-09-13T09:12:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.282604+0000", 2024-09-13T09:12:14.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2310000000000001e-05 2024-09-13T09:12:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.282616+0000", 2024-09-13T09:12:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2299e-05 2024-09-13T09:12:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.284630+0000", 2024-09-13T09:12:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002013328 2024-09-13T09:12:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.284657+0000", 2024-09-13T09:12:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6897999999999999e-05 2024-09-13T09:12:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:159 1.0 e15/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'113, mlcod=15'107)", 2024-09-13T09:12:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:01.282979+0000", 2024-09-13T09:12:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.62497726, 2024-09-13T09:12:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019634600000000002, 2024-09-13T09:12:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:14.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.282979+0000", 2024-09-13T09:12:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.282979+0000", 2024-09-13T09:12:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.282978+0000", 2024-09-13T09:12:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.282983+0000", 2024-09-13T09:12:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.391e-06 2024-09-13T09:12:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.282991+0000", 2024-09-13T09:12:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0080000000000002e-06 2024-09-13T09:12:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.282996+0000", 2024-09-13T09:12:14.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4519999999999996e-06 2024-09-13T09:12:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.284234+0000", 2024-09-13T09:12:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001237742 2024-09-13T09:12:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.284253+0000", 2024-09-13T09:12:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9490000000000001e-05 2024-09-13T09:12:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:14.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.284920+0000", 2024-09-13T09:12:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00066668999999999995 2024-09-13T09:12:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:01.284942+0000", 2024-09-13T09:12:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2184e-05 2024-09-13T09:12:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:14.999 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:14.999 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:15.001 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:15.001 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:15.000+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:15.102 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:15.101+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:15.148 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:15.148 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:15.149 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:15.157 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:15.203 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:15.202+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:15.265 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:15.265 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:15.265 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:15.273 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:15.317 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:15.316+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:15.382 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:12:15.385 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4221.0:5 2.2 e18/16 2:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-13T09:12:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:05.326180+0000", 2024-09-13T09:12:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.057763230999999, 2024-09-13T09:12:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00078215000000000001, 2024-09-13T09:12:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:05.326180+0000", 2024-09-13T09:12:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:05.326180+0000", 2024-09-13T09:12:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:05.326179+0000", 2024-09-13T09:12:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:12:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:05.326191+0000", 2024-09-13T09:12:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1323e-05 2024-09-13T09:12:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:05.326202+0000", 2024-09-13T09:12:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1722000000000001e-05 2024-09-13T09:12:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:05.326212+0000", 2024-09-13T09:12:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8560000000000002e-06 2024-09-13T09:12:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:05.326246+0000", 2024-09-13T09:12:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3207999999999998e-05 2024-09-13T09:12:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:05.326277+0000", 2024-09-13T09:12:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1924000000000001e-05 2024-09-13T09:12:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:05.326925+0000", 2024-09-13T09:12:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00064712600000000004 2024-09-13T09:12:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:05.326962+0000", 2024-09-13T09:12:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:12:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4221.0:7 2.2 e19/16 2:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-13T09:12:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:06.330214+0000", 2024-09-13T09:12:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.0537292219999994, 2024-09-13T09:12:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00084411999999999996, 2024-09-13T09:12:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:06.330214+0000", 2024-09-13T09:12:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:06.330214+0000", 2024-09-13T09:12:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:06.330213+0000", 2024-09-13T09:12:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:12:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:06.330222+0000", 2024-09-13T09:12:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1119999999999996e-06 2024-09-13T09:12:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:06.330233+0000", 2024-09-13T09:12:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1428e-05 2024-09-13T09:12:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:06.330244+0000", 2024-09-13T09:12:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1556000000000001e-05 2024-09-13T09:12:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:06.330300+0000", 2024-09-13T09:12:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5300999999999998e-05 2024-09-13T09:12:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:06.330326+0000", 2024-09-13T09:12:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6696999999999999e-05 2024-09-13T09:12:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:06.331026+0000", 2024-09-13T09:12:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00069923200000000004 2024-09-13T09:12:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:06.331058+0000", 2024-09-13T09:12:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2013e-05 2024-09-13T09:12:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:15.395 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:15.397 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:15.438 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:15.437+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:15.507 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:15.505+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:15.547 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:15.547 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:15.547 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:15.547 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:15.556 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:15.607 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:15.606+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:15.663 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:15.663 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:15.663 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:15.663 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:15.663 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:15.672 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:15.715 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:15.714+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:15.778 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:15.778 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:15.778 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:15.778 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:15.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.778 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4221.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-13T09:12:15.778 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:04.325570+0000", 2024-09-13T09:12:15.778 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.451965431, 2024-09-13T09:12:15.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00084999999999999995, 2024-09-13T09:12:15.779 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:15.779 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:15.779 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:15.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:15.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:04.325570+0000", 2024-09-13T09:12:15.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:15.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:15.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:04.325570+0000", 2024-09-13T09:12:15.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:15.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.780 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:15.780 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:04.325569+0000", 2024-09-13T09:12:15.780 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:15.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.780 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:15.780 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:04.325580+0000", 2024-09-13T09:12:15.780 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0711e-05 2024-09-13T09:12:15.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.781 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:15.781 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:04.325590+0000", 2024-09-13T09:12:15.781 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0463e-05 2024-09-13T09:12:15.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.781 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:15.781 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:04.325604+0000", 2024-09-13T09:12:15.781 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3304e-05 2024-09-13T09:12:15.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.781 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:15.781 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:04.325650+0000", 2024-09-13T09:12:15.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6576000000000003e-05 2024-09-13T09:12:15.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.782 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:15.782 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:04.325675+0000", 2024-09-13T09:12:15.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4854e-05 2024-09-13T09:12:15.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.782 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:15.782 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:04.326375+0000", 2024-09-13T09:12:15.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00069987599999999999 2024-09-13T09:12:15.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:15.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:15.782 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:15.782 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:04.326420+0000", 2024-09-13T09:12:15.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4595999999999998e-05 2024-09-13T09:12:15.783 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:15.783 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:15.783 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:15.783 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:15.783 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:15.783 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:15.786 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:15.832 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:15.830+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:15.902 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:15.902 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:15.902 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:15.902 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:15.911 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:15.912 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:15.910+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:16.012 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:16.011+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:16.056 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:16.056 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:16.056 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:16.056 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:16.056 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:16.065 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:16.113 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:16.112+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:16.168 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:16.168 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:16.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:16.168 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:16.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:11:59.289661+0000", 2024-09-13T09:12:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 16.875690122000002, 2024-09-13T09:12:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.93386499199999995, 2024-09-13T09:12:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1 2024-09-13T09:12:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289661+0000", 2024-09-13T09:12:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289661+0000", 2024-09-13T09:12:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289659+0000", 2024-09-13T09:12:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289669+0000", 2024-09-13T09:12:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:12:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289681+0000", 2024-09-13T09:12:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2316e-05 2024-09-13T09:12:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289693+0000", 2024-09-13T09:12:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2549e-05 2024-09-13T09:12:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289744+0000", 2024-09-13T09:12:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0365999999999999e-05 2024-09-13T09:12:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289752+0000", 2024-09-13T09:12:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5590000000000006e-06 2024-09-13T09:12:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.218301+0000", 2024-09-13T09:12:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.928549085 2024-09-13T09:12:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.218447+0000", 2024-09-13T09:12:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014537400000000001 2024-09-13T09:12:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.218615+0000", 2024-09-13T09:12:16.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000168195 2024-09-13T09:12:16.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.219277+0000", 2024-09-13T09:12:16.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066202600000000002 2024-09-13T09:12:16.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.223461+0000", 2024-09-13T09:12:16.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004184152 2024-09-13T09:12:16.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.223500+0000", 2024-09-13T09:12:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8733000000000003e-05 2024-09-13T09:12:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.223526+0000", 2024-09-13T09:12:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5612000000000001e-05 2024-09-13T09:12:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.917973616999999, 2024-09-13T09:12:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079358310000000008, 2024-09-13T09:12:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.180 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.180 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35 2024-09-13T09:12:16.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.180 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:16.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:16.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247410+0000", 2024-09-13T09:12:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3053999999999999e-05 2024-09-13T09:12:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247420+0000", 2024-09-13T09:12:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7189999999999996e-06 2024-09-13T09:12:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247429+0000", 2024-09-13T09:12:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5340000000000005e-06 2024-09-13T09:12:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.249500+0000", 2024-09-13T09:12:16.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020708990000000002 2024-09-13T09:12:16.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.249551+0000", 2024-09-13T09:12:16.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0762e-05 2024-09-13T09:12:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.249649+0000", 2024-09-13T09:12:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.768e-05 2024-09-13T09:12:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.251260+0000", 2024-09-13T09:12:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001611101 2024-09-13T09:12:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255178+0000", 2024-09-13T09:12:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039180930000000001 2024-09-13T09:12:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255256+0000", 2024-09-13T09:12:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8572000000000006e-05 2024-09-13T09:12:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255313+0000", 2024-09-13T09:12:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6625000000000001e-05 2024-09-13T09:12:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.188 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:16.188 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:16.188 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.917916439000001, 2024-09-13T09:12:16.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079428840000000007, 2024-09-13T09:12:16.188 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.188 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.188 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.188 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.189 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.192 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36 2024-09-13T09:12:16.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.192 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:16.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:16.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247438+0000", 2024-09-13T09:12:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7409999999999998e-06 2024-09-13T09:12:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247442+0000", 2024-09-13T09:12:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:12:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247444+0000", 2024-09-13T09:12:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8880000000000001e-06 2024-09-13T09:12:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250381+0000", 2024-09-13T09:12:16.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0029366800000000001 2024-09-13T09:12:16.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250446+0000", 2024-09-13T09:12:16.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5173999999999996e-05 2024-09-13T09:12:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250546+0000", 2024-09-13T09:12:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100142 2024-09-13T09:12:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.251514+0000", 2024-09-13T09:12:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00096772000000000004 2024-09-13T09:12:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255336+0000", 2024-09-13T09:12:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038214719999999998 2024-09-13T09:12:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255358+0000", 2024-09-13T09:12:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2141e-05 2024-09-13T09:12:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255377+0000", 2024-09-13T09:12:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9398999999999998e-05 2024-09-13T09:12:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.200 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:16.200 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:16.200 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.917901837, 2024-09-13T09:12:16.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079696799999999998, 2024-09-13T09:12:16.200 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.200 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.200 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.200 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37 2024-09-13T09:12:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247451+0000", 2024-09-13T09:12:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6000000000000001e-06 2024-09-13T09:12:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247463+0000", 2024-09-13T09:12:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1609e-05 2024-09-13T09:12:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247473+0000", 2024-09-13T09:12:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0536999999999999e-05 2024-09-13T09:12:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250802+0000", 2024-09-13T09:12:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033287270000000001 2024-09-13T09:12:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250853+0000", 2024-09-13T09:12:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1329000000000003e-05 2024-09-13T09:12:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250935+0000", 2024-09-13T09:12:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1396999999999993e-05 2024-09-13T09:12:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.251796+0000", 2024-09-13T09:12:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00086075099999999999 2024-09-13T09:12:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255382+0000", 2024-09-13T09:12:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003586334 2024-09-13T09:12:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255402+0000", 2024-09-13T09:12:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0392000000000001e-05 2024-09-13T09:12:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255419+0000", 2024-09-13T09:12:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6229e-05 2024-09-13T09:12:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.889967743, 2024-09-13T09:12:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077389640000000001, 2024-09-13T09:12:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 56 2024-09-13T09:12:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275385+0000", 2024-09-13T09:12:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:12:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275388+0000", 2024-09-13T09:12:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:12:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275396+0000", 2024-09-13T09:12:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0099999999999995e-06 2024-09-13T09:12:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.276666+0000", 2024-09-13T09:12:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00126968 2024-09-13T09:12:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.276710+0000", 2024-09-13T09:12:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4141999999999999e-05 2024-09-13T09:12:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.276785+0000", 2024-09-13T09:12:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4777000000000006e-05 2024-09-13T09:12:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.282050+0000", 2024-09-13T09:12:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0052647320000000003 2024-09-13T09:12:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283071+0000", 2024-09-13T09:12:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001021196 2024-09-13T09:12:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283102+0000", 2024-09-13T09:12:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0849999999999998e-05 2024-09-13T09:12:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283122+0000", 2024-09-13T09:12:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0190000000000002e-05 2024-09-13T09:12:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:58 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.881984228, 2024-09-13T09:12:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064599870000000004, 2024-09-13T09:12:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 58 2024-09-13T09:12:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283398+0000", 2024-09-13T09:12:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1646000000000002e-05 2024-09-13T09:12:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283403+0000", 2024-09-13T09:12:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0540000000000002e-06 2024-09-13T09:12:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283406+0000", 2024-09-13T09:12:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3110000000000001e-06 2024-09-13T09:12:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283841+0000", 2024-09-13T09:12:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000434549 2024-09-13T09:12:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283900+0000", 2024-09-13T09:12:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9153000000000001e-05 2024-09-13T09:12:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283975+0000", 2024-09-13T09:12:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4967000000000002e-05 2024-09-13T09:12:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.285844+0000", 2024-09-13T09:12:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001869435 2024-09-13T09:12:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289769+0000", 2024-09-13T09:12:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039245230000000001 2024-09-13T09:12:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289804+0000", 2024-09-13T09:12:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4999000000000002e-05 2024-09-13T09:12:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289826+0000", 2024-09-13T09:12:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2857000000000002e-05 2024-09-13T09:12:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:59 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.283413+0000", 2024-09-13T09:12:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.881937882999999, 2024-09-13T09:12:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064987589999999998, 2024-09-13T09:12:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 59 2024-09-13T09:12:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283413+0000", 2024-09-13T09:12:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283413+0000", 2024-09-13T09:12:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283412+0000", 2024-09-13T09:12:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283416+0000", 2024-09-13T09:12:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:12:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283420+0000", 2024-09-13T09:12:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6789999999999996e-06 2024-09-13T09:12:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283423+0000", 2024-09-13T09:12:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.672e-06 2024-09-13T09:12:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.284235+0000", 2024-09-13T09:12:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00081158900000000004 2024-09-13T09:12:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.284282+0000", 2024-09-13T09:12:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7213999999999999e-05 2024-09-13T09:12:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.284359+0000", 2024-09-13T09:12:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6814e-05 2024-09-13T09:12:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.285868+0000", 2024-09-13T09:12:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015097310000000001 2024-09-13T09:12:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289836+0000", 2024-09-13T09:12:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003967613 2024-09-13T09:12:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289877+0000", 2024-09-13T09:12:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1519e-05 2024-09-13T09:12:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289912+0000", 2024-09-13T09:12:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4088999999999998e-05 2024-09-13T09:12:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.298116+0000", 2024-09-13T09:12:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.867235079, 2024-09-13T09:12:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063667139999999999, 2024-09-13T09:12:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 69 2024-09-13T09:12:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298116+0000", 2024-09-13T09:12:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298116+0000", 2024-09-13T09:12:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298115+0000", 2024-09-13T09:12:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298193+0000", 2024-09-13T09:12:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7441999999999995e-05 2024-09-13T09:12:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298213+0000", 2024-09-13T09:12:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0829999999999999e-05 2024-09-13T09:12:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298224+0000", 2024-09-13T09:12:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0282000000000001e-05 2024-09-13T09:12:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299016+0000", 2024-09-13T09:12:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00079248999999999997 2024-09-13T09:12:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299098+0000", 2024-09-13T09:12:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1607999999999995e-05 2024-09-13T09:12:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299223+0000", 2024-09-13T09:12:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000124838 2024-09-13T09:12:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.300536+0000", 2024-09-13T09:12:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001312843 2024-09-13T09:12:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304428+0000", 2024-09-13T09:12:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038928000000000001 2024-09-13T09:12:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304463+0000", 2024-09-13T09:12:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4214999999999997e-05 2024-09-13T09:12:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304482+0000", 2024-09-13T09:12:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9800999999999999e-05 2024-09-13T09:12:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.298237+0000", 2024-09-13T09:12:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.867114110999999, 2024-09-13T09:12:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063589789999999998, 2024-09-13T09:12:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.245 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.245 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.245 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 70 2024-09-13T09:12:16.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.245 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298237+0000", 2024-09-13T09:12:16.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298237+0000", 2024-09-13T09:12:16.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298236+0000", 2024-09-13T09:12:16.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:16.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298242+0000", 2024-09-13T09:12:16.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5889999999999999e-06 2024-09-13T09:12:16.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298252+0000", 2024-09-13T09:12:16.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6539999999999994e-06 2024-09-13T09:12:16.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298257+0000", 2024-09-13T09:12:16.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4240000000000001e-06 2024-09-13T09:12:16.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299735+0000", 2024-09-13T09:12:16.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0014772120000000001 2024-09-13T09:12:16.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299851+0000", 2024-09-13T09:12:16.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000115998 2024-09-13T09:12:16.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299975+0000", 2024-09-13T09:12:16.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000124805 2024-09-13T09:12:16.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.300830+0000", 2024-09-13T09:12:16.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0008543 2024-09-13T09:12:16.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304519+0000", 2024-09-13T09:12:16.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036896849999999998 2024-09-13T09:12:16.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304560+0000", 2024-09-13T09:12:16.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1009e-05 2024-09-13T09:12:16.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304596+0000", 2024-09-13T09:12:16.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5114999999999998e-05 2024-09-13T09:12:16.252 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.252 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.252 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.253 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:85 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:16.253 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:16.253 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.853163711000001, 2024-09-13T09:12:16.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068319399999999999, 2024-09-13T09:12:16.253 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.253 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.253 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.253 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.253 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.253 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 85 2024-09-13T09:12:16.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.254 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:16.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:16.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:16.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:16.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312190+0000", 2024-09-13T09:12:16.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2389999999999998e-06 2024-09-13T09:12:16.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312198+0000", 2024-09-13T09:12:16.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8599999999999993e-06 2024-09-13T09:12:16.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312201+0000", 2024-09-13T09:12:16.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.399e-06 2024-09-13T09:12:16.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.313614+0000", 2024-09-13T09:12:16.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001413369 2024-09-13T09:12:16.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.313702+0000", 2024-09-13T09:12:16.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7556e-05 2024-09-13T09:12:16.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.313843+0000", 2024-09-13T09:12:16.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000141015 2024-09-13T09:12:16.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318968+0000", 2024-09-13T09:12:16.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0051251680000000003 2024-09-13T09:12:16.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318973+0000", 2024-09-13T09:12:16.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9520000000000002e-06 2024-09-13T09:12:16.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318999+0000", 2024-09-13T09:12:16.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5706000000000001e-05 2024-09-13T09:12:16.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319019+0000", 2024-09-13T09:12:16.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0024e-05 2024-09-13T09:12:16.258 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.258 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.258 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.259 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:16.259 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:16.259 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.853145691, 2024-09-13T09:12:16.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069390709999999998, 2024-09-13T09:12:16.259 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.259 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.259 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.259 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.259 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.259 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 86 2024-09-13T09:12:16.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.260 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:16.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:16.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:16.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:16.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312207+0000", 2024-09-13T09:12:16.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.243e-06 2024-09-13T09:12:16.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312211+0000", 2024-09-13T09:12:16.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1219999999999997e-06 2024-09-13T09:12:16.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312215+0000", 2024-09-13T09:12:16.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4680000000000001e-06 2024-09-13T09:12:16.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314378+0000", 2024-09-13T09:12:16.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021634309999999999 2024-09-13T09:12:16.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314426+0000", 2024-09-13T09:12:16.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7963000000000001e-05 2024-09-13T09:12:16.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314507+0000", 2024-09-13T09:12:16.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0619999999999997e-05 2024-09-13T09:12:16.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319024+0000", 2024-09-13T09:12:16.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045178140000000002 2024-09-13T09:12:16.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319065+0000", 2024-09-13T09:12:16.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0216999999999998e-05 2024-09-13T09:12:16.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319108+0000", 2024-09-13T09:12:16.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3618000000000001e-05 2024-09-13T09:12:16.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319144+0000", 2024-09-13T09:12:16.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5858999999999997e-05 2024-09-13T09:12:16.264 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.265 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.265 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:16.265 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:16.265 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.853131313, 2024-09-13T09:12:16.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011377922, 2024-09-13T09:12:16.265 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.265 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.265 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.265 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.265 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.266 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 87 2024-09-13T09:12:16.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.266 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:16.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:16.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:16.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:16.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312249+0000", 2024-09-13T09:12:16.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9377e-05 2024-09-13T09:12:16.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312255+0000", 2024-09-13T09:12:16.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0220000000000001e-06 2024-09-13T09:12:16.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312259+0000", 2024-09-13T09:12:16.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-13T09:12:16.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314845+0000", 2024-09-13T09:12:16.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0025858750000000001 2024-09-13T09:12:16.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314891+0000", 2024-09-13T09:12:16.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6347999999999997e-05 2024-09-13T09:12:16.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314968+0000", 2024-09-13T09:12:16.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7428999999999993e-05 2024-09-13T09:12:16.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319174+0000", 2024-09-13T09:12:16.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042056999999999997 2024-09-13T09:12:16.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323544+0000", 2024-09-13T09:12:16.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043701119999999998 2024-09-13T09:12:16.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323575+0000", 2024-09-13T09:12:16.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1285000000000003e-05 2024-09-13T09:12:16.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323597+0000", 2024-09-13T09:12:16.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1891000000000001e-05 2024-09-13T09:12:16.270 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.270 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.271 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.271 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:16.271 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:16.271 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.853088555999999, 2024-09-13T09:12:16.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011622512, 2024-09-13T09:12:16.271 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.271 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.271 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.271 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.271 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.271 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 88 2024-09-13T09:12:16.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.272 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:16.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:16.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:16.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:16.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312265+0000", 2024-09-13T09:12:16.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8380000000000002e-06 2024-09-13T09:12:16.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312268+0000", 2024-09-13T09:12:16.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2310000000000001e-06 2024-09-13T09:12:16.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312271+0000", 2024-09-13T09:12:16.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.551e-06 2024-09-13T09:12:16.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318458+0000", 2024-09-13T09:12:16.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061866710000000004 2024-09-13T09:12:16.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318540+0000", 2024-09-13T09:12:16.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1845e-05 2024-09-13T09:12:16.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318668+0000", 2024-09-13T09:12:16.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012830900000000001 2024-09-13T09:12:16.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319598+0000", 2024-09-13T09:12:16.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000929377 2024-09-13T09:12:16.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323840+0000", 2024-09-13T09:12:16.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042420180000000002 2024-09-13T09:12:16.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323866+0000", 2024-09-13T09:12:16.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6262999999999999e-05 2024-09-13T09:12:16.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323885+0000", 2024-09-13T09:12:16.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8675e-05 2024-09-13T09:12:16.276 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.277 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.277 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.277 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:16.277 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:16.277 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.692710562, 2024-09-13T09:12:16.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0062525000000000002, 2024-09-13T09:12:16.277 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.277 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.277 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.277 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.277 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.278 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113 2024-09-13T09:12:16.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.278 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:16.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:16.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:16.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:16.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472644+0000", 2024-09-13T09:12:16.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-13T09:12:16.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472650+0000", 2024-09-13T09:12:16.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:12:16.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472655+0000", 2024-09-13T09:12:16.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:12:16.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477258+0000", 2024-09-13T09:12:16.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004603762 2024-09-13T09:12:16.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477303+0000", 2024-09-13T09:12:16.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5031999999999998e-05 2024-09-13T09:12:16.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477375+0000", 2024-09-13T09:12:16.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1186000000000006e-05 2024-09-13T09:12:16.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478772+0000", 2024-09-13T09:12:16.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00139704 2024-09-13T09:12:16.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478817+0000", 2024-09-13T09:12:16.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5120999999999997e-05 2024-09-13T09:12:16.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478859+0000", 2024-09-13T09:12:16.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2567000000000001e-05 2024-09-13T09:12:16.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478893+0000", 2024-09-13T09:12:16.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3210000000000002e-05 2024-09-13T09:12:16.282 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.282 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.282 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.692687253999999, 2024-09-13T09:12:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063268899999999999, 2024-09-13T09:12:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 114 2024-09-13T09:12:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:16.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:16.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:16.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:16.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472667+0000", 2024-09-13T09:12:16.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9219999999999999e-06 2024-09-13T09:12:16.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472673+0000", 2024-09-13T09:12:16.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.63e-06 2024-09-13T09:12:16.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472678+0000", 2024-09-13T09:12:16.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6689999999999997e-06 2024-09-13T09:12:16.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477608+0000", 2024-09-13T09:12:16.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0049302870000000002 2024-09-13T09:12:16.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477685+0000", 2024-09-13T09:12:16.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6922000000000007e-05 2024-09-13T09:12:16.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477804+0000", 2024-09-13T09:12:16.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011865499999999999 2024-09-13T09:12:16.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478935+0000", 2024-09-13T09:12:16.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0011305099999999999 2024-09-13T09:12:16.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478942+0000", 2024-09-13T09:12:16.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3969999999999997e-06 2024-09-13T09:12:16.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478970+0000", 2024-09-13T09:12:16.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8447000000000001e-05 2024-09-13T09:12:16.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478990+0000", 2024-09-13T09:12:16.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9933e-05 2024-09-13T09:12:16.288 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.288 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.288 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.289 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:16.289 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472870+0000", 2024-09-13T09:12:16.289 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.692480837, 2024-09-13T09:12:16.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068457680000000003, 2024-09-13T09:12:16.289 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.289 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.289 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.289 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.289 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.289 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 115 2024-09-13T09:12:16.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.289 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472870+0000", 2024-09-13T09:12:16.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472870+0000", 2024-09-13T09:12:16.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472869+0000", 2024-09-13T09:12:16.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:16.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472933+0000", 2024-09-13T09:12:16.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3448000000000006e-05 2024-09-13T09:12:16.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472943+0000", 2024-09-13T09:12:16.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0234999999999999e-05 2024-09-13T09:12:16.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472948+0000", 2024-09-13T09:12:16.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:12:16.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478311+0000", 2024-09-13T09:12:16.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053631759999999999 2024-09-13T09:12:16.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478364+0000", 2024-09-13T09:12:16.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3174000000000003e-05 2024-09-13T09:12:16.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478441+0000", 2024-09-13T09:12:16.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6211000000000006e-05 2024-09-13T09:12:16.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479649+0000", 2024-09-13T09:12:16.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001208319 2024-09-13T09:12:16.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479673+0000", 2024-09-13T09:12:16.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4060999999999999e-05 2024-09-13T09:12:16.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479697+0000", 2024-09-13T09:12:16.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3958999999999999e-05 2024-09-13T09:12:16.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479716+0000", 2024-09-13T09:12:16.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8661e-05 2024-09-13T09:12:16.294 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.294 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.295 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.295 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:16.295 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472955+0000", 2024-09-13T09:12:16.295 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.692395782, 2024-09-13T09:12:16.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069091580000000003, 2024-09-13T09:12:16.295 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.295 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.295 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.295 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.295 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.295 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 116 2024-09-13T09:12:16.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.296 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472955+0000", 2024-09-13T09:12:16.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472955+0000", 2024-09-13T09:12:16.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472954+0000", 2024-09-13T09:12:16.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:16.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472959+0000", 2024-09-13T09:12:16.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1309999999999999e-06 2024-09-13T09:12:16.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472964+0000", 2024-09-13T09:12:16.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6659999999999996e-06 2024-09-13T09:12:16.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472968+0000", 2024-09-13T09:12:16.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.495e-06 2024-09-13T09:12:16.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479021+0000", 2024-09-13T09:12:16.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060531990000000004 2024-09-13T09:12:16.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479062+0000", 2024-09-13T09:12:16.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1436000000000001e-05 2024-09-13T09:12:16.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479131+0000", 2024-09-13T09:12:16.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8533000000000003e-05 2024-09-13T09:12:16.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479727+0000", 2024-09-13T09:12:16.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00059612 2024-09-13T09:12:16.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479827+0000", 2024-09-13T09:12:16.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9940999999999997e-05 2024-09-13T09:12:16.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479848+0000", 2024-09-13T09:12:16.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0877999999999999e-05 2024-09-13T09:12:16.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479864+0000", 2024-09-13T09:12:16.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6175999999999999e-05 2024-09-13T09:12:16.301 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.301 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.301 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.301 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:16.301 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472991+0000", 2024-09-13T09:12:16.301 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.692360175999999, 2024-09-13T09:12:16.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071846899999999997, 2024-09-13T09:12:16.302 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.302 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.302 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.302 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.302 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.302 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 117 2024-09-13T09:12:16.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.302 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472991+0000", 2024-09-13T09:12:16.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472991+0000", 2024-09-13T09:12:16.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472990+0000", 2024-09-13T09:12:16.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:16.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472994+0000", 2024-09-13T09:12:16.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4870000000000004e-06 2024-09-13T09:12:16.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.473001+0000", 2024-09-13T09:12:16.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:12:16.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.473005+0000", 2024-09-13T09:12:16.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-13T09:12:16.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479356+0000", 2024-09-13T09:12:16.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063509869999999998 2024-09-13T09:12:16.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479407+0000", 2024-09-13T09:12:16.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0349999999999997e-05 2024-09-13T09:12:16.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479488+0000", 2024-09-13T09:12:16.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0922999999999998e-05 2024-09-13T09:12:16.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479942+0000", 2024-09-13T09:12:16.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045411799999999999 2024-09-13T09:12:16.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.480136+0000", 2024-09-13T09:12:16.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000193909 2024-09-13T09:12:16.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.480158+0000", 2024-09-13T09:12:16.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2427999999999999e-05 2024-09-13T09:12:16.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.480175+0000", 2024-09-13T09:12:16.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7071e-05 2024-09-13T09:12:16.307 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.307 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.307 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.307 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:16.307 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:16.307 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.313012462, 2024-09-13T09:12:16.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066097600000000001, 2024-09-13T09:12:16.307 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.307 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.308 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.308 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.308 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.308 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 140 2024-09-13T09:12:16.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.308 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:16.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:16.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:16.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:16.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852387+0000", 2024-09-13T09:12:16.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9191999999999999e-05 2024-09-13T09:12:16.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852392+0000", 2024-09-13T09:12:16.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7299999999999996e-06 2024-09-13T09:12:16.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852395+0000", 2024-09-13T09:12:16.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9979999999999999e-06 2024-09-13T09:12:16.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857443+0000", 2024-09-13T09:12:16.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005048075 2024-09-13T09:12:16.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857503+0000", 2024-09-13T09:12:16.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9577999999999998e-05 2024-09-13T09:12:16.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857571+0000", 2024-09-13T09:12:16.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8320999999999999e-05 2024-09-13T09:12:16.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858400+0000", 2024-09-13T09:12:16.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00082945100000000004 2024-09-13T09:12:16.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858910+0000", 2024-09-13T09:12:16.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050916999999999998 2024-09-13T09:12:16.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858931+0000", 2024-09-13T09:12:16.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1742e-05 2024-09-13T09:12:16.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858948+0000", 2024-09-13T09:12:16.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6714999999999998e-05 2024-09-13T09:12:16.313 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.313 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.313 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.313 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:16.313 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.852399+0000", 2024-09-13T09:12:16.313 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.312952150999999, 2024-09-13T09:12:16.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068470969999999999, 2024-09-13T09:12:16.313 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.313 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.313 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.313 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:16.313 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:16.314 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 141 2024-09-13T09:12:16.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.314 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852399+0000", 2024-09-13T09:12:16.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852399+0000", 2024-09-13T09:12:16.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852398+0000", 2024-09-13T09:12:16.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:16.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852400+0000", 2024-09-13T09:12:16.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:12:16.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852404+0000", 2024-09-13T09:12:16.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:12:16.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852406+0000", 2024-09-13T09:12:16.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4559999999999999e-06 2024-09-13T09:12:16.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857928+0000", 2024-09-13T09:12:16.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005521593 2024-09-13T09:12:16.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857968+0000", 2024-09-13T09:12:16.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0586000000000001e-05 2024-09-13T09:12:16.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:16.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858029+0000", 2024-09-13T09:12:16.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1140000000000001e-05 2024-09-13T09:12:16.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859187+0000", 2024-09-13T09:12:16.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001157969 2024-09-13T09:12:16.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859198+0000", 2024-09-13T09:12:16.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1028e-05 2024-09-13T09:12:16.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859227+0000", 2024-09-13T09:12:16.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8370000000000001e-05 2024-09-13T09:12:16.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859246+0000", 2024-09-13T09:12:16.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8961e-05 2024-09-13T09:12:16.318 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.319 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.319 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.319 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.319 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.319 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:16.320 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:16.321 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:16.319+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:16.379 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-09-13T09:12:16.427 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:16.427 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:16.427 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:16.427 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:16.436 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:16.438 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:16.437+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:16.507 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:12:16.522 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:16.520+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:16.538 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:16.538 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:16.538 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:16.539 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:16.539 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:16.547 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:16.622 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:16.621+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:16.636 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:16.636 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:16.653 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:16.653 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:16.668 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:16.668 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:16.685 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:16.685 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:16.693 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:16.693 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:16.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:16.693 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:16.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.694 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:12:16.694 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:15.368781+0000", 2024-09-13T09:12:16.694 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.3234315030000001, 2024-09-13T09:12:16.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038167510000000002, 2024-09-13T09:12:16.694 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.694 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:12:16.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:12:16.694 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 7 2024-09-13T09:12:16.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.694 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368781+0000", 2024-09-13T09:12:16.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368781+0000", 2024-09-13T09:12:16.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368778+0000", 2024-09-13T09:12:16.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999971 2024-09-13T09:12:16.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368796+0000", 2024-09-13T09:12:16.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8148999999999999e-05 2024-09-13T09:12:16.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368828+0000", 2024-09-13T09:12:16.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2066000000000001e-05 2024-09-13T09:12:16.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368846+0000", 2024-09-13T09:12:16.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7759999999999999e-05 2024-09-13T09:12:16.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368958+0000", 2024-09-13T09:12:16.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112074 2024-09-13T09:12:16.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:16.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368969+0000", 2024-09-13T09:12:16.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.094e-05 2024-09-13T09:12:16.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.371161+0000", 2024-09-13T09:12:16.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021919869999999998 2024-09-13T09:12:16.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.371274+0000", 2024-09-13T09:12:16.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011297400000000001 2024-09-13T09:12:16.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:16.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.371386+0000", 2024-09-13T09:12:16.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112348 2024-09-13T09:12:16.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372324+0000", 2024-09-13T09:12:16.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00093778700000000004 2024-09-13T09:12:16.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372516+0000", 2024-09-13T09:12:16.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019215700000000001 2024-09-13T09:12:16.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372556+0000", 2024-09-13T09:12:16.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9919999999999997e-05 2024-09-13T09:12:16.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372598+0000", 2024-09-13T09:12:16.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1495000000000002e-05 2024-09-13T09:12:16.700 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.700 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.700 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.700 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop_reply(client.4387.0:7 3.1a e24/21 ondisk, result = 0)", 2024-09-13T09:12:16.700 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:15.372423+0000", 2024-09-13T09:12:16.701 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.3197888980000001, 2024-09-13T09:12:16.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018315000000000001, 2024-09-13T09:12:16.701 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.701 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:16.701 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372423+0000", 2024-09-13T09:12:16.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372423+0000", 2024-09-13T09:12:16.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372422+0000", 2024-09-13T09:12:16.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:16.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372430+0000", 2024-09-13T09:12:16.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8569999999999995e-06 2024-09-13T09:12:16.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372445+0000", 2024-09-13T09:12:16.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4779e-05 2024-09-13T09:12:16.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372470+0000", 2024-09-13T09:12:16.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5340000000000001e-05 2024-09-13T09:12:16.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372505+0000", 2024-09-13T09:12:16.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4578e-05 2024-09-13T09:12:16.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372513+0000", 2024-09-13T09:12:16.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9249999999999995e-06 2024-09-13T09:12:16.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372607+0000", 2024-09-13T09:12:16.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3886000000000002e-05 2024-09-13T09:12:16.704 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.704 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.705 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.705 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:12:16.705 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:16.378424+0000", 2024-09-13T09:12:16.705 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.31378863800000001, 2024-09-13T09:12:16.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061107690000000003, 2024-09-13T09:12:16.705 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.705 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.705 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.705 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:12:16.705 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:12:16.705 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9 2024-09-13T09:12:16.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.706 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378424+0000", 2024-09-13T09:12:16.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378424+0000", 2024-09-13T09:12:16.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378422+0000", 2024-09-13T09:12:16.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:16.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378429+0000", 2024-09-13T09:12:16.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6100000000000002e-06 2024-09-13T09:12:16.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378441+0000", 2024-09-13T09:12:16.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1581e-05 2024-09-13T09:12:16.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378449+0000", 2024-09-13T09:12:16.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2679999999999994e-06 2024-09-13T09:12:16.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378496+0000", 2024-09-13T09:12:16.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7286000000000001e-05 2024-09-13T09:12:16.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:16.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378501+0000", 2024-09-13T09:12:16.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0980000000000001e-06 2024-09-13T09:12:16.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.380081+0000", 2024-09-13T09:12:16.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015801879999999999 2024-09-13T09:12:16.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.380177+0000", 2024-09-13T09:12:16.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5311999999999998e-05 2024-09-13T09:12:16.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:16.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.380299+0000", 2024-09-13T09:12:16.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000121921 2024-09-13T09:12:16.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:16.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.381082+0000", 2024-09-13T09:12:16.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00078348100000000004 2024-09-13T09:12:16.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:16.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384464+0000", 2024-09-13T09:12:16.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033814349999999999 2024-09-13T09:12:16.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384495+0000", 2024-09-13T09:12:16.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1554999999999999e-05 2024-09-13T09:12:16.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384534+0000", 2024-09-13T09:12:16.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9320999999999998e-05 2024-09-13T09:12:16.711 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.711 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.711 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.712 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop_reply(client.4387.0:9 3.1a e25/21 ondisk, result = 0)", 2024-09-13T09:12:16.712 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:16.384386+0000", 2024-09-13T09:12:16.712 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.30782606499999998, 2024-09-13T09:12:16.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00015372300000000001, 2024-09-13T09:12:16.712 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.712 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:16.712 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384386+0000", 2024-09-13T09:12:16.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384386+0000", 2024-09-13T09:12:16.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384386+0000", 2024-09-13T09:12:16.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:16.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384391+0000", 2024-09-13T09:12:16.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.519e-06 2024-09-13T09:12:16.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384402+0000", 2024-09-13T09:12:16.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1277000000000001e-05 2024-09-13T09:12:16.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384409+0000", 2024-09-13T09:12:16.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1789999999999996e-06 2024-09-13T09:12:16.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384447+0000", 2024-09-13T09:12:16.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8071000000000002e-05 2024-09-13T09:12:16.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384459+0000", 2024-09-13T09:12:16.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2208000000000001e-05 2024-09-13T09:12:16.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384540+0000", 2024-09-13T09:12:16.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1207999999999999e-05 2024-09-13T09:12:16.715 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.716 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.716 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.716 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4387.0:10 3.1c e25/21 3:3831950c:::rbd_directory:head v 25'1, mlcod=0'0)", 2024-09-13T09:12:16.716 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:16.388735+0000", 2024-09-13T09:12:16.716 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.30347764500000002, 2024-09-13T09:12:16.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0029530210000000001, 2024-09-13T09:12:16.716 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.716 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.716 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388735+0000", 2024-09-13T09:12:16.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388735+0000", 2024-09-13T09:12:16.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388734+0000", 2024-09-13T09:12:16.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:16.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388740+0000", 2024-09-13T09:12:16.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2859999999999999e-06 2024-09-13T09:12:16.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388784+0000", 2024-09-13T09:12:16.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4119999999999998e-05 2024-09-13T09:12:16.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388792+0000", 2024-09-13T09:12:16.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2390000000000002e-06 2024-09-13T09:12:16.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388835+0000", 2024-09-13T09:12:16.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2744000000000003e-05 2024-09-13T09:12:16.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388852+0000", 2024-09-13T09:12:16.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7515000000000001e-05 2024-09-13T09:12:16.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.391659+0000", 2024-09-13T09:12:16.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0028064959999999999 2024-09-13T09:12:16.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.391688+0000", 2024-09-13T09:12:16.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000001e-05 2024-09-13T09:12:16.720 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.720 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.720 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.721 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4387.0:12 3.1c e25/21 3:3d5facff:::rbd_header.11238d3f93be:head v 25'2, mlcod=0'0)", 2024-09-13T09:12:16.721 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:16.396057+0000", 2024-09-13T09:12:16.721 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.29615508800000001, 2024-09-13T09:12:16.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00060416599999999995, 2024-09-13T09:12:16.721 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.721 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.721 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.396057+0000", 2024-09-13T09:12:16.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.396057+0000", 2024-09-13T09:12:16.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.396056+0000", 2024-09-13T09:12:16.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:16.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.396065+0000", 2024-09-13T09:12:16.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3899999999999999e-06 2024-09-13T09:12:16.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.396078+0000", 2024-09-13T09:12:16.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.241e-05 2024-09-13T09:12:16.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.396085+0000", 2024-09-13T09:12:16.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2150000000000004e-06 2024-09-13T09:12:16.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.396116+0000", 2024-09-13T09:12:16.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1436000000000002e-05 2024-09-13T09:12:16.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.396137+0000", 2024-09-13T09:12:16.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0460999999999999e-05 2024-09-13T09:12:16.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.396638+0000", 2024-09-13T09:12:16.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050109000000000004 2024-09-13T09:12:16.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.396661+0000", 2024-09-13T09:12:16.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3635999999999999e-05 2024-09-13T09:12:16.725 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.725 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.725 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.725 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:23 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:12:16.725 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:16.406214+0000", 2024-09-13T09:12:16.725 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.285998217, 2024-09-13T09:12:16.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000298786, 2024-09-13T09:12:16.726 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.726 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:16.726 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:16.726 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:12:16.726 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:12:16.726 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 23 2024-09-13T09:12:16.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.726 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.406214+0000", 2024-09-13T09:12:16.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.406214+0000", 2024-09-13T09:12:16.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.406213+0000", 2024-09-13T09:12:16.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:16.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.406220+0000", 2024-09-13T09:12:16.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1760000000000004e-06 2024-09-13T09:12:16.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.406232+0000", 2024-09-13T09:12:16.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1975e-05 2024-09-13T09:12:16.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.406238+0000", 2024-09-13T09:12:16.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2859999999999996e-06 2024-09-13T09:12:16.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.406294+0000", 2024-09-13T09:12:16.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5816999999999998e-05 2024-09-13T09:12:16.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.406383+0000", 2024-09-13T09:12:16.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9222999999999996e-05 2024-09-13T09:12:16.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.406513+0000", 2024-09-13T09:12:16.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012981000000000001 2024-09-13T09:12:16.729 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.730 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.730 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.730 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4387.0:25 3.1c e25/21 3:3d5facff:::rbd_header.11238d3f93be:head v 25'3, mlcod=25'1)", 2024-09-13T09:12:16.730 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:16.407692+0000", 2024-09-13T09:12:16.730 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.28451985299999999, 2024-09-13T09:12:16.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00044102300000000002, 2024-09-13T09:12:16.730 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:16.730 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:16.730 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:16.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:16.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.407692+0000", 2024-09-13T09:12:16.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:16.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.407692+0000", 2024-09-13T09:12:16.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:16.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:16.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.407692+0000", 2024-09-13T09:12:16.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:16.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:16.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.407699+0000", 2024-09-13T09:12:16.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1269999999999997e-06 2024-09-13T09:12:16.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:16.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.407709+0000", 2024-09-13T09:12:16.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0284e-05 2024-09-13T09:12:16.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:16.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.407715+0000", 2024-09-13T09:12:16.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:12:16.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:16.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.407751+0000", 2024-09-13T09:12:16.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5907e-05 2024-09-13T09:12:16.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:16.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.407765+0000", 2024-09-13T09:12:16.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4082e-05 2024-09-13T09:12:16.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:16.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.408100+0000", 2024-09-13T09:12:16.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00033529499999999999 2024-09-13T09:12:16.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:16.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:16.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:16.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.408134+0000", 2024-09-13T09:12:16.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3426000000000002e-05 2024-09-13T09:12:16.734 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.734 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.734 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.734 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:16.734 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:16.734 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:16.735 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:16.736 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:16.736 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:16.737 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:16.737 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:16.734+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:16.738 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:16.836 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:16.834+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:16.880 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:16.880 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:16.888 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:16.937 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:16.936+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:17.031 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:17.031 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:17.039 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:17.037+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:17.040 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:17.139 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:17.137+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:17.141 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:17.141 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:17.150 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:17.240 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:17.239+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:17.261 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:17.261 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:17.270 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:17.341 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:17.340+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:17.377 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:17.377 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:17.388 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:17.442 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:17.441+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:17.494 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:17.494 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:17.503 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:17.547 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:17.546+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:17.610 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:17.610 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:17.618 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:17.659 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:17.658+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:17.746 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:17.745+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:17.765 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:17.765 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:17.774 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:17.847 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:17.846+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:17.880 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:17.880 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:17.888 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:17.948 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:17.947+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:17.995 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:17.996 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:18.004 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:18.088 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:18.087+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:18.150 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:18.149+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:18.151 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:18.151 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:18.159 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:18.252 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:18.251+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:18.307 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:18.307 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:18.316 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:18.357 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:18.356+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:18.381 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-13T09:12:18.381 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:12:18.382 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-13T09:12:18.382 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-13T09:12:18.397 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:12:18.431 INFO:teuthology.orchestra.run.smithi157.stdout:3 write 0x3301400 thru 0x3305355 (0x3f56 bytes) 2024-09-13T09:12:18.443 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:18.443 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:18.451 INFO:teuthology.orchestra.run.smithi157.stdout:7 write 0x52dee00 thru 0x52e19ee (0x2bef bytes) 2024-09-13T09:12:18.455 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:18.457 INFO:teuthology.orchestra.run.smithi157.stdout:8 punch from 0x2fea00 to 0x303400, (0x4a00 bytes) 2024-09-13T09:12:18.457 INFO:teuthology.orchestra.run.smithi157.stdout:9 punch from 0x2d12c00 to 0x2d1fc00, (0xd000 bytes) 2024-09-13T09:12:18.457 INFO:teuthology.orchestra.run.smithi157.stdout:truncating to largest ever: 0x8e0c00 2024-09-13T09:12:18.462 INFO:teuthology.orchestra.run.smithi157.stdout:10 trunc from 0x52e1a00 to 0x8e0c00 2024-09-13T09:12:18.470 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:18.469+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:18.494 INFO:teuthology.orchestra.run.smithi157.stdout:13 write 0x165ca00 thru 0x16675da (0xabdb bytes) 2024-09-13T09:12:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:14 read 0x853400 thru 0x856e50 (0x3a51 bytes) 2024-09-13T09:12:18.495 INFO:teuthology.orchestra.run.smithi157.stdout:15 read 0x10ff400 thru 0x1106c77 (0x7878 bytes) 2024-09-13T09:12:18.511 INFO:teuthology.orchestra.run.smithi157.stdout:16 write 0x3e73c00 thru 0x3e805aa (0xc9ab bytes) 2024-09-13T09:12:18.511 INFO:teuthology.orchestra.run.smithi157.stdout:truncating to largest ever: 0xbf65e00 2024-09-13T09:12:18.521 INFO:teuthology.orchestra.run.smithi157.stdout:17 trunc from 0x3e80600 to 0xbf65e00 2024-09-13T09:12:18.527 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-13T09:12:18.533 INFO:teuthology.orchestra.run.smithi157.stdout:19 punch from 0x541ea00 to 0x542a800, (0xbe00 bytes) 2024-09-13T09:12:18.545 INFO:teuthology.orchestra.run.smithi157.stdout:21 trunc from 0xbf65e00 to 0x8062a00 2024-09-13T09:12:18.569 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:18.568+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:18.569 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:18.570 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:18.576 INFO:teuthology.orchestra.run.smithi157.stdout:24 write 0x8cee400 thru 0x8cf9d5e (0xb95f bytes) 2024-09-13T09:12:18.577 INFO:teuthology.orchestra.run.smithi157.stdout:26 write 0x51e8a00 thru 0x51ee6a0 (0x5ca1 bytes) 2024-09-13T09:12:18.577 INFO:teuthology.orchestra.run.smithi157.stdout:30 read 0x1471200 thru 0x1474421 (0x3222 bytes) 2024-09-13T09:12:18.578 INFO:teuthology.orchestra.run.smithi157.stdout:32 read 0x15f5200 thru 0x15f9ea3 (0x4ca4 bytes) 2024-09-13T09:12:18.578 INFO:teuthology.orchestra.run.smithi157.stdout:33 read 0x56a0000 thru 0x56ab46c (0xb46d bytes) 2024-09-13T09:12:18.579 INFO:teuthology.orchestra.run.smithi157.stdout:36 read 0x545de00 thru 0x546b461 (0xd662 bytes) 2024-09-13T09:12:18.579 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:18.602 INFO:teuthology.orchestra.run.smithi157.stdout:39 write 0xb81ca00 thru 0xb8264fa (0x9afb bytes) 2024-09-13T09:12:18.614 INFO:teuthology.orchestra.run.smithi157.stdout:40 trunc from 0xb826600 to 0x30d1000 2024-09-13T09:12:18.644 INFO:teuthology.orchestra.run.smithi157.stdout:42 read 0x1de1600 thru 0x1deb0ed (0x9aee bytes) 2024-09-13T09:12:18.645 INFO:teuthology.orchestra.run.smithi157.stdout:43 read 0x4ef200 thru 0x4fbd10 (0xcb11 bytes) 2024-09-13T09:12:18.655 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:18.654+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:18.672 INFO:teuthology.orchestra.run.smithi157.stdout:44 write 0xd051a00 thru 0xd05a4e2 (0x8ae3 bytes) 2024-09-13T09:12:18.672 INFO:teuthology.orchestra.run.smithi157.stdout:45 write 0x13a3200 thru 0x13b1f6f (0xed70 bytes) 2024-09-13T09:12:18.672 INFO:teuthology.orchestra.run.smithi157.stdout:47 read 0x75fe400 thru 0x7600bd6 (0x27d7 bytes) 2024-09-13T09:12:18.672 INFO:teuthology.orchestra.run.smithi157.stdout:50 punch from 0x2ddf000 to 0x2de6400, (0x7400 bytes) 2024-09-13T09:12:18.672 INFO:teuthology.orchestra.run.smithi157.stdout:55 read 0x7c10a00 thru 0x7c12537 (0x1b38 bytes) 2024-09-13T09:12:18.673 INFO:teuthology.orchestra.run.smithi157.stdout:57 read 0x5158000 thru 0x5167524 (0xf525 bytes) 2024-09-13T09:12:18.673 INFO:teuthology.orchestra.run.smithi157.stdout:59 read 0x59b1200 thru 0x59b6596 (0x5397 bytes) 2024-09-13T09:12:18.673 INFO:teuthology.orchestra.run.smithi157.stdout:truncating to largest ever: 0xe480a00 2024-09-13T09:12:18.673 INFO:teuthology.orchestra.run.smithi157.stdout:60 trunc from 0xd05a600 to 0xe480a00 2024-09-13T09:12:18.690 INFO:teuthology.orchestra.run.smithi157.stdout:62 punch from 0xa72a800 to 0xa732200, (0x7a00 bytes) 2024-09-13T09:12:18.691 INFO:teuthology.orchestra.run.smithi157.stdout:63 read 0x104ce00 thru 0x1055e8c (0x908d bytes) 2024-09-13T09:12:18.692 INFO:teuthology.orchestra.run.smithi157.stdout:64 read 0x85a0c00 thru 0x85af070 (0xe471 bytes) 2024-09-13T09:12:18.692 INFO:teuthology.orchestra.run.smithi157.stdout:65 write 0x5abe800 thru 0x5ace6a1 (0xfea2 bytes) 2024-09-13T09:12:18.692 INFO:teuthology.orchestra.run.smithi157.stdout:66 read 0x7721000 thru 0x7728ba6 (0x7ba7 bytes) 2024-09-13T09:12:18.692 INFO:teuthology.orchestra.run.smithi157.stdout:67 read 0xb4d5c00 thru 0xb4d9739 (0x3b3a bytes) 2024-09-13T09:12:18.692 INFO:teuthology.orchestra.run.smithi157.stdout:68 write 0x1fd9200 thru 0x1fdfcba (0x6abb bytes) 2024-09-13T09:12:18.692 INFO:teuthology.orchestra.run.smithi157.stdout:69 write 0x2567800 thru 0x256a26e (0x2a6f bytes) 2024-09-13T09:12:18.693 INFO:teuthology.orchestra.run.smithi157.stdout:70 write 0x966b200 thru 0x966cebd (0x1cbe bytes) 2024-09-13T09:12:18.693 INFO:teuthology.orchestra.run.smithi157.stdout:73 write 0xb613a00 thru 0xb621f86 (0xe587 bytes) 2024-09-13T09:12:18.693 INFO:teuthology.orchestra.run.smithi157.stdout:75 write 0xd214800 thru 0xd21b1c0 (0x69c1 bytes) 2024-09-13T09:12:18.693 INFO:teuthology.orchestra.run.smithi157.stdout:76 write 0x35c7200 thru 0x35d408f (0xce90 bytes) 2024-09-13T09:12:18.694 INFO:teuthology.orchestra.run.smithi157.stdout:77 punch from 0xad3fa00 to 0xad4b200, (0xb800 bytes) 2024-09-13T09:12:18.694 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:18.694 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:18.694 INFO:teuthology.orchestra.run.smithi157.stdout:78 read 0x956c000 thru 0x956f758 (0x3759 bytes) 2024-09-13T09:12:18.694 INFO:teuthology.orchestra.run.smithi157.stdout:80 write 0x9d86200 thru 0x9d938f4 (0xd6f5 bytes) 2024-09-13T09:12:18.705 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:18.706 INFO:teuthology.orchestra.run.smithi157.stdout:84 trunc from 0xe480a00 to 0x4059600 2024-09-13T09:12:18.717 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-13T09:12:18.732 INFO:teuthology.orchestra.run.smithi157.stdout:85 write 0x184dc00 thru 0x1850a96 (0x2e97 bytes) 2024-09-13T09:12:18.732 INFO:teuthology.orchestra.run.smithi157.stdout:86 read 0x33a7800 thru 0x33afeb5 (0x86b6 bytes) 2024-09-13T09:12:18.739 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:12:18.759 INFO:teuthology.orchestra.run.smithi157.stdout:87 write 0xb36fc00 thru 0xb37d80c (0xdc0d bytes) 2024-09-13T09:12:18.761 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:18.760+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:18.776 INFO:teuthology.orchestra.run.smithi157.stdout:89 write 0xb75ea00 thru 0xb76ba2a (0xd02b bytes) 2024-09-13T09:12:18.777 INFO:teuthology.orchestra.run.smithi157.stdout:91 punch from 0x198e000 to 0x1990400, (0x2400 bytes) 2024-09-13T09:12:18.777 INFO:teuthology.orchestra.run.smithi157.stdout:93 write 0x81be600 thru 0x81c8033 (0x9a34 bytes) 2024-09-13T09:12:18.777 INFO:teuthology.orchestra.run.smithi157.stdout:94 read 0xbbba00 thru 0xbc77e8 (0xbde9 bytes) 2024-09-13T09:12:18.777 INFO:teuthology.orchestra.run.smithi157.stdout:truncating to largest ever: 0xe510800 2024-09-13T09:12:18.780 INFO:teuthology.orchestra.run.smithi157.stdout:97 trunc from 0xb76bc00 to 0xe510800 2024-09-13T09:12:18.791 INFO:teuthology.orchestra.run.smithi157.stdout:98 read 0x2742000 thru 0x27492a6 (0x72a7 bytes) 2024-09-13T09:12:18.793 INFO:teuthology.orchestra.run.smithi157.stdout:100 write 0x418b400 thru 0x419689f (0xb4a0 bytes) 2024-09-13T09:12:18.793 INFO:teuthology.orchestra.run.smithi157.stdout:102 write 0x2b0e600 thru 0x2b1050b (0x1f0c bytes) 2024-09-13T09:12:18.796 INFO:teuthology.orchestra.run.smithi157.stdout:103 trunc from 0xe510800 to 0xc877600 2024-09-13T09:12:18.814 INFO:teuthology.orchestra.run.smithi157.stdout:104 punch from 0xb135200 to 0xb13aa00, (0x5800 bytes) 2024-09-13T09:12:18.814 INFO:teuthology.orchestra.run.smithi157.stdout:106 read 0xb233c00 thru 0xb23d792 (0x9b93 bytes) 2024-09-13T09:12:18.816 INFO:teuthology.orchestra.run.smithi157.stdout:107 write 0xb4c4400 thru 0xb4c9617 (0x5218 bytes) 2024-09-13T09:12:18.817 INFO:teuthology.orchestra.run.smithi157.stdout:109 punch from 0xc7e3a00 to 0xc7f1c00, (0xe200 bytes) 2024-09-13T09:12:18.817 INFO:teuthology.orchestra.run.smithi157.stdout:112 read 0x522c600 thru 0x52302d3 (0x3cd4 bytes) 2024-09-13T09:12:18.817 INFO:teuthology.orchestra.run.smithi157.stdout:113 read 0x96a4800 thru 0x96aa523 (0x5d24 bytes) 2024-09-13T09:12:18.817 INFO:teuthology.orchestra.run.smithi157.stdout:115 read 0x1eee600 thru 0x1eef2e8 (0xce9 bytes) 2024-09-13T09:12:18.818 INFO:teuthology.orchestra.run.smithi157.stdout:116 write 0x91d2200 thru 0x91dc5da (0xa3db bytes) 2024-09-13T09:12:18.818 INFO:teuthology.orchestra.run.smithi157.stdout:118 punch from 0xb03da00 to 0xb044800, (0x6e00 bytes) 2024-09-13T09:12:18.839 INFO:teuthology.orchestra.run.smithi157.stdout:119 write 0xd6d5600 thru 0xd6d6665 (0x1066 bytes) 2024-09-13T09:12:18.840 INFO:teuthology.orchestra.run.smithi157.stdout:120 write 0x7572600 thru 0x75782cb (0x5ccc bytes) 2024-09-13T09:12:18.840 INFO:teuthology.orchestra.run.smithi157.stdout:truncating to largest ever: 0xe563200 2024-09-13T09:12:18.840 INFO:teuthology.orchestra.run.smithi157.stdout:121 trunc from 0xd6d6800 to 0xe563200 2024-09-13T09:12:18.856 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:18.855+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:18.858 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:18.858 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:18.860 INFO:teuthology.orchestra.run.smithi157.stdout:122 read 0xe40f400 thru 0xe4106f8 (0x12f9 bytes) 2024-09-13T09:12:18.863 INFO:teuthology.orchestra.run.smithi157.stdout:123 trunc from 0xe563200 to 0xc619600 2024-09-13T09:12:18.866 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-13T09:12:18.868 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:18.885 INFO:teuthology.orchestra.run.smithi157.stdout:124 punch from 0x905a800 to 0x9068400, (0xdc00 bytes) 2024-09-13T09:12:18.885 INFO:teuthology.orchestra.run.smithi157.stdout:125 read 0x2bed200 thru 0x2beff3b (0x2d3c bytes) 2024-09-13T09:12:18.886 INFO:teuthology.orchestra.run.smithi157.stdout:127 read 0x5e2e200 thru 0x5e39e15 (0xbc16 bytes) 2024-09-13T09:12:18.886 INFO:teuthology.orchestra.run.smithi157.stdout:128 write 0x8027800 thru 0x80348b1 (0xd0b2 bytes) 2024-09-13T09:12:18.887 INFO:teuthology.orchestra.run.smithi157.stdout:129 write 0x5bd2400 thru 0x5bd3c93 (0x1894 bytes) 2024-09-13T09:12:18.887 INFO:teuthology.orchestra.run.smithi157.stdout:130 read 0x451c600 thru 0x4527ffc (0xb9fd bytes) 2024-09-13T09:12:18.887 INFO:teuthology.orchestra.run.smithi157.stdout:133 read 0xc0e600 thru 0xc19ed1 (0xb8d2 bytes) 2024-09-13T09:12:18.887 INFO:teuthology.orchestra.run.smithi157.stdout:134 read 0x5105e00 thru 0x51156d7 (0xf8d8 bytes) 2024-09-13T09:12:18.887 INFO:teuthology.orchestra.run.smithi157.stdout:139 punch from 0x984d800 to 0x9852e00, (0x5600 bytes) 2024-09-13T09:12:18.887 INFO:teuthology.orchestra.run.smithi157.stdout:140 read 0x9353200 thru 0x93603b8 (0xd1b9 bytes) 2024-09-13T09:12:18.888 INFO:teuthology.orchestra.run.smithi157.stdout:141 punch from 0xbe6a600 to 0xbe6b800, (0x1200 bytes) 2024-09-13T09:12:18.888 INFO:teuthology.orchestra.run.smithi157.stdout:142 punch from 0x80d5400 to 0x80d9400, (0x4000 bytes) 2024-09-13T09:12:18.888 INFO:teuthology.orchestra.run.smithi157.stdout:143 punch from 0x97d5000 to 0x97dd200, (0x8200 bytes) 2024-09-13T09:12:18.888 INFO:teuthology.orchestra.run.smithi157.stdout:144 punch from 0x1110e00 to 0x1113800, (0x2a00 bytes) 2024-09-13T09:12:18.888 INFO:teuthology.orchestra.run.smithi157.stdout:145 write 0x1ef7a00 thru 0x1f02902 (0xaf03 bytes) 2024-09-13T09:12:18.892 INFO:teuthology.orchestra.run.smithi157.stdout:147 trunc from 0xc619600 to 0xa0dc200 2024-09-13T09:12:18.897 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:18.898 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:18.917 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:18.917 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:18.930 INFO:teuthology.orchestra.run.smithi157.stdout:148 write 0xe208c00 thru 0xe208f22 (0x323 bytes) 2024-09-13T09:12:18.930 INFO:teuthology.orchestra.run.smithi157.stdout:149 read 0xdc8aa00 thru 0xdc93dcf (0x93d0 bytes) 2024-09-13T09:12:18.931 INFO:teuthology.orchestra.run.smithi157.stdout:150 read 0xcecbe00 thru 0xceccf0b (0x110c bytes) 2024-09-13T09:12:18.934 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:18.934 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:18.935 INFO:teuthology.orchestra.run.smithi157.stdout:151 trunc from 0xe209000 to 0x9eff800 2024-09-13T09:12:18.952 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:18.953 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:18.957 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:18.955+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:18.958 INFO:teuthology.orchestra.run.smithi157.stdout:152 write 0x40b0e00 thru 0x40c0ae7 (0xfce8 bytes) 2024-09-13T09:12:18.959 INFO:teuthology.orchestra.run.smithi157.stdout:153 write 0x60ff600 thru 0x6109bec (0xa5ed bytes) 2024-09-13T09:12:18.959 INFO:teuthology.orchestra.run.smithi157.stdout:156 punch from 0x585e600 to 0x5860800, (0x2200 bytes) 2024-09-13T09:12:18.959 INFO:teuthology.orchestra.run.smithi157.stdout:160 punch from 0x93d6c00 to 0x93d8400, (0x1800 bytes) 2024-09-13T09:12:18.959 INFO:teuthology.orchestra.run.smithi157.stdout:161 read 0x9b7fa00 thru 0x9b87a87 (0x8088 bytes) 2024-09-13T09:12:18.967 INFO:teuthology.orchestra.run.smithi157.stdout:164 trunc from 0x9eff800 to 0x3b66400 2024-09-13T09:12:18.971 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:18.971 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:18.984 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:18.984 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:18.984 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:18.984 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:18.984 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:18.988 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:18.989 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:18.992 INFO:teuthology.orchestra.run.smithi157.stdout:165 punch from 0x2b20800 to 0x2b26200, (0x5a00 bytes) 2024-09-13T09:12:18.993 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:19.008 INFO:teuthology.orchestra.run.smithi157.stdout:167 write 0x5f2c400 thru 0x5f2d9ae (0x15af bytes) 2024-09-13T09:12:19.014 INFO:teuthology.orchestra.run.smithi157.stdout:168 trunc from 0x5f2da00 to 0x5e4000 2024-09-13T09:12:19.048 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-13T09:12:19.076 INFO:teuthology.orchestra.run.smithi157.stdout:169 write 0xd9df600 thru 0xd9e8f07 (0x9908 bytes) 2024-09-13T09:12:19.077 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:19.076+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:19.096 INFO:teuthology.orchestra.run.smithi157.stdout:170 write 0xe7e7000 thru 0xe7f4814 (0xd815 bytes) 2024-09-13T09:12:19.096 INFO:teuthology.orchestra.run.smithi157.stdout:171 write 0xa4c8200 thru 0xa4d4108 (0xbf09 bytes) 2024-09-13T09:12:19.096 INFO:teuthology.orchestra.run.smithi157.stdout:172 write 0x2b67e00 thru 0x2b702ef (0x84f0 bytes) 2024-09-13T09:12:19.097 INFO:teuthology.orchestra.run.smithi157.stdout:177 read 0x7eaae00 thru 0x7eb4d8c (0x9f8d bytes) 2024-09-13T09:12:19.097 INFO:teuthology.orchestra.run.smithi157.stdout:truncating to largest ever: 0xed00c00 2024-09-13T09:12:19.097 INFO:teuthology.orchestra.run.smithi157.stdout:178 trunc from 0xe7f4a00 to 0xed00c00 2024-09-13T09:12:19.119 INFO:teuthology.orchestra.run.smithi157.stdout:186 read 0x42b6400 thru 0x42b8d0f (0x2910 bytes) 2024-09-13T09:12:19.120 INFO:teuthology.orchestra.run.smithi157.stdout:188 read 0x79eac00 thru 0x79f7dc3 (0xd1c4 bytes) 2024-09-13T09:12:19.120 INFO:teuthology.orchestra.run.smithi157.stdout:189 write 0xac14800 thru 0xac1b5c5 (0x6dc6 bytes) 2024-09-13T09:12:19.120 INFO:teuthology.orchestra.run.smithi157.stdout:190 read 0xe47a800 thru 0xe48609a (0xb89b bytes) 2024-09-13T09:12:19.121 INFO:teuthology.orchestra.run.smithi157.stdout:192 write 0x5a2d800 thru 0x5a3b3e1 (0xdbe2 bytes) 2024-09-13T09:12:19.121 INFO:teuthology.orchestra.run.smithi157.stdout:194 read 0x2557200 thru 0x2562588 (0xb389 bytes) 2024-09-13T09:12:19.121 INFO:teuthology.orchestra.run.smithi157.stdout:199 read 0xc1eec00 thru 0xc1fe59e (0xf99f bytes) 2024-09-13T09:12:19.121 INFO:teuthology.orchestra.run.smithi157.stdout:200 read 0x651b000 thru 0x6529093 (0xe094 bytes) 2024-09-13T09:12:19.121 INFO:teuthology.orchestra.run.smithi157.stdout:203 write 0x2829600 thru 0x28299fe (0x3ff bytes) 2024-09-13T09:12:19.122 INFO:teuthology.orchestra.run.smithi157.stdout:206 punch from 0xc183a00 to 0xc187600, (0x3c00 bytes) 2024-09-13T09:12:19.122 INFO:teuthology.orchestra.run.smithi157.stdout:207 write 0x5fe8e00 thru 0x5feb589 (0x278a bytes) 2024-09-13T09:12:19.122 INFO:teuthology.orchestra.run.smithi157.stdout:208 write 0x43a4a00 thru 0x43abf59 (0x755a bytes) 2024-09-13T09:12:19.122 INFO:teuthology.orchestra.run.smithi157.stdout:209 punch from 0xd766000 to 0xd76d800, (0x7800 bytes) 2024-09-13T09:12:19.123 INFO:teuthology.orchestra.run.smithi157.stdout:210 read 0x1714000 thru 0x17152c7 (0x12c8 bytes) 2024-09-13T09:12:19.123 INFO:teuthology.orchestra.run.smithi157.stdout:213 read 0x3fc400 thru 0x40b4a0 (0xf0a1 bytes) 2024-09-13T09:12:19.133 INFO:teuthology.orchestra.run.smithi157.stdout:217 trunc from 0xed00c00 to 0x6371000 2024-09-13T09:12:19.146 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:12:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:15.368781+0000", 2024-09-13T09:12:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.7748647339999999, 2024-09-13T09:12:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038167510000000002, 2024-09-13T09:12:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:12:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:12:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 7 2024-09-13T09:12:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368781+0000", 2024-09-13T09:12:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368781+0000", 2024-09-13T09:12:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368778+0000", 2024-09-13T09:12:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999971 2024-09-13T09:12:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368796+0000", 2024-09-13T09:12:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8148999999999999e-05 2024-09-13T09:12:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368828+0000", 2024-09-13T09:12:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2066000000000001e-05 2024-09-13T09:12:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368846+0000", 2024-09-13T09:12:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7759999999999999e-05 2024-09-13T09:12:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368958+0000", 2024-09-13T09:12:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112074 2024-09-13T09:12:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368969+0000", 2024-09-13T09:12:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.094e-05 2024-09-13T09:12:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.371161+0000", 2024-09-13T09:12:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021919869999999998 2024-09-13T09:12:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.371274+0000", 2024-09-13T09:12:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011297400000000001 2024-09-13T09:12:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.371386+0000", 2024-09-13T09:12:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112348 2024-09-13T09:12:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372324+0000", 2024-09-13T09:12:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00093778700000000004 2024-09-13T09:12:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372516+0000", 2024-09-13T09:12:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019215700000000001 2024-09-13T09:12:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372556+0000", 2024-09-13T09:12:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9919999999999997e-05 2024-09-13T09:12:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372598+0000", 2024-09-13T09:12:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1495000000000002e-05 2024-09-13T09:12:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:12:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:16.378424+0000", 2024-09-13T09:12:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.7652218689999999, 2024-09-13T09:12:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061107690000000003, 2024-09-13T09:12:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:12:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:12:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9 2024-09-13T09:12:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378424+0000", 2024-09-13T09:12:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378424+0000", 2024-09-13T09:12:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378422+0000", 2024-09-13T09:12:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378429+0000", 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6100000000000002e-06 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378441+0000", 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1581e-05 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378449+0000", 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2679999999999994e-06 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378496+0000", 2024-09-13T09:12:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7286000000000001e-05 2024-09-13T09:12:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378501+0000", 2024-09-13T09:12:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0980000000000001e-06 2024-09-13T09:12:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.380081+0000", 2024-09-13T09:12:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015801879999999999 2024-09-13T09:12:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.380177+0000", 2024-09-13T09:12:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5311999999999998e-05 2024-09-13T09:12:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.380299+0000", 2024-09-13T09:12:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000121921 2024-09-13T09:12:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.381082+0000", 2024-09-13T09:12:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00078348100000000004 2024-09-13T09:12:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384464+0000", 2024-09-13T09:12:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033814349999999999 2024-09-13T09:12:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384495+0000", 2024-09-13T09:12:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1554999999999999e-05 2024-09-13T09:12:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384534+0000", 2024-09-13T09:12:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9320999999999998e-05 2024-09-13T09:12:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4387.0:10 3.1c e25/21 3:3831950c:::rbd_directory:head v 25'1, mlcod=0'0)", 2024-09-13T09:12:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:16.388735+0000", 2024-09-13T09:12:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.7549108759999998, 2024-09-13T09:12:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0029530210000000001, 2024-09-13T09:12:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388735+0000", 2024-09-13T09:12:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388735+0000", 2024-09-13T09:12:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388734+0000", 2024-09-13T09:12:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388740+0000", 2024-09-13T09:12:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2859999999999999e-06 2024-09-13T09:12:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388784+0000", 2024-09-13T09:12:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4119999999999998e-05 2024-09-13T09:12:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388792+0000", 2024-09-13T09:12:19.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2390000000000002e-06 2024-09-13T09:12:19.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388835+0000", 2024-09-13T09:12:19.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2744000000000003e-05 2024-09-13T09:12:19.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388852+0000", 2024-09-13T09:12:19.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7515000000000001e-05 2024-09-13T09:12:19.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.391659+0000", 2024-09-13T09:12:19.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0028064959999999999 2024-09-13T09:12:19.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.391688+0000", 2024-09-13T09:12:19.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000001e-05 2024-09-13T09:12:19.167 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1190 3.a 3:562e7548:::rbd_data.11238d3f93be.0000000000000020:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:18.971634+0000", 2024-09-13T09:12:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.172011515, 2024-09-13T09:12:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036582519999999999, 2024-09-13T09:12:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1190 2024-09-13T09:12:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971634+0000", 2024-09-13T09:12:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971634+0000", 2024-09-13T09:12:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971633+0000", 2024-09-13T09:12:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971639+0000", 2024-09-13T09:12:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5280000000000003e-06 2024-09-13T09:12:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971653+0000", 2024-09-13T09:12:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4375999999999999e-05 2024-09-13T09:12:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971662+0000", 2024-09-13T09:12:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5739999999999996e-06 2024-09-13T09:12:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971740+0000", 2024-09-13T09:12:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7891000000000004e-05 2024-09-13T09:12:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971822+0000", 2024-09-13T09:12:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2488e-05 2024-09-13T09:12:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971922+0000", 2024-09-13T09:12:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9920999999999994e-05 2024-09-13T09:12:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.973264+0000", 2024-09-13T09:12:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0013414810000000001 2024-09-13T09:12:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.975239+0000", 2024-09-13T09:12:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0019751769999999998 2024-09-13T09:12:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.975272+0000", 2024-09-13T09:12:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2941000000000002e-05 2024-09-13T09:12:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.975292+0000", 2024-09-13T09:12:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0678000000000001e-05 2024-09-13T09:12:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1191 3.9 3:90ceac53:::rbd_data.11238d3f93be.0000000000000024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:18.971675+0000", 2024-09-13T09:12:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.171970236, 2024-09-13T09:12:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001738613, 2024-09-13T09:12:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1191 2024-09-13T09:12:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971675+0000", 2024-09-13T09:12:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971675+0000", 2024-09-13T09:12:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971675+0000", 2024-09-13T09:12:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971678+0000", 2024-09-13T09:12:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:12:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971687+0000", 2024-09-13T09:12:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7879999999999995e-06 2024-09-13T09:12:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971692+0000", 2024-09-13T09:12:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6749999999999999e-06 2024-09-13T09:12:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971742+0000", 2024-09-13T09:12:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9654000000000003e-05 2024-09-13T09:12:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971819+0000", 2024-09-13T09:12:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7198000000000002e-05 2024-09-13T09:12:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971918+0000", 2024-09-13T09:12:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8629999999999996e-05 2024-09-13T09:12:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972806+0000", 2024-09-13T09:12:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00088855200000000003 2024-09-13T09:12:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.973332+0000", 2024-09-13T09:12:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052601799999999995 2024-09-13T09:12:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.973377+0000", 2024-09-13T09:12:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4778999999999999e-05 2024-09-13T09:12:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.973414+0000", 2024-09-13T09:12:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6681000000000003e-05 2024-09-13T09:12:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4400.0:1188 3.e e26/21 3:76ba6fb0:::rbd_data.11238d3f93be.0000000000000018:head v 26'2, mlcod=0'0)", 2024-09-13T09:12:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:18.971950+0000", 2024-09-13T09:12:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.17169527200000001, 2024-09-13T09:12:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00089852400000000005, 2024-09-13T09:12:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971950+0000", 2024-09-13T09:12:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971950+0000", 2024-09-13T09:12:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971950+0000", 2024-09-13T09:12:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971967+0000", 2024-09-13T09:12:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7402e-05 2024-09-13T09:12:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971976+0000", 2024-09-13T09:12:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1600000000000004e-06 2024-09-13T09:12:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971982+0000", 2024-09-13T09:12:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9329999999999998e-06 2024-09-13T09:12:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972027+0000", 2024-09-13T09:12:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4614999999999999e-05 2024-09-13T09:12:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972051+0000", 2024-09-13T09:12:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3802e-05 2024-09-13T09:12:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972816+0000", 2024-09-13T09:12:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00076539100000000003 2024-09-13T09:12:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972849+0000", 2024-09-13T09:12:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2843999999999999e-05 2024-09-13T09:12:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4400.0:1187 3.1 e26/21 3:86c6e2eb:::rbd_data.11238d3f93be.0000000000000016:head v 26'5, mlcod=26'3)", 2024-09-13T09:12:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:18.971953+0000", 2024-09-13T09:12:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.171692395, 2024-09-13T09:12:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00092331399999999995, 2024-09-13T09:12:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971953+0000", 2024-09-13T09:12:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971953+0000", 2024-09-13T09:12:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971952+0000", 2024-09-13T09:12:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971960+0000", 2024-09-13T09:12:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9999999999999996e-06 2024-09-13T09:12:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971970+0000", 2024-09-13T09:12:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1579999999999994e-06 2024-09-13T09:12:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.971975+0000", 2024-09-13T09:12:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9100000000000002e-06 2024-09-13T09:12:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972011+0000", 2024-09-13T09:12:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5064999999999997e-05 2024-09-13T09:12:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972031+0000", 2024-09-13T09:12:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0582e-05 2024-09-13T09:12:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972839+0000", 2024-09-13T09:12:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00080810699999999997 2024-09-13T09:12:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972876+0000", 2024-09-13T09:12:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7258000000000001e-05 2024-09-13T09:12:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4400.0:1186 3.1d e26/21 3:bfcaca10:::rbd_data.11238d3f93be.0000000000000010:head v 26'5, mlcod=26'3)", 2024-09-13T09:12:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:18.972535+0000", 2024-09-13T09:12:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.17111084500000001, 2024-09-13T09:12:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0027368230000000002, 2024-09-13T09:12:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972535+0000", 2024-09-13T09:12:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972535+0000", 2024-09-13T09:12:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972534+0000", 2024-09-13T09:12:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972541+0000", 2024-09-13T09:12:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3760000000000002e-06 2024-09-13T09:12:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972554+0000", 2024-09-13T09:12:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.216e-05 2024-09-13T09:12:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972560+0000", 2024-09-13T09:12:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8059999999999997e-06 2024-09-13T09:12:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972610+0000", 2024-09-13T09:12:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0130999999999998e-05 2024-09-13T09:12:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.972630+0000", 2024-09-13T09:12:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9774000000000001e-05 2024-09-13T09:12:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.975245+0000", 2024-09-13T09:12:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026151709999999999 2024-09-13T09:12:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.975272+0000", 2024-09-13T09:12:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6148000000000001e-05 2024-09-13T09:12:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.194 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.194 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4400.0:1230 3.1c e26/21 3:3b8551ab:::rbd_data.11238d3f93be.000000000000000a:head v 26'65, mlcod=26'63)", 2024-09-13T09:12:19.194 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:18.991735+0000", 2024-09-13T09:12:19.194 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.15191037499999999, 2024-09-13T09:12:19.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00080675799999999998, 2024-09-13T09:12:19.194 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.194 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.194 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.991735+0000", 2024-09-13T09:12:19.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.991735+0000", 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.991734+0000", 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.991767+0000", 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3300000000000003e-05 2024-09-13T09:12:19.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.991781+0000", 2024-09-13T09:12:19.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3854e-05 2024-09-13T09:12:19.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.991788+0000", 2024-09-13T09:12:19.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9380000000000003e-06 2024-09-13T09:12:19.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.991828+0000", 2024-09-13T09:12:19.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9684999999999997e-05 2024-09-13T09:12:19.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.991845+0000", 2024-09-13T09:12:19.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7054999999999998e-05 2024-09-13T09:12:19.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.992513+0000", 2024-09-13T09:12:19.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066771599999999995 2024-09-13T09:12:19.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:18.992542+0000", 2024-09-13T09:12:19.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9300000000000001e-05 2024-09-13T09:12:19.198 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.198 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.198 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.198 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1275 3.9 3:91edc27b:::rbd_data.11238d3f93be.0000000000000017:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3325952~6656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.198 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.017320+0000", 2024-09-13T09:12:19.198 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.12632592600000001, 2024-09-13T09:12:19.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001249109, 2024-09-13T09:12:19.198 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.198 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.198 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.198 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.199 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.199 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1275 2024-09-13T09:12:19.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.199 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.017320+0000", 2024-09-13T09:12:19.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.017320+0000", 2024-09-13T09:12:19.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.017319+0000", 2024-09-13T09:12:19.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.017329+0000", 2024-09-13T09:12:19.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0193e-05 2024-09-13T09:12:19.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.017339+0000", 2024-09-13T09:12:19.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0412e-05 2024-09-13T09:12:19.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.017347+0000", 2024-09-13T09:12:19.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3440000000000003e-06 2024-09-13T09:12:19.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.017381+0000", 2024-09-13T09:12:19.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4134000000000002e-05 2024-09-13T09:12:19.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.017471+0000", 2024-09-13T09:12:19.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0284000000000005e-05 2024-09-13T09:12:19.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:19.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.017570+0000", 2024-09-13T09:12:19.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9192999999999997e-05 2024-09-13T09:12:19.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.018161+0000", 2024-09-13T09:12:19.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00059067099999999997 2024-09-13T09:12:19.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.018520+0000", 2024-09-13T09:12:19.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00035888800000000002 2024-09-13T09:12:19.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.018548+0000", 2024-09-13T09:12:19.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7956000000000001e-05 2024-09-13T09:12:19.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.018569+0000", 2024-09-13T09:12:19.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0845000000000001e-05 2024-09-13T09:12:19.204 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.204 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.204 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.205 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1279 3.1a 3:5f4b1450:::rbd_data.11238d3f93be.0000000000000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.205 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.022772+0000", 2024-09-13T09:12:19.205 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.120873644, 2024-09-13T09:12:19.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015680640000000001, 2024-09-13T09:12:19.205 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.205 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.205 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1279 2024-09-13T09:12:19.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.206 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.022772+0000", 2024-09-13T09:12:19.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.022772+0000", 2024-09-13T09:12:19.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.022769+0000", 2024-09-13T09:12:19.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:12:19.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.022803+0000", 2024-09-13T09:12:19.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3395999999999997e-05 2024-09-13T09:12:19.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.022841+0000", 2024-09-13T09:12:19.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8043000000000002e-05 2024-09-13T09:12:19.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.022861+0000", 2024-09-13T09:12:19.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9769e-05 2024-09-13T09:12:19.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.022987+0000", 2024-09-13T09:12:19.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012597800000000001 2024-09-13T09:12:19.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023155+0000", 2024-09-13T09:12:19.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016874199999999999 2024-09-13T09:12:19.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:19.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023280+0000", 2024-09-13T09:12:19.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000124585 2024-09-13T09:12:19.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.024239+0000", 2024-09-13T09:12:19.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00095892899999999997 2024-09-13T09:12:19.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.024280+0000", 2024-09-13T09:12:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.138e-05 2024-09-13T09:12:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.024316+0000", 2024-09-13T09:12:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6056999999999997e-05 2024-09-13T09:12:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.024340+0000", 2024-09-13T09:12:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3642000000000002e-05 2024-09-13T09:12:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1290 3.9 3:91edc27b:::rbd_data.11238d3f93be.0000000000000017:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.022903+0000", 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.12074259699999999, 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020085519999999998, 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1290 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.211 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.022903+0000", 2024-09-13T09:12:19.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.022903+0000", 2024-09-13T09:12:19.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.022902+0000", 2024-09-13T09:12:19.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:19.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.022910+0000", 2024-09-13T09:12:19.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7290000000000006e-06 2024-09-13T09:12:19.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.022934+0000", 2024-09-13T09:12:19.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3405999999999999e-05 2024-09-13T09:12:19.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.022964+0000", 2024-09-13T09:12:19.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9992999999999999e-05 2024-09-13T09:12:19.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023100+0000", 2024-09-13T09:12:19.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000136441 2024-09-13T09:12:19.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023213+0000", 2024-09-13T09:12:19.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011333899999999999 2024-09-13T09:12:19.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:19.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023332+0000", 2024-09-13T09:12:19.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000118106 2024-09-13T09:12:19.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.024217+0000", 2024-09-13T09:12:19.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00088557900000000005 2024-09-13T09:12:19.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.024855+0000", 2024-09-13T09:12:19.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00063754700000000003 2024-09-13T09:12:19.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.024891+0000", 2024-09-13T09:12:19.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5905000000000003e-05 2024-09-13T09:12:19.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.024912+0000", 2024-09-13T09:12:19.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0922999999999999e-05 2024-09-13T09:12:19.217 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.217 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.217 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.217 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4400.0:1300 3.1c e26/21 3:3d5facff:::rbd_header.11238d3f93be:head v 26'70, mlcod=26'68)", 2024-09-13T09:12:19.217 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.030655+0000", 2024-09-13T09:12:19.217 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.11299029100000001, 2024-09-13T09:12:19.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0016288019999999999, 2024-09-13T09:12:19.217 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.217 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.217 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.030655+0000", 2024-09-13T09:12:19.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.030655+0000", 2024-09-13T09:12:19.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.030655+0000", 2024-09-13T09:12:19.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:19.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.030660+0000", 2024-09-13T09:12:19.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2580000000000003e-06 2024-09-13T09:12:19.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.030669+0000", 2024-09-13T09:12:19.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4399999999999994e-06 2024-09-13T09:12:19.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.030675+0000", 2024-09-13T09:12:19.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5310000000000001e-06 2024-09-13T09:12:19.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.030721+0000", 2024-09-13T09:12:19.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5658000000000002e-05 2024-09-13T09:12:19.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.030733+0000", 2024-09-13T09:12:19.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.217e-05 2024-09-13T09:12:19.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.032250+0000", 2024-09-13T09:12:19.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015176059999999999 2024-09-13T09:12:19.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.032284+0000", 2024-09-13T09:12:19.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3797000000000002e-05 2024-09-13T09:12:19.222 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.222 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.222 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.222 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4400.0:1332 3.1c e26/21 3:3d5facff:::rbd_header.11238d3f93be:head v 26'72, mlcod=26'70)", 2024-09-13T09:12:19.222 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.064870+0000", 2024-09-13T09:12:19.222 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.078775712999999997, 2024-09-13T09:12:19.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020625579999999999, 2024-09-13T09:12:19.222 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.222 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.223 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064870+0000", 2024-09-13T09:12:19.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064870+0000", 2024-09-13T09:12:19.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064869+0000", 2024-09-13T09:12:19.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064877+0000", 2024-09-13T09:12:19.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0569999999999993e-06 2024-09-13T09:12:19.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064889+0000", 2024-09-13T09:12:19.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1987000000000001e-05 2024-09-13T09:12:19.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064896+0000", 2024-09-13T09:12:19.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:12:19.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064937+0000", 2024-09-13T09:12:19.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1151e-05 2024-09-13T09:12:19.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064960+0000", 2024-09-13T09:12:19.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2651999999999999e-05 2024-09-13T09:12:19.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.066898+0000", 2024-09-13T09:12:19.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001938431 2024-09-13T09:12:19.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.066932+0000", 2024-09-13T09:12:19.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4375999999999997e-05 2024-09-13T09:12:19.226 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.226 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.226 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.226 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1369 3.11 3:8ebbb892:::rbd_data.11238d3f93be.0000000000000036:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1961984~40960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.226 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.079330+0000", 2024-09-13T09:12:19.226 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.064315281000000002, 2024-09-13T09:12:19.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001204922, 2024-09-13T09:12:19.227 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.227 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.227 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.227 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.227 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.227 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1369 2024-09-13T09:12:19.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.227 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079330+0000", 2024-09-13T09:12:19.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079330+0000", 2024-09-13T09:12:19.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079329+0000", 2024-09-13T09:12:19.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079366+0000", 2024-09-13T09:12:19.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6678999999999999e-05 2024-09-13T09:12:19.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079378+0000", 2024-09-13T09:12:19.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1578e-05 2024-09-13T09:12:19.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079382+0000", 2024-09-13T09:12:19.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8620000000000002e-06 2024-09-13T09:12:19.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079418+0000", 2024-09-13T09:12:19.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5859999999999999e-05 2024-09-13T09:12:19.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079520+0000", 2024-09-13T09:12:19.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010165799999999999 2024-09-13T09:12:19.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:19.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079614+0000", 2024-09-13T09:12:19.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3875999999999994e-05 2024-09-13T09:12:19.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.080159+0000", 2024-09-13T09:12:19.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00054515 2024-09-13T09:12:19.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.080475+0000", 2024-09-13T09:12:19.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00031621399999999999 2024-09-13T09:12:19.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.080504+0000", 2024-09-13T09:12:19.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9090000000000001e-05 2024-09-13T09:12:19.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.080535+0000", 2024-09-13T09:12:19.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0982999999999998e-05 2024-09-13T09:12:19.232 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.232 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.232 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.232 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1370 3.11 3:8ebbb892:::rbd_data.11238d3f93be.0000000000000036:head [sparse-read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.232 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.079555+0000", 2024-09-13T09:12:19.232 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.064091006000000006, 2024-09-13T09:12:19.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0011121779999999999, 2024-09-13T09:12:19.233 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.233 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:19.233 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.233 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.233 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.233 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1370 2024-09-13T09:12:19.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.233 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079555+0000", 2024-09-13T09:12:19.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079555+0000", 2024-09-13T09:12:19.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079554+0000", 2024-09-13T09:12:19.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:19.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079562+0000", 2024-09-13T09:12:19.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5769999999999996e-06 2024-09-13T09:12:19.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079567+0000", 2024-09-13T09:12:19.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-13T09:12:19.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079571+0000", 2024-09-13T09:12:19.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.613e-06 2024-09-13T09:12:19.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079926+0000", 2024-09-13T09:12:19.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00035561099999999998 2024-09-13T09:12:19.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for rw locks", 2024-09-13T09:12:19.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.079975+0000", 2024-09-13T09:12:19.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8053000000000002e-05 2024-09-13T09:12:19.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.080557+0000", 2024-09-13T09:12:19.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00058286700000000004 2024-09-13T09:12:19.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.080598+0000", 2024-09-13T09:12:19.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0855000000000001e-05 2024-09-13T09:12:19.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.080667+0000", 2024-09-13T09:12:19.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8503000000000005e-05 2024-09-13T09:12:19.237 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.237 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.237 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.238 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1375 3.11 3:8ebbb892:::rbd_data.11238d3f93be.0000000000000036:head [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.238 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.080973+0000", 2024-09-13T09:12:19.238 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.062672625999999995, 2024-09-13T09:12:19.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00081141299999999998, 2024-09-13T09:12:19.238 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.238 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:19.239 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.239 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.239 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.239 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1375 2024-09-13T09:12:19.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.239 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.080973+0000", 2024-09-13T09:12:19.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.080973+0000", 2024-09-13T09:12:19.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.080973+0000", 2024-09-13T09:12:19.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:19.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.080976+0000", 2024-09-13T09:12:19.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5949999999999999e-06 2024-09-13T09:12:19.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.080983+0000", 2024-09-13T09:12:19.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7839999999999998e-06 2024-09-13T09:12:19.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.080987+0000", 2024-09-13T09:12:19.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.454e-06 2024-09-13T09:12:19.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.081643+0000", 2024-09-13T09:12:19.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00065591399999999995 2024-09-13T09:12:19.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.081715+0000", 2024-09-13T09:12:19.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1824999999999998e-05 2024-09-13T09:12:19.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.081784+0000", 2024-09-13T09:12:19.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9182000000000003e-05 2024-09-13T09:12:19.246 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.246 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.246 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.247 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4400.0:1413 3.1c e26/21 3:3b8551ab:::rbd_data.11238d3f93be.000000000000000a:head v 26'75, mlcod=26'73)", 2024-09-13T09:12:19.247 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.100276+0000", 2024-09-13T09:12:19.247 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.043369976999999997, 2024-09-13T09:12:19.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001616948, 2024-09-13T09:12:19.247 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.247 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.247 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.100276+0000", 2024-09-13T09:12:19.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.100276+0000", 2024-09-13T09:12:19.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.100275+0000", 2024-09-13T09:12:19.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.100294+0000", 2024-09-13T09:12:19.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9587999999999999e-05 2024-09-13T09:12:19.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.100306+0000", 2024-09-13T09:12:19.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1164e-05 2024-09-13T09:12:19.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.100310+0000", 2024-09-13T09:12:19.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.848e-06 2024-09-13T09:12:19.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.100351+0000", 2024-09-13T09:12:19.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0243000000000001e-05 2024-09-13T09:12:19.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.100376+0000", 2024-09-13T09:12:19.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5735e-05 2024-09-13T09:12:19.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.101856+0000", 2024-09-13T09:12:19.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0014794420000000001 2024-09-13T09:12:19.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.101893+0000", 2024-09-13T09:12:19.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6718000000000003e-05 2024-09-13T09:12:19.251 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.251 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.251 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.251 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4400.0:1437 3.1c e26/21 3:3d5facff:::rbd_header.11238d3f93be:head v 26'77, mlcod=26'75)", 2024-09-13T09:12:19.251 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.107782+0000", 2024-09-13T09:12:19.252 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.035863155000000001, 2024-09-13T09:12:19.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00198313, 2024-09-13T09:12:19.252 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.252 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.252 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.107782+0000", 2024-09-13T09:12:19.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.107782+0000", 2024-09-13T09:12:19.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.107782+0000", 2024-09-13T09:12:19.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.107789+0000", 2024-09-13T09:12:19.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.593e-06 2024-09-13T09:12:19.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.107802+0000", 2024-09-13T09:12:19.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3191e-05 2024-09-13T09:12:19.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.107809+0000", 2024-09-13T09:12:19.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2110000000000003e-06 2024-09-13T09:12:19.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.107849+0000", 2024-09-13T09:12:19.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0197000000000002e-05 2024-09-13T09:12:19.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.107870+0000", 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1138e-05 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.109735+0000", 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001865083 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.109766+0000", 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0465999999999999e-05 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.255 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.256 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4400.0:1458 3.1c e26/21 3:3b8551ab:::rbd_data.11238d3f93be.000000000000000a:head v 26'78, mlcod=26'76)", 2024-09-13T09:12:19.256 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.122065+0000", 2024-09-13T09:12:19.256 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.021580841999999999, 2024-09-13T09:12:19.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00085708500000000005, 2024-09-13T09:12:19.256 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.256 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.256 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122065+0000", 2024-09-13T09:12:19.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122065+0000", 2024-09-13T09:12:19.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122064+0000", 2024-09-13T09:12:19.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122088+0000", 2024-09-13T09:12:19.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4172000000000002e-05 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122103+0000", 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4591e-05 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122109+0000", 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7510000000000002e-06 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122159+0000", 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9424e-05 2024-09-13T09:12:19.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122183+0000", 2024-09-13T09:12:19.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4722000000000001e-05 2024-09-13T09:12:19.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122880+0000", 2024-09-13T09:12:19.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00069663699999999995 2024-09-13T09:12:19.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122922+0000", 2024-09-13T09:12:19.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1767999999999997e-05 2024-09-13T09:12:19.259 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.260 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.260 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.260 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.260 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.260 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:19.262 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:19.260+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:19.262 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:19.263 INFO:teuthology.orchestra.run.smithi157.stdout:218 read 0x3daac00 thru 0x3dac372 (0x1773 bytes) 2024-09-13T09:12:19.263 INFO:teuthology.orchestra.run.smithi157.stdout:219 read 0x36da200 thru 0x36dab6a (0x96b bytes) 2024-09-13T09:12:19.263 INFO:teuthology.orchestra.run.smithi157.stdout:220 write 0x961a000 thru 0x9626fe2 (0xcfe3 bytes) 2024-09-13T09:12:19.264 INFO:teuthology.orchestra.run.smithi157.stdout:221 trunc from 0x9627000 to 0x6ca9e00 2024-09-13T09:12:19.264 INFO:teuthology.orchestra.run.smithi157.stdout:222 read 0x1165200 thru 0x11665f4 (0x13f5 bytes) 2024-09-13T09:12:19.264 INFO:teuthology.orchestra.run.smithi157.stdout:226 trunc from 0x6ca9e00 to 0x99fa000 2024-09-13T09:12:19.264 INFO:teuthology.orchestra.run.smithi157.stdout:227 punch from 0x6537800 to 0x6546400, (0xec00 bytes) 2024-09-13T09:12:19.264 INFO:teuthology.orchestra.run.smithi157.stdout:229 read 0x1c20600 thru 0x1c28c0a (0x860b bytes) 2024-09-13T09:12:19.264 INFO:teuthology.orchestra.run.smithi157.stdout:230 read 0x3085800 thru 0x308b449 (0x5c4a bytes) 2024-09-13T09:12:19.264 INFO:teuthology.orchestra.run.smithi157.stdout:232 read 0x1d19400 thru 0x1d1cbdb (0x37dc bytes) 2024-09-13T09:12:19.264 INFO:teuthology.orchestra.run.smithi157.stdout:233 clone 1 order 20 su 131072 sc 4 2024-09-13T09:12:19.361 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:19.360+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:19.385 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-09-13T09:12:19.409 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:19.409 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:19.410 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:19.410 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:19.419 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:19.462 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:19.461+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:19.565 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:19.565 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:19.565 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:19.565 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:19.566 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:19.566 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:19.563+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:19.572 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:19.665 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:19.664+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:19.698 INFO:teuthology.orchestra.run.smithi157.stdout:234 read 0x1ae0c00 thru 0x1aee839 (0xdc3a bytes) 2024-09-13T09:12:19.703 INFO:teuthology.orchestra.run.smithi157.stdout:236 trunc from 0x99fa000 to 0x5808200 2024-09-13T09:12:19.721 INFO:teuthology.orchestra.run.smithi157.stdout:238 read 0x3118400 thru 0x312615e (0xdd5f bytes) 2024-09-13T09:12:19.722 INFO:teuthology.orchestra.run.smithi157.stdout:239 write 0x332c200 thru 0x332fd89 (0x3b8a bytes) 2024-09-13T09:12:19.722 INFO:teuthology.orchestra.run.smithi157.stdout:240 punch from 0x329f600 to 0x32a2a00, (0x3400 bytes) 2024-09-13T09:12:19.723 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:19.724 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:19.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:19.724 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:19.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.724 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4221.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:12:19.724 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:04.320224+0000", 2024-09-13T09:12:19.724 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.401101891, 2024-09-13T09:12:19.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010683929, 2024-09-13T09:12:19.724 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.724 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.725 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.725 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4221", 2024-09-13T09:12:19.725 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.44:0/558773684", 2024-09-13T09:12:19.725 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1 2024-09-13T09:12:19.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.725 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320224+0000", 2024-09-13T09:12:19.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320224+0000", 2024-09-13T09:12:19.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320220+0000", 2024-09-13T09:12:19.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999967 2024-09-13T09:12:19.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320230+0000", 2024-09-13T09:12:19.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0275000000000001e-05 2024-09-13T09:12:19.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320241+0000", 2024-09-13T09:12:19.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0154e-05 2024-09-13T09:12:19.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320256+0000", 2024-09-13T09:12:19.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5068999999999999e-05 2024-09-13T09:12:19.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320328+0000", 2024-09-13T09:12:19.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2507000000000002e-05 2024-09-13T09:12:19.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:19.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320336+0000", 2024-09-13T09:12:19.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:12:19.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.322475+0000", 2024-09-13T09:12:19.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021399430000000001 2024-09-13T09:12:19.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:19.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.322484+0000", 2024-09-13T09:12:19.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5779999999999999e-06 2024-09-13T09:12:19.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.325065+0000", 2024-09-13T09:12:19.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002581323 2024-09-13T09:12:19.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.325211+0000", 2024-09-13T09:12:19.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014534899999999999 2024-09-13T09:12:19.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:19.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.325374+0000", 2024-09-13T09:12:19.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000162898 2024-09-13T09:12:19.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.326528+0000", 2024-09-13T09:12:19.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001154415 2024-09-13T09:12:19.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.330811+0000", 2024-09-13T09:12:19.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042825090000000003 2024-09-13T09:12:19.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.330862+0000", 2024-09-13T09:12:19.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1430000000000001e-05 2024-09-13T09:12:19.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.330908+0000", 2024-09-13T09:12:19.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5636000000000001e-05 2024-09-13T09:12:19.731 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.731 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.731 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.732 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4221.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:12:19.732 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:05.322972+0000", 2024-09-13T09:12:19.732 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.398353745, 2024-09-13T09:12:19.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0041565500000000002, 2024-09-13T09:12:19.732 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.732 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.732 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.732 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4221", 2024-09-13T09:12:19.732 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.44:0/558773684", 2024-09-13T09:12:19.732 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 5 2024-09-13T09:12:19.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.732 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.322972+0000", 2024-09-13T09:12:19.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.322972+0000", 2024-09-13T09:12:19.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.322970+0000", 2024-09-13T09:12:19.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:12:19.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.322980+0000", 2024-09-13T09:12:19.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0866999999999999e-05 2024-09-13T09:12:19.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.322999+0000", 2024-09-13T09:12:19.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8349999999999999e-05 2024-09-13T09:12:19.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.323016+0000", 2024-09-13T09:12:19.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6988999999999999e-05 2024-09-13T09:12:19.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.323090+0000", 2024-09-13T09:12:19.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4369000000000006e-05 2024-09-13T09:12:19.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:19.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.323098+0000", 2024-09-13T09:12:19.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3760000000000002e-06 2024-09-13T09:12:19.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.325767+0000", 2024-09-13T09:12:19.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026697729999999999 2024-09-13T09:12:19.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.325881+0000", 2024-09-13T09:12:19.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011373499999999999 2024-09-13T09:12:19.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:19.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.325985+0000", 2024-09-13T09:12:19.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103627 2024-09-13T09:12:19.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.326752+0000", 2024-09-13T09:12:19.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00076745899999999996 2024-09-13T09:12:19.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.327067+0000", 2024-09-13T09:12:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000314369 2024-09-13T09:12:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.327104+0000", 2024-09-13T09:12:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7320000000000002e-05 2024-09-13T09:12:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:05.327128+0000", 2024-09-13T09:12:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4527000000000001e-05 2024-09-13T09:12:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4221.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:12:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:06.327956+0000", 2024-09-13T09:12:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.393369461000001, 2024-09-13T09:12:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033160099999999999, 2024-09-13T09:12:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4221", 2024-09-13T09:12:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.44:0/558773684", 2024-09-13T09:12:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 7 2024-09-13T09:12:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.327956+0000", 2024-09-13T09:12:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.327956+0000", 2024-09-13T09:12:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.327954+0000", 2024-09-13T09:12:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:12:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.327965+0000", 2024-09-13T09:12:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0861e-05 2024-09-13T09:12:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.327983+0000", 2024-09-13T09:12:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8189000000000002e-05 2024-09-13T09:12:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.327995+0000", 2024-09-13T09:12:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1943000000000001e-05 2024-09-13T09:12:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.328059+0000", 2024-09-13T09:12:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3905000000000006e-05 2024-09-13T09:12:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.328065+0000", 2024-09-13T09:12:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3450000000000002e-06 2024-09-13T09:12:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.329859+0000", 2024-09-13T09:12:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0017938209999999999 2024-09-13T09:12:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.329928+0000", 2024-09-13T09:12:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8823000000000002e-05 2024-09-13T09:12:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.330025+0000", 2024-09-13T09:12:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7536999999999998e-05 2024-09-13T09:12:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.330915+0000", 2024-09-13T09:12:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00088944299999999996 2024-09-13T09:12:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.331211+0000", 2024-09-13T09:12:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029682500000000002 2024-09-13T09:12:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.331245+0000", 2024-09-13T09:12:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3454999999999997e-05 2024-09-13T09:12:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:06.331272+0000", 2024-09-13T09:12:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7341000000000001e-05 2024-09-13T09:12:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:12:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:16.385493+0000", 2024-09-13T09:12:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.3358320849999998, 2024-09-13T09:12:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063914480000000001, 2024-09-13T09:12:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:12:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:12:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 10 2024-09-13T09:12:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385493+0000", 2024-09-13T09:12:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385493+0000", 2024-09-13T09:12:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385492+0000", 2024-09-13T09:12:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385498+0000", 2024-09-13T09:12:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2219999999999999e-06 2024-09-13T09:12:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385509+0000", 2024-09-13T09:12:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1207e-05 2024-09-13T09:12:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385519+0000", 2024-09-13T09:12:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8560000000000002e-06 2024-09-13T09:12:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385560+0000", 2024-09-13T09:12:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1223000000000002e-05 2024-09-13T09:12:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385565+0000", 2024-09-13T09:12:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9409999999999998e-06 2024-09-13T09:12:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388159+0000", 2024-09-13T09:12:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0025934 2024-09-13T09:12:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388261+0000", 2024-09-13T09:12:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102799 2024-09-13T09:12:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388406+0000", 2024-09-13T09:12:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000144857 2024-09-13T09:12:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.389373+0000", 2024-09-13T09:12:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00096632900000000004 2024-09-13T09:12:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.391797+0000", 2024-09-13T09:12:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0024246580000000001 2024-09-13T09:12:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.391847+0000", 2024-09-13T09:12:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9811999999999997e-05 2024-09-13T09:12:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.391885+0000", 2024-09-13T09:12:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7796999999999997e-05 2024-09-13T09:12:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1285 3.1c 3:3b8551ab:::rbd_data.11238d3f93be.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.023041+0000", 2024-09-13T09:12:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.69828488899999996, 2024-09-13T09:12:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0041865260000000003, 2024-09-13T09:12:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1285 2024-09-13T09:12:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023041+0000", 2024-09-13T09:12:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023041+0000", 2024-09-13T09:12:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023039+0000", 2024-09-13T09:12:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023050+0000", 2024-09-13T09:12:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0166999999999999e-05 2024-09-13T09:12:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023067+0000", 2024-09-13T09:12:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7825999999999998e-05 2024-09-13T09:12:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023081+0000", 2024-09-13T09:12:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3550000000000001e-05 2024-09-13T09:12:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023752+0000", 2024-09-13T09:12:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00067136500000000005 2024-09-13T09:12:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023812+0000", 2024-09-13T09:12:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9794999999999999e-05 2024-09-13T09:12:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023906+0000", 2024-09-13T09:12:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3425999999999996e-05 2024-09-13T09:12:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.024817+0000", 2024-09-13T09:12:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00091123300000000001 2024-09-13T09:12:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.027177+0000", 2024-09-13T09:12:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002360371 2024-09-13T09:12:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.027207+0000", 2024-09-13T09:12:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9430000000000001e-05 2024-09-13T09:12:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.027227+0000", 2024-09-13T09:12:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.065e-05 2024-09-13T09:12:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1288 3.5 3:a01381ee:::rbd_data.11238d3f93be.000000000000000d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.023125+0000", 2024-09-13T09:12:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.69820087900000005, 2024-09-13T09:12:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002475727, 2024-09-13T09:12:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1288 2024-09-13T09:12:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023125+0000", 2024-09-13T09:12:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023125+0000", 2024-09-13T09:12:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023124+0000", 2024-09-13T09:12:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023126+0000", 2024-09-13T09:12:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3519999999999999e-06 2024-09-13T09:12:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023129+0000", 2024-09-13T09:12:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9059999999999998e-06 2024-09-13T09:12:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023131+0000", 2024-09-13T09:12:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:12:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023573+0000", 2024-09-13T09:12:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00044168499999999998 2024-09-13T09:12:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023646+0000", 2024-09-13T09:12:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3467000000000007e-05 2024-09-13T09:12:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.023725+0000", 2024-09-13T09:12:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8328e-05 2024-09-13T09:12:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.024474+0000", 2024-09-13T09:12:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00074954000000000004 2024-09-13T09:12:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.025562+0000", 2024-09-13T09:12:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010874389999999999 2024-09-13T09:12:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.025583+0000", 2024-09-13T09:12:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1831000000000002e-05 2024-09-13T09:12:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.025600+0000", 2024-09-13T09:12:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.698e-05 2024-09-13T09:12:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1299 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.027520+0000", 2024-09-13T09:12:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.69380573199999995, 2024-09-13T09:12:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0024384620000000002, 2024-09-13T09:12:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1299 2024-09-13T09:12:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.027520+0000", 2024-09-13T09:12:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.027520+0000", 2024-09-13T09:12:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.027519+0000", 2024-09-13T09:12:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.027524+0000", 2024-09-13T09:12:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3769999999999998e-06 2024-09-13T09:12:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.027532+0000", 2024-09-13T09:12:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9099999999999994e-06 2024-09-13T09:12:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.027537+0000", 2024-09-13T09:12:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:12:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.027564+0000", 2024-09-13T09:12:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7250000000000002e-05 2024-09-13T09:12:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.027618+0000", 2024-09-13T09:12:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3585000000000003e-05 2024-09-13T09:12:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.027845+0000", 2024-09-13T09:12:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022729099999999999 2024-09-13T09:12:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.028601+0000", 2024-09-13T09:12:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00075610300000000005 2024-09-13T09:12:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.029899+0000", 2024-09-13T09:12:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0012978009999999999 2024-09-13T09:12:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.029926+0000", 2024-09-13T09:12:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6948e-05 2024-09-13T09:12:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.029958+0000", 2024-09-13T09:12:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2224000000000002e-05 2024-09-13T09:12:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1332 3.1c 3:3d5facff:::rbd_header.11238d3f93be:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.064591+0000", 2024-09-13T09:12:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.65673504900000002, 2024-09-13T09:12:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0024693559999999998, 2024-09-13T09:12:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1332 2024-09-13T09:12:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064591+0000", 2024-09-13T09:12:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064591+0000", 2024-09-13T09:12:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064590+0000", 2024-09-13T09:12:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064595+0000", 2024-09-13T09:12:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:12:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064602+0000", 2024-09-13T09:12:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9430000000000003e-06 2024-09-13T09:12:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064605+0000", 2024-09-13T09:12:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.518e-06 2024-09-13T09:12:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064633+0000", 2024-09-13T09:12:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7450999999999999e-05 2024-09-13T09:12:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064682+0000", 2024-09-13T09:12:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8859000000000001e-05 2024-09-13T09:12:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.064800+0000", 2024-09-13T09:12:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011796 2024-09-13T09:12:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.066878+0000", 2024-09-13T09:12:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002078604 2024-09-13T09:12:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.067012+0000", 2024-09-13T09:12:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013353 2024-09-13T09:12:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.067039+0000", 2024-09-13T09:12:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7166999999999999e-05 2024-09-13T09:12:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.067060+0000", 2024-09-13T09:12:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0999e-05 2024-09-13T09:12:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1400 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.096977+0000", 2024-09-13T09:12:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.624348232, 2024-09-13T09:12:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026188359999999998, 2024-09-13T09:12:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1400 2024-09-13T09:12:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.096977+0000", 2024-09-13T09:12:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.096977+0000", 2024-09-13T09:12:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.096976+0000", 2024-09-13T09:12:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.096982+0000", 2024-09-13T09:12:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5960000000000003e-06 2024-09-13T09:12:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.096993+0000", 2024-09-13T09:12:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0827e-05 2024-09-13T09:12:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.096998+0000", 2024-09-13T09:12:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9729999999999997e-06 2024-09-13T09:12:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097035+0000", 2024-09-13T09:12:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6758000000000003e-05 2024-09-13T09:12:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097117+0000", 2024-09-13T09:12:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2693000000000003e-05 2024-09-13T09:12:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097526+0000", 2024-09-13T09:12:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00040819000000000001 2024-09-13T09:12:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.099498+0000", 2024-09-13T09:12:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001972264 2024-09-13T09:12:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.099508+0000", 2024-09-13T09:12:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8260000000000005e-06 2024-09-13T09:12:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.099558+0000", 2024-09-13T09:12:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0018000000000001e-05 2024-09-13T09:12:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.099596+0000", 2024-09-13T09:12:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8569000000000003e-05 2024-09-13T09:12:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1401 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.097006+0000", 2024-09-13T09:12:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.62431932300000004, 2024-09-13T09:12:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0028145589999999999, 2024-09-13T09:12:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1401 2024-09-13T09:12:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097006+0000", 2024-09-13T09:12:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097006+0000", 2024-09-13T09:12:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097006+0000", 2024-09-13T09:12:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097009+0000", 2024-09-13T09:12:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9050000000000001e-06 2024-09-13T09:12:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097015+0000", 2024-09-13T09:12:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9059999999999999e-06 2024-09-13T09:12:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097021+0000", 2024-09-13T09:12:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6089999999999997e-06 2024-09-13T09:12:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097915+0000", 2024-09-13T09:12:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00089416500000000004 2024-09-13T09:12:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.098002+0000", 2024-09-13T09:12:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6248999999999994e-05 2024-09-13T09:12:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.098371+0000", 2024-09-13T09:12:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036974799999999998 2024-09-13T09:12:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.099604+0000", 2024-09-13T09:12:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0012327690000000001 2024-09-13T09:12:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.099764+0000", 2024-09-13T09:12:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016013299999999999 2024-09-13T09:12:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.099788+0000", 2024-09-13T09:12:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4046999999999999e-05 2024-09-13T09:12:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.099821+0000", 2024-09-13T09:12:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2521000000000003e-05 2024-09-13T09:12:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1402 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.097026+0000", 2024-09-13T09:12:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.62429938799999996, 2024-09-13T09:12:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0047909620000000002, 2024-09-13T09:12:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1402 2024-09-13T09:12:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097026+0000", 2024-09-13T09:12:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097026+0000", 2024-09-13T09:12:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097026+0000", 2024-09-13T09:12:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097028+0000", 2024-09-13T09:12:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5849999999999998e-06 2024-09-13T09:12:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097035+0000", 2024-09-13T09:12:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8090000000000004e-06 2024-09-13T09:12:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.097040+0000", 2024-09-13T09:12:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1070000000000004e-06 2024-09-13T09:12:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.098692+0000", 2024-09-13T09:12:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0016521260000000001 2024-09-13T09:12:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.098773+0000", 2024-09-13T09:12:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0599999999999994e-05 2024-09-13T09:12:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.099135+0000", 2024-09-13T09:12:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036187399999999998 2024-09-13T09:12:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.100127+0000", 2024-09-13T09:12:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00099219600000000009 2024-09-13T09:12:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.101727+0000", 2024-09-13T09:12:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015995040000000001 2024-09-13T09:12:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.101780+0000", 2024-09-13T09:12:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3097999999999999e-05 2024-09-13T09:12:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.101817+0000", 2024-09-13T09:12:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7460999999999999e-05 2024-09-13T09:12:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1457 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.121699+0000", 2024-09-13T09:12:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.59962625800000002, 2024-09-13T09:12:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0037659099999999999, 2024-09-13T09:12:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1457 2024-09-13T09:12:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121699+0000", 2024-09-13T09:12:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121699+0000", 2024-09-13T09:12:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121699+0000", 2024-09-13T09:12:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121702+0000", 2024-09-13T09:12:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6090000000000001e-06 2024-09-13T09:12:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121709+0000", 2024-09-13T09:12:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8750000000000002e-06 2024-09-13T09:12:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121714+0000", 2024-09-13T09:12:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6949999999999999e-06 2024-09-13T09:12:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121746+0000", 2024-09-13T09:12:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1985e-05 2024-09-13T09:12:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121824+0000", 2024-09-13T09:12:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8631e-05 2024-09-13T09:12:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122200+0000", 2024-09-13T09:12:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00037505299999999997 2024-09-13T09:12:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124304+0000", 2024-09-13T09:12:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021042809999999999 2024-09-13T09:12:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.125408+0000", 2024-09-13T09:12:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001103712 2024-09-13T09:12:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.125435+0000", 2024-09-13T09:12:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7081e-05 2024-09-13T09:12:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.125465+0000", 2024-09-13T09:12:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0642999999999998e-05 2024-09-13T09:12:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1459 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.121753+0000", 2024-09-13T09:12:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.59957302899999998, 2024-09-13T09:12:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065126389999999998, 2024-09-13T09:12:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1459 2024-09-13T09:12:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121753+0000", 2024-09-13T09:12:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121753+0000", 2024-09-13T09:12:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121752+0000", 2024-09-13T09:12:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121754+0000", 2024-09-13T09:12:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9750000000000001e-06 2024-09-13T09:12:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121759+0000", 2024-09-13T09:12:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0239999999999997e-06 2024-09-13T09:12:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121762+0000", 2024-09-13T09:12:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8049999999999998e-06 2024-09-13T09:12:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122595+0000", 2024-09-13T09:12:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00083291799999999998 2024-09-13T09:12:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122697+0000", 2024-09-13T09:12:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101727 2024-09-13T09:12:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122961+0000", 2024-09-13T09:12:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026407399999999999 2024-09-13T09:12:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124336+0000", 2024-09-13T09:12:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0013751270000000001 2024-09-13T09:12:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128157+0000", 2024-09-13T09:12:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038212419999999999 2024-09-13T09:12:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128205+0000", 2024-09-13T09:12:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7722999999999997e-05 2024-09-13T09:12:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128265+0000", 2024-09-13T09:12:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.033e-05 2024-09-13T09:12:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1460 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.121842+0000", 2024-09-13T09:12:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.599483286, 2024-09-13T09:12:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065154749999999997, 2024-09-13T09:12:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1460 2024-09-13T09:12:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121842+0000", 2024-09-13T09:12:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121842+0000", 2024-09-13T09:12:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121842+0000", 2024-09-13T09:12:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121845+0000", 2024-09-13T09:12:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:12:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121849+0000", 2024-09-13T09:12:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2540000000000003e-06 2024-09-13T09:12:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121852+0000", 2024-09-13T09:12:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8710000000000001e-06 2024-09-13T09:12:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.123228+0000", 2024-09-13T09:12:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001376592 2024-09-13T09:12:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.123280+0000", 2024-09-13T09:12:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1975999999999998e-05 2024-09-13T09:12:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.123509+0000", 2024-09-13T09:12:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022916000000000001 2024-09-13T09:12:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.125491+0000", 2024-09-13T09:12:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0019817379999999998 2024-09-13T09:12:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128275+0000", 2024-09-13T09:12:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002784201 2024-09-13T09:12:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128328+0000", 2024-09-13T09:12:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3006e-05 2024-09-13T09:12:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128358+0000", 2024-09-13T09:12:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9397e-05 2024-09-13T09:12:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1462 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.122137+0000", 2024-09-13T09:12:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.59918897500000001, 2024-09-13T09:12:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063313809999999996, 2024-09-13T09:12:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1462 2024-09-13T09:12:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122137+0000", 2024-09-13T09:12:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122137+0000", 2024-09-13T09:12:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122136+0000", 2024-09-13T09:12:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122140+0000", 2024-09-13T09:12:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.425e-06 2024-09-13T09:12:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122148+0000", 2024-09-13T09:12:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1170000000000004e-06 2024-09-13T09:12:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122154+0000", 2024-09-13T09:12:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5990000000000001e-06 2024-09-13T09:12:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.123777+0000", 2024-09-13T09:12:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00162301 2024-09-13T09:12:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.123826+0000", 2024-09-13T09:12:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9051999999999997e-05 2024-09-13T09:12:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124038+0000", 2024-09-13T09:12:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021218499999999999 2024-09-13T09:12:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.125699+0000", 2024-09-13T09:12:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0016603589999999999 2024-09-13T09:12:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128373+0000", 2024-09-13T09:12:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026739770000000001 2024-09-13T09:12:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128426+0000", 2024-09-13T09:12:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3643000000000001e-05 2024-09-13T09:12:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128468+0000", 2024-09-13T09:12:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1826000000000002e-05 2024-09-13T09:12:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1463 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.122162+0000", 2024-09-13T09:12:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.59916373000000001, 2024-09-13T09:12:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064746990000000004, 2024-09-13T09:12:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1463 2024-09-13T09:12:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122162+0000", 2024-09-13T09:12:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122162+0000", 2024-09-13T09:12:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122161+0000", 2024-09-13T09:12:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122165+0000", 2024-09-13T09:12:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5209999999999998e-06 2024-09-13T09:12:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122172+0000", 2024-09-13T09:12:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2540000000000003e-06 2024-09-13T09:12:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122177+0000", 2024-09-13T09:12:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-13T09:12:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124358+0000", 2024-09-13T09:12:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021809350000000002 2024-09-13T09:12:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124407+0000", 2024-09-13T09:12:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9469999999999999e-05 2024-09-13T09:12:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124633+0000", 2024-09-13T09:12:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022538300000000001 2024-09-13T09:12:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.126674+0000", 2024-09-13T09:12:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020412540000000002 2024-09-13T09:12:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128475+0000", 2024-09-13T09:12:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001801137 2024-09-13T09:12:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128598+0000", 2024-09-13T09:12:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012318600000000001 2024-09-13T09:12:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128637+0000", 2024-09-13T09:12:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8219000000000001e-05 2024-09-13T09:12:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1465 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.122262+0000", 2024-09-13T09:12:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.59906311599999995, 2024-09-13T09:12:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006503043, 2024-09-13T09:12:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1465 2024-09-13T09:12:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122262+0000", 2024-09-13T09:12:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122262+0000", 2024-09-13T09:12:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122261+0000", 2024-09-13T09:12:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122268+0000", 2024-09-13T09:12:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1720000000000004e-06 2024-09-13T09:12:19.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122277+0000", 2024-09-13T09:12:19.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7459999999999995e-06 2024-09-13T09:12:19.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122284+0000", 2024-09-13T09:12:19.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.511e-06 2024-09-13T09:12:19.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124897+0000", 2024-09-13T09:12:19.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026131420000000002 2024-09-13T09:12:19.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124947+0000", 2024-09-13T09:12:19.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9543000000000003e-05 2024-09-13T09:12:19.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:19.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.125149+0000", 2024-09-13T09:12:19.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000202709 2024-09-13T09:12:19.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.126698+0000", 2024-09-13T09:12:19.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015487649999999999 2024-09-13T09:12:19.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128645+0000", 2024-09-13T09:12:19.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001946632 2024-09-13T09:12:19.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128684+0000", 2024-09-13T09:12:19.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9860000000000001e-05 2024-09-13T09:12:19.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128766+0000", 2024-09-13T09:12:19.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1025000000000004e-05 2024-09-13T09:12:19.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.840 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.840 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.840 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4400.0:1475 3.17 e26/21 3:ec2b659c:::rbd_data.11238d3f93be.000000000000002b:head v 26'3, mlcod=26'1)", 2024-09-13T09:12:19.840 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.126151+0000", 2024-09-13T09:12:19.840 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.59517490900000003, 2024-09-13T09:12:19.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0024387369999999998, 2024-09-13T09:12:19.840 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.840 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.840 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.126151+0000", 2024-09-13T09:12:19.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.126151+0000", 2024-09-13T09:12:19.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.126150+0000", 2024-09-13T09:12:19.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.126167+0000", 2024-09-13T09:12:19.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7778000000000002e-05 2024-09-13T09:12:19.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.126178+0000", 2024-09-13T09:12:19.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0421999999999999e-05 2024-09-13T09:12:19.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.126182+0000", 2024-09-13T09:12:19.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:12:19.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.126220+0000", 2024-09-13T09:12:19.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7997000000000002e-05 2024-09-13T09:12:19.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.126236+0000", 2024-09-13T09:12:19.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5631999999999999e-05 2024-09-13T09:12:19.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128553+0000", 2024-09-13T09:12:19.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0023168379999999999 2024-09-13T09:12:19.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128589+0000", 2024-09-13T09:12:19.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6559e-05 2024-09-13T09:12:19.845 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.845 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.845 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.846 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1599 3.1c 3:3d5facff:::rbd_header.11238d3f93be:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:19.846 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.188780+0000", 2024-09-13T09:12:19.846 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.53254539899999997, 2024-09-13T09:12:19.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0024815390000000001, 2024-09-13T09:12:19.846 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.846 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.846 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.846 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.846 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.847 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1599 2024-09-13T09:12:19.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.847 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.188780+0000", 2024-09-13T09:12:19.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.188780+0000", 2024-09-13T09:12:19.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.188779+0000", 2024-09-13T09:12:19.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.188784+0000", 2024-09-13T09:12:19.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0629999999999999e-06 2024-09-13T09:12:19.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.188794+0000", 2024-09-13T09:12:19.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0797e-05 2024-09-13T09:12:19.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.188799+0000", 2024-09-13T09:12:19.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8040000000000001e-06 2024-09-13T09:12:19.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.188825+0000", 2024-09-13T09:12:19.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6016e-05 2024-09-13T09:12:19.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.188872+0000", 2024-09-13T09:12:19.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6573000000000003e-05 2024-09-13T09:12:19.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:19.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.188993+0000", 2024-09-13T09:12:19.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120858 2024-09-13T09:12:19.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.191209+0000", 2024-09-13T09:12:19.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022159889999999998 2024-09-13T09:12:19.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.191213+0000", 2024-09-13T09:12:19.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1540000000000004e-06 2024-09-13T09:12:19.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.191241+0000", 2024-09-13T09:12:19.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8687000000000002e-05 2024-09-13T09:12:19.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.191262+0000", 2024-09-13T09:12:19.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0330999999999999e-05 2024-09-13T09:12:19.852 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.852 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.852 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.852 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1682 3.1c 3:3d5facff:::rbd_header.11238d3f93be:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:19.852 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.384805+0000", 2024-09-13T09:12:19.852 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.33652079800000001, 2024-09-13T09:12:19.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003300384, 2024-09-13T09:12:19.852 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:19.852 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:19.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:19.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:19.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:19.853 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1682 2024-09-13T09:12:19.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.853 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:19.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:19.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.384805+0000", 2024-09-13T09:12:19.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:19.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.384805+0000", 2024-09-13T09:12:19.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:19.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:19.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.384803+0000", 2024-09-13T09:12:19.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:12:19.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:19.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.384812+0000", 2024-09-13T09:12:19.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1220000000000001e-06 2024-09-13T09:12:19.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:19.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.384830+0000", 2024-09-13T09:12:19.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8031999999999999e-05 2024-09-13T09:12:19.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.384838+0000", 2024-09-13T09:12:19.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1769999999999998e-06 2024-09-13T09:12:19.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.384886+0000", 2024-09-13T09:12:19.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7840999999999998e-05 2024-09-13T09:12:19.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:19.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.384892+0000", 2024-09-13T09:12:19.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2809999999999999e-06 2024-09-13T09:12:19.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:19.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.386712+0000", 2024-09-13T09:12:19.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001820487 2024-09-13T09:12:19.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:19.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.386783+0000", 2024-09-13T09:12:19.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1458999999999995e-05 2024-09-13T09:12:19.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:19.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.387027+0000", 2024-09-13T09:12:19.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000243415 2024-09-13T09:12:19.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:19.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.387715+0000", 2024-09-13T09:12:19.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00068826399999999995 2024-09-13T09:12:19.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:19.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.388009+0000", 2024-09-13T09:12:19.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029422999999999998 2024-09-13T09:12:19.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:19.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.388059+0000", 2024-09-13T09:12:19.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9573000000000001e-05 2024-09-13T09:12:19.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:19.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:19.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:19.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.388105+0000", 2024-09-13T09:12:19.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6223000000000001e-05 2024-09-13T09:12:19.858 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.858 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.858 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.858 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:19.858 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:19.859 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:19.861 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:19.864 INFO:teuthology.orchestra.run.smithi157.stdout:243 write 0xee32000 thru 0xee345ce (0x25cf bytes) 2024-09-13T09:12:19.864 INFO:teuthology.orchestra.run.smithi157.stdout:244 write 0xa540800 thru 0xa54aebb (0xa6bc bytes) 2024-09-13T09:12:19.864 INFO:teuthology.orchestra.run.smithi157.stdout:245 write 0x2205000 thru 0x220afde (0x5fdf bytes) 2024-09-13T09:12:19.864 INFO:teuthology.orchestra.run.smithi157.stdout:246 punch from 0xa51fa00 to 0xa520a00, (0x1000 bytes) 2024-09-13T09:12:19.864 INFO:teuthology.orchestra.run.smithi157.stdout:248 write 0x439ce00 thru 0x43a6d2e (0x9f2f bytes) 2024-09-13T09:12:19.864 INFO:teuthology.orchestra.run.smithi157.stdout:249 write 0x301c400 thru 0x301fa49 (0x364a bytes) 2024-09-13T09:12:19.864 INFO:teuthology.orchestra.run.smithi157.stdout:250 write 0xc9dfa00 thru 0xc9e6d92 (0x7393 bytes) 2024-09-13T09:12:19.865 INFO:teuthology.orchestra.run.smithi157.stdout:251 write 0x21be200 thru 0x21be800 (0x601 bytes) 2024-09-13T09:12:19.865 INFO:teuthology.orchestra.run.smithi157.stdout:252 write 0x7a2e400 thru 0x7a35275 (0x6e76 bytes) 2024-09-13T09:12:19.865 INFO:teuthology.orchestra.run.smithi157.stdout:255 read 0x214aa00 thru 0x2158182 (0xd783 bytes) 2024-09-13T09:12:19.865 INFO:teuthology.orchestra.run.smithi157.stdout:257 write 0x45be00 thru 0x464c1b (0x8e1c bytes) 2024-09-13T09:12:19.865 INFO:teuthology.orchestra.run.smithi157.stdout:261 punch from 0x282fc00 to 0x283c400, (0xc800 bytes) 2024-09-13T09:12:19.865 INFO:teuthology.orchestra.run.smithi157.stdout:264 read 0x4f51400 thru 0x4f531bb (0x1dbc bytes) 2024-09-13T09:12:19.865 INFO:teuthology.orchestra.run.smithi157.stdout:265 read 0x6e60800 thru 0x6e62706 (0x1f07 bytes) 2024-09-13T09:12:19.865 INFO:teuthology.orchestra.run.smithi157.stdout:266 write 0xc14a800 thru 0xc152816 (0x8017 bytes) 2024-09-13T09:12:19.865 INFO:teuthology.orchestra.run.smithi157.stdout:267 read 0x5547e00 thru 0x554d011 (0x5212 bytes) 2024-09-13T09:12:19.865 INFO:teuthology.orchestra.run.smithi157.stdout:268 write 0x587f800 thru 0x58877a4 (0x7fa5 bytes) 2024-09-13T09:12:19.865 INFO:teuthology.orchestra.run.smithi157.stdout:269 write 0x619dc00 thru 0x61a82a5 (0xa6a6 bytes) 2024-09-13T09:12:19.865 INFO:teuthology.orchestra.run.smithi157.stdout:272 write 0x342b400 thru 0x3433887 (0x8488 bytes) 2024-09-13T09:12:19.865 INFO:teuthology.orchestra.run.smithi157.stdout:273 write 0x2db9c00 thru 0x2dc1107 (0x7508 bytes) 2024-09-13T09:12:19.866 INFO:teuthology.orchestra.run.smithi157.stdout:277 read 0xbfe8800 thru 0xbfed0d2 (0x48d3 bytes) 2024-09-13T09:12:19.866 INFO:teuthology.orchestra.run.smithi157.stdout:278 read 0x2078e00 thru 0x207e13b (0x533c bytes) 2024-09-13T09:12:19.866 INFO:teuthology.orchestra.run.smithi157.stdout:279 read 0x2de3600 thru 0x2df31c8 (0xfbc9 bytes) 2024-09-13T09:12:19.866 INFO:teuthology.orchestra.run.smithi157.stdout:280 write 0x24b6600 thru 0x24ba30f (0x3d10 bytes) 2024-09-13T09:12:19.866 INFO:teuthology.orchestra.run.smithi157.stdout:281 write 0xc2bfc00 thru 0xc2cd15b (0xd55c bytes) 2024-09-13T09:12:19.866 INFO:teuthology.orchestra.run.smithi157.stdout:282 write 0xe6d2600 thru 0xe6ddabd (0xb4be bytes) 2024-09-13T09:12:19.866 INFO:teuthology.orchestra.run.smithi157.stdout:283 trunc from 0xee34600 to 0xaaa3a00 2024-09-13T09:12:19.866 INFO:teuthology.orchestra.run.smithi157.stdout:284 read 0x7c73a00 thru 0x7c75354 (0x1955 bytes) 2024-09-13T09:12:19.866 INFO:teuthology.orchestra.run.smithi157.stdout:287 read 0x6bb4000 thru 0x6bb5110 (0x1111 bytes) 2024-09-13T09:12:19.866 INFO:teuthology.orchestra.run.smithi157.stdout:288 read 0x194fc00 thru 0x1956d6a (0x716b bytes) 2024-09-13T09:12:19.866 INFO:teuthology.orchestra.run.smithi157.stdout:293 read 0x57c0000 thru 0x57c0aba (0xabb bytes) 2024-09-13T09:12:19.866 INFO:teuthology.orchestra.run.smithi157.stdout:295 read 0x5371c00 thru 0x537f6f4 (0xdaf5 bytes) 2024-09-13T09:12:19.866 INFO:teuthology.orchestra.run.smithi157.stdout:298 write 0xe15d200 thru 0xe165728 (0x8529 bytes) 2024-09-13T09:12:19.867 INFO:teuthology.orchestra.run.smithi157.stdout:299 write 0x4ac7200 thru 0x4ac74af (0x2b0 bytes) 2024-09-13T09:12:19.867 INFO:teuthology.orchestra.run.smithi157.stdout:301 read 0xe4dc00 thru 0xe53758 (0x5b59 bytes) 2024-09-13T09:12:19.867 INFO:teuthology.orchestra.run.smithi157.stdout:303 trunc from 0xe165800 to 0x71d4c00 2024-09-13T09:12:19.867 INFO:teuthology.orchestra.run.smithi157.stdout:305 read 0x719d600 thru 0x719dcbf (0x6c0 bytes) 2024-09-13T09:12:19.867 INFO:teuthology.orchestra.run.smithi157.stdout:306 trunc from 0x71d4c00 to 0x22d8800 2024-09-13T09:12:19.867 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:19.859+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:19.915 INFO:teuthology.orchestra.run.smithi157.stdout:307 write 0xe894a00 thru 0xe8a4362 (0xf963 bytes) 2024-09-13T09:12:19.921 INFO:teuthology.orchestra.run.smithi157.stdout:308 read 0xedd400 thru 0xee8687 (0xb288 bytes) 2024-09-13T09:12:19.922 INFO:teuthology.orchestra.run.smithi157.stdout:311 write 0xa565400 thru 0xa5661ca (0xdcb bytes) 2024-09-13T09:12:19.922 INFO:teuthology.orchestra.run.smithi157.stdout:314 read 0x7c11800 thru 0x7c14847 (0x3048 bytes) 2024-09-13T09:12:19.923 INFO:teuthology.orchestra.run.smithi157.stdout:317 trunc from 0xe8a4400 to 0xdbc3000 2024-09-13T09:12:19.940 INFO:teuthology.orchestra.run.smithi157.stdout:319 read 0x7ec800 thru 0x7f1bc7 (0x53c8 bytes) 2024-09-13T09:12:19.954 INFO:teuthology.orchestra.run.smithi157.stdout:320 trunc from 0xdbc3000 to 0x8c7d600 2024-09-13T09:12:19.961 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:19.959+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:19.969 INFO:teuthology.orchestra.run.smithi157.stdout:321 punch from 0x2e56c00 to 0x2e5a200, (0x3600 bytes) 2024-09-13T09:12:19.973 INFO:teuthology.orchestra.run.smithi157.stdout:322 punch from 0x631ba00 to 0x631e400, (0x2a00 bytes) 2024-09-13T09:12:19.973 INFO:teuthology.orchestra.run.smithi157.stdout:325 punch from 0x51a9200 to 0x51b7e00, (0xec00 bytes) 2024-09-13T09:12:19.974 INFO:teuthology.orchestra.run.smithi157.stdout:328 read 0x78e4600 thru 0x78f1ad8 (0xd4d9 bytes) 2024-09-13T09:12:19.974 INFO:teuthology.orchestra.run.smithi157.stdout:330 read 0x8550a00 thru 0x8552910 (0x1f11 bytes) 2024-09-13T09:12:19.974 INFO:teuthology.orchestra.run.smithi157.stdout:331 punch from 0x319ea00 to 0x31a3200, (0x4800 bytes) 2024-09-13T09:12:19.974 INFO:teuthology.orchestra.run.smithi157.stdout:333 write 0x5455200 thru 0x545df54 (0x8d55 bytes) 2024-09-13T09:12:19.974 INFO:teuthology.orchestra.run.smithi157.stdout:334 write 0x39bde00 thru 0x39bf5cc (0x17cd bytes) 2024-09-13T09:12:19.974 INFO:teuthology.orchestra.run.smithi157.stdout:335 read 0x3af0a00 thru 0x3b004ea (0xfaeb bytes) 2024-09-13T09:12:19.974 INFO:teuthology.orchestra.run.smithi157.stdout:336 punch from 0x8978a00 to 0x8985e00, (0xd400 bytes) 2024-09-13T09:12:19.974 INFO:teuthology.orchestra.run.smithi157.stdout:339 read 0x3fbda00 thru 0x3fc692b (0x8f2c bytes) 2024-09-13T09:12:19.974 INFO:teuthology.orchestra.run.smithi157.stdout:340 write 0x650a000 thru 0x650f420 (0x5421 bytes) 2024-09-13T09:12:19.984 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:19.984 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:19.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:19.984 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:712 3.17 3:ef11896e:::rbd_data.1123ffac6bc2.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write 363008~30208 in=30208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:19.984 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.981113+0000", 2024-09-13T09:12:19.984 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0022550769999999999, 2024-09-13T09:12:19.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022740159999999998, 2024-09-13T09:12:19.984 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:19.984 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:12:19.984 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 712 2024-09-13T09:12:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981113+0000", 2024-09-13T09:12:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:19.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981113+0000", 2024-09-13T09:12:19.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:19.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:19.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:19.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:19.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981112+0000", 2024-09-13T09:12:19.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:19.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:19.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:19.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:19.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981162+0000", 2024-09-13T09:12:19.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0034999999999998e-05 2024-09-13T09:12:19.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:19.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:19.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:19.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981173+0000", 2024-09-13T09:12:19.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1092e-05 2024-09-13T09:12:19.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:19.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:19.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:19.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981178+0000", 2024-09-13T09:12:19.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1080000000000001e-06 2024-09-13T09:12:19.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:19.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:19.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:19.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981227+0000", 2024-09-13T09:12:19.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8513999999999997e-05 2024-09-13T09:12:19.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:19.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:19.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:19.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981357+0000", 2024-09-13T09:12:19.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012995699999999999 2024-09-13T09:12:19.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:19.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:19.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:19.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981510+0000", 2024-09-13T09:12:19.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015285799999999999 2024-09-13T09:12:19.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:19.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:19.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:19.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.982145+0000", 2024-09-13T09:12:19.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00063491700000000003 2024-09-13T09:12:19.988 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:19.989 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:19.989 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:19.989 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:19.989 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:12:19.989 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 1 2024-09-13T09:12:19.989 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:19.994 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:19.995 INFO:teuthology.orchestra.run.smithi157.stdout:341 write 0xbe85a00 thru 0xbe92fa1 (0xd5a2 bytes) 2024-09-13T09:12:20.013 INFO:teuthology.orchestra.run.smithi157.stdout:342 trunc from 0xbe93000 to 0x2c8bc00 2024-09-13T09:12:20.034 INFO:teuthology.orchestra.run.smithi157.stdout:344 read 0x805000 thru 0x80ce13 (0x7e14 bytes) 2024-09-13T09:12:20.035 INFO:teuthology.orchestra.run.smithi157.stdout:345 read 0x1549600 thru 0x1551055 (0x7a56 bytes) 2024-09-13T09:12:20.035 INFO:teuthology.orchestra.run.smithi157.stdout:347 punch from 0x1ca600 to 0x1d7400, (0xce00 bytes) 2024-09-13T09:12:20.035 INFO:teuthology.orchestra.run.smithi157.stdout:351 read 0x44a200 thru 0x459c88 (0xfa89 bytes) 2024-09-13T09:12:20.036 INFO:teuthology.orchestra.run.smithi157.stdout:352 read 0xdc2c00 thru 0xdcfebb (0xd2bc bytes) 2024-09-13T09:12:20.053 INFO:teuthology.orchestra.run.smithi157.stdout:353 write 0x75a3600 thru 0x75a9e05 (0x6806 bytes) 2024-09-13T09:12:20.062 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:20.061+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:20.075 INFO:teuthology.orchestra.run.smithi157.stdout:354 write 0xe234800 thru 0xe2413e2 (0xcbe3 bytes) 2024-09-13T09:12:20.076 INFO:teuthology.orchestra.run.smithi157.stdout:355 write 0x2dd4400 thru 0x2ddfcf2 (0xb8f3 bytes) 2024-09-13T09:12:20.077 INFO:teuthology.orchestra.run.smithi157.stdout:357 write 0x713a200 thru 0x7149a87 (0xf888 bytes) 2024-09-13T09:12:20.077 INFO:teuthology.orchestra.run.smithi157.stdout:358 write 0x15bf000 thru 0x15c767b (0x867c bytes) 2024-09-13T09:12:20.077 INFO:teuthology.orchestra.run.smithi157.stdout:359 punch from 0xde95c00 to 0xde98800, (0x2c00 bytes) 2024-09-13T09:12:20.078 INFO:teuthology.orchestra.run.smithi157.stdout:361 read 0xc371c00 thru 0xc38076b (0xeb6c bytes) 2024-09-13T09:12:20.078 INFO:teuthology.orchestra.run.smithi157.stdout:362 write 0x66fc600 thru 0x66fdf56 (0x1957 bytes) 2024-09-13T09:12:20.078 INFO:teuthology.orchestra.run.smithi157.stdout:363 write 0x7733000 thru 0x773bba6 (0x8ba7 bytes) 2024-09-13T09:12:20.092 INFO:teuthology.orchestra.run.smithi157.stdout:364 trunc from 0xe241400 to 0x26b1200 2024-09-13T09:12:20.112 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:20.112 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:20.112 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:20.113 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:20.113 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:20.122 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:20.123 INFO:teuthology.orchestra.run.smithi157.stdout:365 trunc from 0x26b1200 to 0xea5d000 2024-09-13T09:12:20.131 INFO:teuthology.orchestra.run.smithi157.stdout:366 write 0xa033e00 thru 0xa03dd2c (0x9f2d bytes) 2024-09-13T09:12:20.132 INFO:teuthology.orchestra.run.smithi157.stdout:367 read 0xdbdfe00 thru 0xdbeba0d (0xbc0e bytes) 2024-09-13T09:12:20.133 INFO:teuthology.orchestra.run.smithi157.stdout:368 read 0x6a15a00 thru 0x6a18d0a (0x330b bytes) 2024-09-13T09:12:20.133 INFO:teuthology.orchestra.run.smithi157.stdout:369 punch from 0x296c200 to 0x297b000, (0xee00 bytes) 2024-09-13T09:12:20.135 INFO:teuthology.orchestra.run.smithi157.stdout:372 read 0x6ee4400 thru 0x6ef2d6a (0xe96b bytes) 2024-09-13T09:12:20.135 INFO:teuthology.orchestra.run.smithi157.stdout:373 write 0x919b400 thru 0x91a4670 (0x9271 bytes) 2024-09-13T09:12:20.135 INFO:teuthology.orchestra.run.smithi157.stdout:374 read 0xaba7200 thru 0xabb2bae (0xb9af bytes) 2024-09-13T09:12:20.135 INFO:teuthology.orchestra.run.smithi157.stdout:376 read 0x7ab0800 thru 0x7ab669c (0x5e9d bytes) 2024-09-13T09:12:20.135 INFO:teuthology.orchestra.run.smithi157.stdout:380 read 0x427e600 thru 0x4283117 (0x4b18 bytes) 2024-09-13T09:12:20.135 INFO:teuthology.orchestra.run.smithi157.stdout:381 write 0x5ec5e00 thru 0x5ec6084 (0x285 bytes) 2024-09-13T09:12:20.135 INFO:teuthology.orchestra.run.smithi157.stdout:382 punch from 0xa0a7000 to 0xa0adc00, (0x6c00 bytes) 2024-09-13T09:12:20.136 INFO:teuthology.orchestra.run.smithi157.stdout:383 read 0xe7b1800 thru 0xe7bd788 (0xbf89 bytes) 2024-09-13T09:12:20.136 INFO:teuthology.orchestra.run.smithi157.stdout:385 write 0x4fa5800 thru 0x4fadcdb (0x84dc bytes) 2024-09-13T09:12:20.136 INFO:teuthology.orchestra.run.smithi157.stdout:386 read 0x9e05a00 thru 0x9e0e9a0 (0x8fa1 bytes) 2024-09-13T09:12:20.136 INFO:teuthology.orchestra.run.smithi157.stdout:388 write 0xf3be00 thru 0xf43569 (0x776a bytes) 2024-09-13T09:12:20.136 INFO:teuthology.orchestra.run.smithi157.stdout:392 punch from 0x355f600 to 0x3564e00, (0x5800 bytes) 2024-09-13T09:12:20.136 INFO:teuthology.orchestra.run.smithi157.stdout:393 read 0x7e3b400 thru 0x7e3cc19 (0x181a bytes) 2024-09-13T09:12:20.136 INFO:teuthology.orchestra.run.smithi157.stdout:394 write 0x4ab8c00 thru 0x4ac4c67 (0xc068 bytes) 2024-09-13T09:12:20.136 INFO:teuthology.orchestra.run.smithi157.stdout:396 write 0xb371c00 thru 0xb378771 (0x6b72 bytes) 2024-09-13T09:12:20.136 INFO:teuthology.orchestra.run.smithi157.stdout:397 write 0x653bc00 thru 0x653cd6d (0x116e bytes) 2024-09-13T09:12:20.136 INFO:teuthology.orchestra.run.smithi157.stdout:398 write 0x1e66c00 thru 0x1e69e5a (0x325b bytes) 2024-09-13T09:12:20.136 INFO:teuthology.orchestra.run.smithi157.stdout:399 write 0xb56d600 thru 0xb574d61 (0x7762 bytes) 2024-09-13T09:12:20.136 INFO:teuthology.orchestra.run.smithi157.stdout:406 read 0xa10aa00 thru 0xa117299 (0xc89a bytes) 2024-09-13T09:12:20.136 INFO:teuthology.orchestra.run.smithi157.stdout:407 write 0x2a37a00 thru 0x2a37e75 (0x476 bytes) 2024-09-13T09:12:20.148 INFO:teuthology.orchestra.run.smithi157.stdout:408 trunc from 0xea5d000 to 0x41ae600 2024-09-13T09:12:20.163 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:20.161+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:20.172 INFO:teuthology.orchestra.run.smithi157.stdout:409 read 0x3600800 thru 0x360eaed (0xe2ee bytes) 2024-09-13T09:12:20.173 INFO:teuthology.orchestra.run.smithi157.stdout:414 read 0x812e00 thru 0x81ecdb (0xbedc bytes) 2024-09-13T09:12:20.173 INFO:teuthology.orchestra.run.smithi157.stdout:415 punch from 0x11f8a00 to 0x1203400, (0xaa00 bytes) 2024-09-13T09:12:20.176 INFO:teuthology.orchestra.run.smithi157.stdout:420 trunc from 0x41ae600 to 0x5cd5400 2024-09-13T09:12:20.207 INFO:teuthology.orchestra.run.smithi157.stdout:423 write 0xd432600 thru 0xd4374e7 (0x4ee8 bytes) 2024-09-13T09:12:20.218 INFO:teuthology.orchestra.run.smithi157.stdout:424 trunc from 0xd437600 to 0x52ce200 2024-09-13T09:12:20.234 INFO:teuthology.orchestra.run.smithi157.stdout:425 read 0x3c4b200 thru 0x3c59163 (0xdf64 bytes) 2024-09-13T09:12:20.261 INFO:teuthology.orchestra.run.smithi157.stdout:426 write 0xe430000 thru 0xe43e78a (0xe78b bytes) 2024-09-13T09:12:20.262 INFO:teuthology.orchestra.run.smithi157.stdout:427 read 0x72b1600 thru 0x72be8b8 (0xd2b9 bytes) 2024-09-13T09:12:20.262 INFO:teuthology.orchestra.run.smithi157.stdout:428 read 0xaac1200 thru 0xaacc514 (0xb315 bytes) 2024-09-13T09:12:20.263 INFO:teuthology.orchestra.run.smithi157.stdout:432 read 0x8cb0a00 thru 0x8cbcaae (0xc0af bytes) 2024-09-13T09:12:20.263 INFO:teuthology.orchestra.run.smithi157.stdout:433 punch from 0x47e6400 to 0x47efa00, (0x9600 bytes) 2024-09-13T09:12:20.264 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:20.262+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:20.267 INFO:teuthology.orchestra.run.smithi157.stdout:434 read 0x4bb5600 thru 0x4bb5c8b (0x68c bytes) 2024-09-13T09:12:20.267 INFO:teuthology.orchestra.run.smithi157.stdout:438 write 0xab21c00 thru 0xab2f5cc (0xd9cd bytes) 2024-09-13T09:12:20.267 INFO:teuthology.orchestra.run.smithi157.stdout:440 write 0x920c600 thru 0x921c157 (0xfb58 bytes) 2024-09-13T09:12:20.267 INFO:teuthology.orchestra.run.smithi157.stdout:443 punch from 0x54a9000 to 0x54b3800, (0xa800 bytes) 2024-09-13T09:12:20.276 INFO:teuthology.orchestra.run.smithi157.stdout:447 trunc from 0xe43e800 to 0x7205000 2024-09-13T09:12:20.283 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:20.283 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:20.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:20.284 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:20.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.284 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:1 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'1, mlcod=0'0)", 2024-09-13T09:12:20.284 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.218895+0000", 2024-09-13T09:12:20.284 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.062498998999999, 2024-09-13T09:12:20.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0043868170000000003, 2024-09-13T09:12:20.284 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.284 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.284 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.218895+0000", 2024-09-13T09:12:20.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.218895+0000", 2024-09-13T09:12:20.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.218894+0000", 2024-09-13T09:12:20.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.286 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.218920+0000", 2024-09-13T09:12:20.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6044e-05 2024-09-13T09:12:20.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.286 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.286 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.218955+0000", 2024-09-13T09:12:20.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4555999999999999e-05 2024-09-13T09:12:20.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.286 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.286 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.218976+0000", 2024-09-13T09:12:20.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1103000000000001e-05 2024-09-13T09:12:20.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.287 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.219081+0000", 2024-09-13T09:12:20.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105015 2024-09-13T09:12:20.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.287 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.219174+0000", 2024-09-13T09:12:20.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2917999999999994e-05 2024-09-13T09:12:20.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.287 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.223249+0000", 2024-09-13T09:12:20.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0040751870000000001 2024-09-13T09:12:20.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.223281+0000", 2024-09-13T09:12:20.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2691000000000003e-05 2024-09-13T09:12:20.288 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.288 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.288 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.288 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:56 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'34, mlcod=14'29)", 2024-09-13T09:12:20.288 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.276889+0000", 2024-09-13T09:12:20.288 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.004504953000001, 2024-09-13T09:12:20.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0060300759999999997, 2024-09-13T09:12:20.289 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.289 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.289 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276889+0000", 2024-09-13T09:12:20.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276889+0000", 2024-09-13T09:12:20.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276888+0000", 2024-09-13T09:12:20.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:20.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276895+0000", 2024-09-13T09:12:20.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:12:20.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276908+0000", 2024-09-13T09:12:20.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3074999999999999e-05 2024-09-13T09:12:20.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.291 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.276914+0000", 2024-09-13T09:12:20.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4739999999999996e-06 2024-09-13T09:12:20.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.291 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.277225+0000", 2024-09-13T09:12:20.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00031188100000000002 2024-09-13T09:12:20.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.292 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.292 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.277242+0000", 2024-09-13T09:12:20.292 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7079e-05 2024-09-13T09:12:20.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.292 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.292 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.282885+0000", 2024-09-13T09:12:20.292 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0056424370000000001 2024-09-13T09:12:20.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.292 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.292 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.282919+0000", 2024-09-13T09:12:20.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3807999999999999e-05 2024-09-13T09:12:20.293 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.293 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.293 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.293 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:69 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'43, mlcod=14'39)", 2024-09-13T09:12:20.293 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.299371+0000", 2024-09-13T09:12:20.293 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.982022583999999, 2024-09-13T09:12:20.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049236979999999998, 2024-09-13T09:12:20.293 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.293 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.293 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299371+0000", 2024-09-13T09:12:20.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299371+0000", 2024-09-13T09:12:20.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299370+0000", 2024-09-13T09:12:20.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:20.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299424+0000", 2024-09-13T09:12:20.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3879999999999999e-05 2024-09-13T09:12:20.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299436+0000", 2024-09-13T09:12:20.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2150000000000001e-05 2024-09-13T09:12:20.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299441+0000", 2024-09-13T09:12:20.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1789999999999999e-06 2024-09-13T09:12:20.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299481+0000", 2024-09-13T09:12:20.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9249999999999999e-05 2024-09-13T09:12:20.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.299500+0000", 2024-09-13T09:12:20.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9188999999999999e-05 2024-09-13T09:12:20.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.297 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.297 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.304259+0000", 2024-09-13T09:12:20.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047587269999999999 2024-09-13T09:12:20.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.297 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.297 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.304295+0000", 2024-09-13T09:12:20.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6137000000000003e-05 2024-09-13T09:12:20.297 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.297 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.297 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.298 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:70 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'44, mlcod=14'39)", 2024-09-13T09:12:20.298 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.300075+0000", 2024-09-13T09:12:20.298 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.981318172999998, 2024-09-13T09:12:20.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042356549999999996, 2024-09-13T09:12:20.298 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.298 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.298 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300075+0000", 2024-09-13T09:12:20.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300075+0000", 2024-09-13T09:12:20.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300075+0000", 2024-09-13T09:12:20.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300080+0000", 2024-09-13T09:12:20.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7010000000000004e-06 2024-09-13T09:12:20.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300088+0000", 2024-09-13T09:12:20.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0160000000000008e-06 2024-09-13T09:12:20.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300091+0000", 2024-09-13T09:12:20.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8489999999999999e-06 2024-09-13T09:12:20.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300127+0000", 2024-09-13T09:12:20.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5212000000000001e-05 2024-09-13T09:12:20.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.300141+0000", 2024-09-13T09:12:20.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4048e-05 2024-09-13T09:12:20.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.304297+0000", 2024-09-13T09:12:20.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004156234 2024-09-13T09:12:20.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.302 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.304311+0000", 2024-09-13T09:12:20.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4126e-05 2024-09-13T09:12:20.302 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.302 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.302 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.302 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4192.0:122 1.0 e15/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'84, mlcod=15'82)", 2024-09-13T09:12:20.302 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:00.484733+0000", 2024-09-13T09:12:20.302 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.796661103000002, 2024-09-13T09:12:20.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042987140000000004, 2024-09-13T09:12:20.302 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.302 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.303 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484733+0000", 2024-09-13T09:12:20.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484733+0000", 2024-09-13T09:12:20.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484732+0000", 2024-09-13T09:12:20.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484736+0000", 2024-09-13T09:12:20.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0620000000000002e-06 2024-09-13T09:12:20.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484744+0000", 2024-09-13T09:12:20.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1859999999999992e-06 2024-09-13T09:12:20.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484751+0000", 2024-09-13T09:12:20.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.985e-06 2024-09-13T09:12:20.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484778+0000", 2024-09-13T09:12:20.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.69e-05 2024-09-13T09:12:20.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.484792+0000", 2024-09-13T09:12:20.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3329000000000001e-05 2024-09-13T09:12:20.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.306 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.489000+0000", 2024-09-13T09:12:20.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042089040000000003 2024-09-13T09:12:20.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.306 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:00.489031+0000", 2024-09-13T09:12:20.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0806000000000003e-05 2024-09-13T09:12:20.306 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.306 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.306 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.306 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4400.0:1499 3.11 e26/22 3:8ebbb892:::rbd_data.11238d3f93be.0000000000000036:head v 26'4, mlcod=26'2)", 2024-09-13T09:12:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.137751+0000", 2024-09-13T09:12:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.143642861, 2024-09-13T09:12:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0054978259999999999, 2024-09-13T09:12:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.137751+0000", 2024-09-13T09:12:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.137751+0000", 2024-09-13T09:12:20.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.137750+0000", 2024-09-13T09:12:20.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.137756+0000", 2024-09-13T09:12:20.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0809999999999999e-06 2024-09-13T09:12:20.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.137766+0000", 2024-09-13T09:12:20.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8199999999999992e-06 2024-09-13T09:12:20.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.137772+0000", 2024-09-13T09:12:20.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1519999999999997e-06 2024-09-13T09:12:20.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.137821+0000", 2024-09-13T09:12:20.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9184000000000002e-05 2024-09-13T09:12:20.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.137843+0000", 2024-09-13T09:12:20.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1491000000000001e-05 2024-09-13T09:12:20.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.143221+0000", 2024-09-13T09:12:20.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.005378491 2024-09-13T09:12:20.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.311 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.311 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.143249+0000", 2024-09-13T09:12:20.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7319000000000001e-05 2024-09-13T09:12:20.311 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.311 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.311 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.311 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:231 3.12 3:4ca9cdc5:::rbd_data.1123ffac6bc2.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 516096~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.311 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.758927+0000", 2024-09-13T09:12:20.311 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.522466125, 2024-09-13T09:12:20.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063432189999999998, 2024-09-13T09:12:20.311 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.312 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.312 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.312 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.312 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.312 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 231 2024-09-13T09:12:20.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.312 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.758927+0000", 2024-09-13T09:12:20.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.758927+0000", 2024-09-13T09:12:20.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.758927+0000", 2024-09-13T09:12:20.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:20.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.758935+0000", 2024-09-13T09:12:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3920000000000003e-06 2024-09-13T09:12:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.758947+0000", 2024-09-13T09:12:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.203e-05 2024-09-13T09:12:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.758954+0000", 2024-09-13T09:12:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3159999999999999e-06 2024-09-13T09:12:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.758995+0000", 2024-09-13T09:12:20.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0191999999999997e-05 2024-09-13T09:12:20.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.759076+0000", 2024-09-13T09:12:20.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1662999999999994e-05 2024-09-13T09:12:20.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:20.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.759214+0000", 2024-09-13T09:12:20.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137732 2024-09-13T09:12:20.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.760685+0000", 2024-09-13T09:12:20.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001470758 2024-09-13T09:12:20.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.765161+0000", 2024-09-13T09:12:20.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044758749999999998 2024-09-13T09:12:20.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.765216+0000", 2024-09-13T09:12:20.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4991000000000003e-05 2024-09-13T09:12:20.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.765271+0000", 2024-09-13T09:12:20.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5124999999999998e-05 2024-09-13T09:12:20.317 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.317 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.317 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.317 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:234 3.17 3:e97a3561:::rbd_data.1123ffac6bc2.0000000000000079:head [set-alloc-hint object_size 1048576 write_size 1048576,write 581632~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.317 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.759078+0000", 2024-09-13T09:12:20.317 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.52231516200000006, 2024-09-13T09:12:20.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071288979999999998, 2024-09-13T09:12:20.318 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.318 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.318 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.318 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.318 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.318 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 234 2024-09-13T09:12:20.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.318 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.759078+0000", 2024-09-13T09:12:20.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.759078+0000", 2024-09-13T09:12:20.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.759074+0000", 2024-09-13T09:12:20.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999957 2024-09-13T09:12:20.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.759112+0000", 2024-09-13T09:12:20.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8214999999999999e-05 2024-09-13T09:12:20.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.759122+0000", 2024-09-13T09:12:20.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.815e-06 2024-09-13T09:12:20.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.759130+0000", 2024-09-13T09:12:20.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8879999999999997e-06 2024-09-13T09:12:20.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.759170+0000", 2024-09-13T09:12:20.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9458000000000001e-05 2024-09-13T09:12:20.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.759252+0000", 2024-09-13T09:12:20.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2692e-05 2024-09-13T09:12:20.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.759343+0000", 2024-09-13T09:12:20.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0425000000000002e-05 2024-09-13T09:12:20.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761047+0000", 2024-09-13T09:12:20.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001704239 2024-09-13T09:12:20.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.766129+0000", 2024-09-13T09:12:20.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.005081721 2024-09-13T09:12:20.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.766174+0000", 2024-09-13T09:12:20.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5469999999999997e-05 2024-09-13T09:12:20.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.766207+0000", 2024-09-13T09:12:20.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3213999999999997e-05 2024-09-13T09:12:20.323 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.323 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.323 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.323 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:291 3.3 e27/21 3:c3e921d9:::rbd_data.1123ffac6bc2.00000000000000ed:head v 27'6, mlcod=26'4)", 2024-09-13T09:12:20.323 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.778699+0000", 2024-09-13T09:12:20.323 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.50269430999999998, 2024-09-13T09:12:20.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0043504879999999996, 2024-09-13T09:12:20.324 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.324 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.324 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.324 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.324 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778699+0000", 2024-09-13T09:12:20.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.324 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.324 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778699+0000", 2024-09-13T09:12:20.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778699+0000", 2024-09-13T09:12:20.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:20.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778707+0000", 2024-09-13T09:12:20.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3010000000000007e-06 2024-09-13T09:12:20.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778747+0000", 2024-09-13T09:12:20.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0241000000000003e-05 2024-09-13T09:12:20.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778784+0000", 2024-09-13T09:12:20.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7190999999999997e-05 2024-09-13T09:12:20.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778868+0000", 2024-09-13T09:12:20.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3454000000000004e-05 2024-09-13T09:12:20.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778886+0000", 2024-09-13T09:12:20.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.855e-05 2024-09-13T09:12:20.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783012+0000", 2024-09-13T09:12:20.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0041261819999999999 2024-09-13T09:12:20.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783050+0000", 2024-09-13T09:12:20.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7329999999999997e-05 2024-09-13T09:12:20.328 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.328 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.328 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.328 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:396 3.f e27/22 3:f1a6b8aa:::rbd_data.1123ffac6bc2.00000000000000a6:head v 27'9, mlcod=27'7)", 2024-09-13T09:12:20.328 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.837206+0000", 2024-09-13T09:12:20.328 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.44418740800000001, 2024-09-13T09:12:20.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042209639999999998, 2024-09-13T09:12:20.328 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.328 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.329 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837206+0000", 2024-09-13T09:12:20.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837206+0000", 2024-09-13T09:12:20.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837206+0000", 2024-09-13T09:12:20.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837212+0000", 2024-09-13T09:12:20.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3049999999999998e-06 2024-09-13T09:12:20.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837223+0000", 2024-09-13T09:12:20.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.096e-05 2024-09-13T09:12:20.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837231+0000", 2024-09-13T09:12:20.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5930000000000006e-06 2024-09-13T09:12:20.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837325+0000", 2024-09-13T09:12:20.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3412000000000006e-05 2024-09-13T09:12:20.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837347+0000", 2024-09-13T09:12:20.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.211e-05 2024-09-13T09:12:20.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.841387+0000", 2024-09-13T09:12:20.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0040400080000000003 2024-09-13T09:12:20.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.841427+0000", 2024-09-13T09:12:20.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0222000000000002e-05 2024-09-13T09:12:20.332 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.332 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.332 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.332 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:448 3.6 3:63c839dd:::rbd_data.1123ffac6bc2.0000000000000025:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.333 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.865119+0000", 2024-09-13T09:12:20.333 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.416274376, 2024-09-13T09:12:20.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0054284950000000002, 2024-09-13T09:12:20.333 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.333 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.333 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.333 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.333 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.333 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 448 2024-09-13T09:12:20.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.333 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865119+0000", 2024-09-13T09:12:20.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865119+0000", 2024-09-13T09:12:20.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865118+0000", 2024-09-13T09:12:20.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:20.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865132+0000", 2024-09-13T09:12:20.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3569e-05 2024-09-13T09:12:20.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865143+0000", 2024-09-13T09:12:20.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0866000000000001e-05 2024-09-13T09:12:20.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865152+0000", 2024-09-13T09:12:20.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4979999999999995e-06 2024-09-13T09:12:20.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865203+0000", 2024-09-13T09:12:20.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0476000000000003e-05 2024-09-13T09:12:20.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865407+0000", 2024-09-13T09:12:20.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00020450500000000001 2024-09-13T09:12:20.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:20.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865515+0000", 2024-09-13T09:12:20.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000108083 2024-09-13T09:12:20.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.869968+0000", 2024-09-13T09:12:20.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044526599999999998 2024-09-13T09:12:20.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870481+0000", 2024-09-13T09:12:20.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051277799999999998 2024-09-13T09:12:20.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870512+0000", 2024-09-13T09:12:20.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1634000000000002e-05 2024-09-13T09:12:20.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870548+0000", 2024-09-13T09:12:20.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5225999999999998e-05 2024-09-13T09:12:20.338 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.338 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.338 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.339 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:450 3.1b 3:dc49caed:::rbd_data.1123ffac6bc2.0000000000000029:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.339 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.865175+0000", 2024-09-13T09:12:20.339 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.41621877899999998, 2024-09-13T09:12:20.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.005130753, 2024-09-13T09:12:20.339 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.339 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.339 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.339 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.339 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.339 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 450 2024-09-13T09:12:20.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.340 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865175+0000", 2024-09-13T09:12:20.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865175+0000", 2024-09-13T09:12:20.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865174+0000", 2024-09-13T09:12:20.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865178+0000", 2024-09-13T09:12:20.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3390000000000001e-06 2024-09-13T09:12:20.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865184+0000", 2024-09-13T09:12:20.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2820000000000002e-06 2024-09-13T09:12:20.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865191+0000", 2024-09-13T09:12:20.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7070000000000003e-06 2024-09-13T09:12:20.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866304+0000", 2024-09-13T09:12:20.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011136080000000001 2024-09-13T09:12:20.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866389+0000", 2024-09-13T09:12:20.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4377999999999997e-05 2024-09-13T09:12:20.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:20.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866502+0000", 2024-09-13T09:12:20.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000113298 2024-09-13T09:12:20.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.867108+0000", 2024-09-13T09:12:20.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00060625100000000003 2024-09-13T09:12:20.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870163+0000", 2024-09-13T09:12:20.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0030553759999999998 2024-09-13T09:12:20.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870262+0000", 2024-09-13T09:12:20.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8937999999999994e-05 2024-09-13T09:12:20.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870306+0000", 2024-09-13T09:12:20.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3179000000000001e-05 2024-09-13T09:12:20.345 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.345 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.345 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.345 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:456 3.10 e27/21 3:0ce54145:::rbd_data.1123ffac6bc2.0000000000000031:head v 27'3, mlcod=27'1)", 2024-09-13T09:12:20.345 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.865855+0000", 2024-09-13T09:12:20.345 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.41553905699999999, 2024-09-13T09:12:20.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0041926639999999996, 2024-09-13T09:12:20.345 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.346 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.346 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865855+0000", 2024-09-13T09:12:20.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865855+0000", 2024-09-13T09:12:20.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865854+0000", 2024-09-13T09:12:20.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865859+0000", 2024-09-13T09:12:20.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6859999999999997e-06 2024-09-13T09:12:20.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865868+0000", 2024-09-13T09:12:20.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3819999999999993e-06 2024-09-13T09:12:20.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865874+0000", 2024-09-13T09:12:20.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7019999999999998e-06 2024-09-13T09:12:20.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865923+0000", 2024-09-13T09:12:20.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9215000000000002e-05 2024-09-13T09:12:20.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865946+0000", 2024-09-13T09:12:20.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.313e-05 2024-09-13T09:12:20.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870007+0000", 2024-09-13T09:12:20.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0040611850000000001 2024-09-13T09:12:20.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870047+0000", 2024-09-13T09:12:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0005000000000001e-05 2024-09-13T09:12:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:451 3.18 e27/21 3:1c88c91a:::rbd_data.1123ffac6bc2.000000000000002d:head v 27'2, mlcod=0'0)", 2024-09-13T09:12:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.865986+0000", 2024-09-13T09:12:20.350 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.41540761599999998, 2024-09-13T09:12:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0040939929999999998, 2024-09-13T09:12:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865986+0000", 2024-09-13T09:12:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865986+0000", 2024-09-13T09:12:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865985+0000", 2024-09-13T09:12:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866022+0000", 2024-09-13T09:12:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6395e-05 2024-09-13T09:12:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866041+0000", 2024-09-13T09:12:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9301999999999999e-05 2024-09-13T09:12:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866047+0000", 2024-09-13T09:12:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3010000000000003e-06 2024-09-13T09:12:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866093+0000", 2024-09-13T09:12:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5877999999999997e-05 2024-09-13T09:12:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866111+0000", 2024-09-13T09:12:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7813e-05 2024-09-13T09:12:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870049+0000", 2024-09-13T09:12:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0039382380000000002 2024-09-13T09:12:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870080+0000", 2024-09-13T09:12:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0781000000000002e-05 2024-09-13T09:12:20.354 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:462 3.b e27/21 3:d5af952d:::rbd_data.1123ffac6bc2.000000000000004a:head v 27'9, mlcod=27'7)", 2024-09-13T09:12:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.866065+0000", 2024-09-13T09:12:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.415328156, 2024-09-13T09:12:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0040888649999999997, 2024-09-13T09:12:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866065+0000", 2024-09-13T09:12:20.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866065+0000", 2024-09-13T09:12:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866065+0000", 2024-09-13T09:12:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866072+0000", 2024-09-13T09:12:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:12:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866080+0000", 2024-09-13T09:12:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1270000000000003e-06 2024-09-13T09:12:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866085+0000", 2024-09-13T09:12:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2510000000000002e-06 2024-09-13T09:12:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866122+0000", 2024-09-13T09:12:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7110999999999998e-05 2024-09-13T09:12:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866140+0000", 2024-09-13T09:12:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7915999999999999e-05 2024-09-13T09:12:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870104+0000", 2024-09-13T09:12:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0039643370000000001 2024-09-13T09:12:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870154+0000", 2024-09-13T09:12:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9938999999999997e-05 2024-09-13T09:12:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:486 3.b e27/21 3:d6806dca:::rbd_data.1123ffac6bc2.0000000000000060:head v 27'10, mlcod=27'7)", 2024-09-13T09:12:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.870606+0000", 2024-09-13T09:12:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.41078769100000001, 2024-09-13T09:12:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.005054111, 2024-09-13T09:12:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870606+0000", 2024-09-13T09:12:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870606+0000", 2024-09-13T09:12:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870605+0000", 2024-09-13T09:12:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870611+0000", 2024-09-13T09:12:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6919999999999999e-06 2024-09-13T09:12:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870621+0000", 2024-09-13T09:12:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5310000000000007e-06 2024-09-13T09:12:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870625+0000", 2024-09-13T09:12:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6190000000000002e-06 2024-09-13T09:12:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870665+0000", 2024-09-13T09:12:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.015e-05 2024-09-13T09:12:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870717+0000", 2024-09-13T09:12:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1841000000000001e-05 2024-09-13T09:12:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.875633+0000", 2024-09-13T09:12:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049161880000000002 2024-09-13T09:12:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.875660+0000", 2024-09-13T09:12:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.669e-05 2024-09-13T09:12:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:712 3.17 3:ef11896e:::rbd_data.1123ffac6bc2.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write 363008~30208 in=30208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.981113+0000", 2024-09-13T09:12:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.30028048000000002, 2024-09-13T09:12:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0053854510000000003, 2024-09-13T09:12:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 712 2024-09-13T09:12:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981113+0000", 2024-09-13T09:12:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981113+0000", 2024-09-13T09:12:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981112+0000", 2024-09-13T09:12:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981162+0000", 2024-09-13T09:12:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0034999999999998e-05 2024-09-13T09:12:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981173+0000", 2024-09-13T09:12:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1092e-05 2024-09-13T09:12:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981178+0000", 2024-09-13T09:12:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1080000000000001e-06 2024-09-13T09:12:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981227+0000", 2024-09-13T09:12:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8513999999999997e-05 2024-09-13T09:12:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981357+0000", 2024-09-13T09:12:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012995699999999999 2024-09-13T09:12:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.981510+0000", 2024-09-13T09:12:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015285799999999999 2024-09-13T09:12:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.982145+0000", 2024-09-13T09:12:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00063491700000000003 2024-09-13T09:12:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.986443+0000", 2024-09-13T09:12:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042983079999999998 2024-09-13T09:12:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.986475+0000", 2024-09-13T09:12:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2231999999999999e-05 2024-09-13T09:12:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.986499+0000", 2024-09-13T09:12:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3297000000000001e-05 2024-09-13T09:12:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:1055 3.18 e27/21 3:192ac1d1:::rbd_data.1123ffac6bc2.0000000000000091:head v 27'5, mlcod=27'3)", 2024-09-13T09:12:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.140741+0000", 2024-09-13T09:12:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.140652739, 2024-09-13T09:12:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047765689999999996, 2024-09-13T09:12:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.370 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140741+0000", 2024-09-13T09:12:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140741+0000", 2024-09-13T09:12:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140740+0000", 2024-09-13T09:12:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140761+0000", 2024-09-13T09:12:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1120000000000001e-05 2024-09-13T09:12:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140772+0000", 2024-09-13T09:12:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0971e-05 2024-09-13T09:12:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140779+0000", 2024-09-13T09:12:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2489999999999998e-06 2024-09-13T09:12:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140823+0000", 2024-09-13T09:12:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4493999999999998e-05 2024-09-13T09:12:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140841+0000", 2024-09-13T09:12:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7876999999999998e-05 2024-09-13T09:12:20.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145481+0000", 2024-09-13T09:12:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046399379999999997 2024-09-13T09:12:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145517+0000", 2024-09-13T09:12:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6622000000000003e-05 2024-09-13T09:12:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.374 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:1067 3.f e27/22 3:f00ae4d0:::rbd_data.1123ffac6bc2.000000000000000d:head v 27'11, mlcod=27'9)", 2024-09-13T09:12:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.142938+0000", 2024-09-13T09:12:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.138455474, 2024-09-13T09:12:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0053471320000000001, 2024-09-13T09:12:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142938+0000", 2024-09-13T09:12:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142938+0000", 2024-09-13T09:12:20.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142937+0000", 2024-09-13T09:12:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142980+0000", 2024-09-13T09:12:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2852999999999998e-05 2024-09-13T09:12:20.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142989+0000", 2024-09-13T09:12:20.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0890000000000006e-06 2024-09-13T09:12:20.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142995+0000", 2024-09-13T09:12:20.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8289999999999996e-06 2024-09-13T09:12:20.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143055+0000", 2024-09-13T09:12:20.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9590999999999999e-05 2024-09-13T09:12:20.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143078+0000", 2024-09-13T09:12:20.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2884999999999999e-05 2024-09-13T09:12:20.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148244+0000", 2024-09-13T09:12:20.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051659089999999998 2024-09-13T09:12:20.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148285+0000", 2024-09-13T09:12:20.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1637999999999997e-05 2024-09-13T09:12:20.382 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.382 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.383 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.383 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:1068 3.c e27/21 3:313e6343:::rbd_data.1123ffac6bc2.000000000000000e:head v 27'17, mlcod=27'15)", 2024-09-13T09:12:20.383 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.143008+0000", 2024-09-13T09:12:20.383 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.138385798, 2024-09-13T09:12:20.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0053220990000000003, 2024-09-13T09:12:20.383 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.383 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.383 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.383 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143008+0000", 2024-09-13T09:12:20.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143008+0000", 2024-09-13T09:12:20.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143007+0000", 2024-09-13T09:12:20.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143022+0000", 2024-09-13T09:12:20.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4149e-05 2024-09-13T09:12:20.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143028+0000", 2024-09-13T09:12:20.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8909999999999998e-06 2024-09-13T09:12:20.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143033+0000", 2024-09-13T09:12:20.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.967e-06 2024-09-13T09:12:20.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143077+0000", 2024-09-13T09:12:20.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3307999999999999e-05 2024-09-13T09:12:20.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143095+0000", 2024-09-13T09:12:20.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8652999999999999e-05 2024-09-13T09:12:20.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148283+0000", 2024-09-13T09:12:20.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051879930000000001 2024-09-13T09:12:20.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148330+0000", 2024-09-13T09:12:20.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6511999999999998e-05 2024-09-13T09:12:20.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.387 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.387 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.387 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:20.390 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:20.390 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:20.387+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:20.390 INFO:teuthology.orchestra.run.smithi157.stdout:450 write 0x103ba00 thru 0x1048881 (0xce82 bytes) 2024-09-13T09:12:20.391 INFO:teuthology.orchestra.run.smithi157.stdout:451 write 0xec57200 thru 0xec57aba (0x8bb bytes) 2024-09-13T09:12:20.391 INFO:teuthology.orchestra.run.smithi157.stdout:452 punch from 0xa2d7c00 to 0xa2dba00, (0x3e00 bytes) 2024-09-13T09:12:20.391 INFO:teuthology.orchestra.run.smithi157.stdout:453 read 0x71ed600 thru 0x71f1374 (0x3d75 bytes) 2024-09-13T09:12:20.391 INFO:teuthology.orchestra.run.smithi157.stdout:457 trunc from 0xec57c00 to 0x7459e00 2024-09-13T09:12:20.391 INFO:teuthology.orchestra.run.smithi157.stdout:459 punch from 0x6190400 to 0x619c800, (0xc400 bytes) 2024-09-13T09:12:20.391 INFO:teuthology.orchestra.run.smithi157.stdout:460 write 0xaa6ce00 thru 0xaa7184f (0x4a50 bytes) 2024-09-13T09:12:20.391 INFO:teuthology.orchestra.run.smithi157.stdout:462 read 0x32dc600 thru 0x32e3352 (0x6d53 bytes) 2024-09-13T09:12:20.391 INFO:teuthology.orchestra.run.smithi157.stdout:465 trunc from 0xaa71a00 to 0xb71c800 2024-09-13T09:12:20.391 INFO:teuthology.orchestra.run.smithi157.stdout:470 punch from 0xacf6e00 to 0xacf9200, (0x2400 bytes) 2024-09-13T09:12:20.391 INFO:teuthology.orchestra.run.smithi157.stdout:473 trunc from 0xb71c800 to 0x82e2600 2024-09-13T09:12:20.415 INFO:teuthology.orchestra.run.smithi157.stdout:474 write 0x9085200 thru 0x9090ddc (0xbbdd bytes) 2024-09-13T09:12:20.415 INFO:teuthology.orchestra.run.smithi157.stdout:476 read 0x76b200 thru 0x77af81 (0xfd82 bytes) 2024-09-13T09:12:20.416 INFO:teuthology.orchestra.run.smithi157.stdout:487 write 0xdbd800 thru 0xdcbfee (0xe7ef bytes) 2024-09-13T09:12:20.417 INFO:teuthology.orchestra.run.smithi157.stdout:488 write 0x6c60800 thru 0x6c62d0f (0x2510 bytes) 2024-09-13T09:12:20.417 INFO:teuthology.orchestra.run.smithi157.stdout:491 clone 2 order 24 su 524288 sc 10 2024-09-13T09:12:20.490 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:20.488+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:20.536 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:20.536 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:20.536 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:20.536 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:20.545 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:20.590 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:20.589+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:20.651 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:20.651 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:20.651 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:20.651 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:20.651 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:20.660 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:20.705 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:20.704+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:20.769 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:20.769 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:20.769 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:20.769 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:20.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.769 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:217 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.770 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.754821+0000", 2024-09-13T09:12:20.770 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.011494144, 2024-09-13T09:12:20.770 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010056801000000001, 2024-09-13T09:12:20.770 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.770 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.770 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.770 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.770 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.770 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 217 2024-09-13T09:12:20.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.770 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.770 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.770 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.754821+0000", 2024-09-13T09:12:20.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.754821+0000", 2024-09-13T09:12:20.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.754820+0000", 2024-09-13T09:12:20.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:20.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.754826+0000", 2024-09-13T09:12:20.772 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9819999999999996e-06 2024-09-13T09:12:20.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.772 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.772 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.754842+0000", 2024-09-13T09:12:20.772 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5209e-05 2024-09-13T09:12:20.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.772 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.772 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.754849+0000", 2024-09-13T09:12:20.772 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8809999999999996e-06 2024-09-13T09:12:20.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.773 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.773 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.757798+0000", 2024-09-13T09:12:20.773 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029486429999999999 2024-09-13T09:12:20.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.773 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.773 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.773 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.757940+0000", 2024-09-13T09:12:20.773 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014207799999999999 2024-09-13T09:12:20.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.773 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.773 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.773 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.758328+0000", 2024-09-13T09:12:20.773 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000387346 2024-09-13T09:12:20.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.774 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.774 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.760830+0000", 2024-09-13T09:12:20.774 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025024019999999999 2024-09-13T09:12:20.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.774 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.774 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.764802+0000", 2024-09-13T09:12:20.774 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0039724260000000003 2024-09-13T09:12:20.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.774 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.775 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.764842+0000", 2024-09-13T09:12:20.775 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9832000000000001e-05 2024-09-13T09:12:20.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.775 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.775 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.764878+0000", 2024-09-13T09:12:20.775 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6090000000000002e-05 2024-09-13T09:12:20.775 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.775 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.775 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.775 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:220 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.775 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.755617+0000", 2024-09-13T09:12:20.776 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.01069853, 2024-09-13T09:12:20.776 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.009312542, 2024-09-13T09:12:20.776 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.776 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.776 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.776 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.776 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.776 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 220 2024-09-13T09:12:20.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.776 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.776 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.776 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.755617+0000", 2024-09-13T09:12:20.777 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.777 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.777 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.755617+0000", 2024-09-13T09:12:20.777 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.777 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.777 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.755617+0000", 2024-09-13T09:12:20.777 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.778 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.778 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.755620+0000", 2024-09-13T09:12:20.778 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6519999999999999e-06 2024-09-13T09:12:20.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.778 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.778 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.755627+0000", 2024-09-13T09:12:20.778 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.878e-06 2024-09-13T09:12:20.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.778 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.778 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.755632+0000", 2024-09-13T09:12:20.778 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.091e-06 2024-09-13T09:12:20.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.758961+0000", 2024-09-13T09:12:20.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033289880000000002 2024-09-13T09:12:20.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.759036+0000", 2024-09-13T09:12:20.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4805e-05 2024-09-13T09:12:20.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.759286+0000", 2024-09-13T09:12:20.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00025035299999999998 2024-09-13T09:12:20.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.780 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.780 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.764753+0000", 2024-09-13T09:12:20.780 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.005466243 2024-09-13T09:12:20.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.780 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.780 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.764885+0000", 2024-09-13T09:12:20.780 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013284599999999999 2024-09-13T09:12:20.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.780 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.781 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.764910+0000", 2024-09-13T09:12:20.781 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4295999999999999e-05 2024-09-13T09:12:20.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.781 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.781 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.764930+0000", 2024-09-13T09:12:20.781 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9916e-05 2024-09-13T09:12:20.781 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.781 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.781 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.781 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:224 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.781 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.756626+0000", 2024-09-13T09:12:20.782 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.0096897039999999, 2024-09-13T09:12:20.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0083762669999999997, 2024-09-13T09:12:20.782 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.782 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.782 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.782 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.782 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.782 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 224 2024-09-13T09:12:20.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.782 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.782 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.782 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.756626+0000", 2024-09-13T09:12:20.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.756626+0000", 2024-09-13T09:12:20.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.756625+0000", 2024-09-13T09:12:20.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:20.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.784 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.784 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.756631+0000", 2024-09-13T09:12:20.784 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9150000000000001e-06 2024-09-13T09:12:20.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.784 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.784 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.756643+0000", 2024-09-13T09:12:20.784 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2152e-05 2024-09-13T09:12:20.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.784 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.784 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.756651+0000", 2024-09-13T09:12:20.784 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:12:20.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.785 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.785 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.759652+0000", 2024-09-13T09:12:20.785 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0030012070000000001 2024-09-13T09:12:20.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.785 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.785 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.759737+0000", 2024-09-13T09:12:20.785 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4555000000000006e-05 2024-09-13T09:12:20.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.785 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.785 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.760107+0000", 2024-09-13T09:12:20.786 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00037027400000000003 2024-09-13T09:12:20.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.786 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.786 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.764797+0000", 2024-09-13T09:12:20.786 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046902769999999996 2024-09-13T09:12:20.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.788 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.788 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.764940+0000", 2024-09-13T09:12:20.788 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000142998 2024-09-13T09:12:20.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.788 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.789 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.764965+0000", 2024-09-13T09:12:20.789 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4661e-05 2024-09-13T09:12:20.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.789 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.789 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.765002+0000", 2024-09-13T09:12:20.789 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7338000000000001e-05 2024-09-13T09:12:20.789 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.789 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.789 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.789 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:245 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.790 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.761049+0000", 2024-09-13T09:12:20.790 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.0052663470000001, 2024-09-13T09:12:20.790 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0090658449999999995, 2024-09-13T09:12:20.790 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.790 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.790 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.790 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.790 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.790 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 245 2024-09-13T09:12:20.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.790 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.791 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761049+0000", 2024-09-13T09:12:20.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.791 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761049+0000", 2024-09-13T09:12:20.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.791 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761049+0000", 2024-09-13T09:12:20.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.792 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.792 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761052+0000", 2024-09-13T09:12:20.792 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6900000000000001e-06 2024-09-13T09:12:20.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.792 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.792 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761058+0000", 2024-09-13T09:12:20.792 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0889999999999996e-06 2024-09-13T09:12:20.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.792 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.792 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761062+0000", 2024-09-13T09:12:20.793 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7949999999999998e-06 2024-09-13T09:12:20.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.793 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.793 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.765049+0000", 2024-09-13T09:12:20.793 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0039861929999999999 2024-09-13T09:12:20.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.793 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.793 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.765164+0000", 2024-09-13T09:12:20.793 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000115579 2024-09-13T09:12:20.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.794 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.794 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.765536+0000", 2024-09-13T09:12:20.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00037136299999999999 2024-09-13T09:12:20.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.794 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.794 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.766987+0000", 2024-09-13T09:12:20.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001450925 2024-09-13T09:12:20.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.794 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.794 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770050+0000", 2024-09-13T09:12:20.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0030629839999999999 2024-09-13T09:12:20.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770079+0000", 2024-09-13T09:12:20.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9207999999999999e-05 2024-09-13T09:12:20.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770115+0000", 2024-09-13T09:12:20.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6390000000000002e-05 2024-09-13T09:12:20.795 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.795 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.795 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.796 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:246 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.796 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.761067+0000", 2024-09-13T09:12:20.796 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.005248921, 2024-09-13T09:12:20.796 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0091784280000000006, 2024-09-13T09:12:20.796 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.796 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.796 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.796 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.796 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.796 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 246 2024-09-13T09:12:20.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.796 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.797 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761067+0000", 2024-09-13T09:12:20.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.797 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761067+0000", 2024-09-13T09:12:20.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.797 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761066+0000", 2024-09-13T09:12:20.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.798 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.798 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761069+0000", 2024-09-13T09:12:20.798 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3740000000000001e-06 2024-09-13T09:12:20.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.798 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.798 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761074+0000", 2024-09-13T09:12:20.798 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.485e-06 2024-09-13T09:12:20.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.798 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.799 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761079+0000", 2024-09-13T09:12:20.799 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7620000000000003e-06 2024-09-13T09:12:20.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.799 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.799 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.765888+0000", 2024-09-13T09:12:20.799 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048087549999999996 2024-09-13T09:12:20.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.799 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.799 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.765963+0000", 2024-09-13T09:12:20.799 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5093999999999996e-05 2024-09-13T09:12:20.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.800 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.800 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.766365+0000", 2024-09-13T09:12:20.800 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00040246200000000002 2024-09-13T09:12:20.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.800 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.800 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.769146+0000", 2024-09-13T09:12:20.800 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027811160000000001 2024-09-13T09:12:20.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.801 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770157+0000", 2024-09-13T09:12:20.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001010719 2024-09-13T09:12:20.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.801 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770206+0000", 2024-09-13T09:12:20.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:12:20.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.801 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770245+0000", 2024-09-13T09:12:20.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9153000000000003e-05 2024-09-13T09:12:20.802 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.802 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.802 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.802 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.802 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:247 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.802 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.761083+0000", 2024-09-13T09:12:20.802 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.005232438, 2024-09-13T09:12:20.802 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0092823990000000002, 2024-09-13T09:12:20.802 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.802 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.802 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.802 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.803 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.803 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 247 2024-09-13T09:12:20.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.803 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761083+0000", 2024-09-13T09:12:20.803 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761083+0000", 2024-09-13T09:12:20.803 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.804 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761083+0000", 2024-09-13T09:12:20.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.804 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761085+0000", 2024-09-13T09:12:20.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.508e-06 2024-09-13T09:12:20.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.805 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761095+0000", 2024-09-13T09:12:20.805 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8339999999999994e-06 2024-09-13T09:12:20.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.805 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.805 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761101+0000", 2024-09-13T09:12:20.805 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3360000000000002e-06 2024-09-13T09:12:20.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.805 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.805 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.767371+0000", 2024-09-13T09:12:20.806 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0062708089999999996 2024-09-13T09:12:20.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.806 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.806 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.806 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.767453+0000", 2024-09-13T09:12:20.806 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1172000000000001e-05 2024-09-13T09:12:20.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.806 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.806 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.806 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.767831+0000", 2024-09-13T09:12:20.806 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00037822099999999998 2024-09-13T09:12:20.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.806 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.807 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.807 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770124+0000", 2024-09-13T09:12:20.807 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022932569999999999 2024-09-13T09:12:20.807 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.807 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.807 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.807 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770293+0000", 2024-09-13T09:12:20.807 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016867299999999999 2024-09-13T09:12:20.807 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.807 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.807 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.807 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770327+0000", 2024-09-13T09:12:20.807 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4504e-05 2024-09-13T09:12:20.807 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.808 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.808 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.808 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770366+0000", 2024-09-13T09:12:20.808 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8420000000000001e-05 2024-09-13T09:12:20.808 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.808 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.808 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.808 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.808 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.808 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:248 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.808 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.761112+0000", 2024-09-13T09:12:20.808 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.005203259, 2024-09-13T09:12:20.808 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0093668889999999998, 2024-09-13T09:12:20.808 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.809 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.809 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.809 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.809 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.809 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 248 2024-09-13T09:12:20.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.809 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.809 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.809 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.809 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761112+0000", 2024-09-13T09:12:20.809 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.809 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.810 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.810 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761112+0000", 2024-09-13T09:12:20.810 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.810 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.810 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.810 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761112+0000", 2024-09-13T09:12:20.810 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.810 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.810 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.810 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761115+0000", 2024-09-13T09:12:20.810 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2550000000000002e-06 2024-09-13T09:12:20.811 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.811 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.811 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.811 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761123+0000", 2024-09-13T09:12:20.811 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2849999999999995e-06 2024-09-13T09:12:20.811 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.811 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.811 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.811 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761129+0000", 2024-09-13T09:12:20.811 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5169999999999999e-06 2024-09-13T09:12:20.811 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.811 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.811 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.811 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.768299+0000", 2024-09-13T09:12:20.812 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00717017 2024-09-13T09:12:20.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.812 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.812 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.768384+0000", 2024-09-13T09:12:20.812 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4812e-05 2024-09-13T09:12:20.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.812 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.812 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.768846+0000", 2024-09-13T09:12:20.812 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00046207900000000001 2024-09-13T09:12:20.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.812 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.813 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770406+0000", 2024-09-13T09:12:20.813 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015596760000000001 2024-09-13T09:12:20.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.813 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.813 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770420+0000", 2024-09-13T09:12:20.813 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4181e-05 2024-09-13T09:12:20.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.813 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.813 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770458+0000", 2024-09-13T09:12:20.813 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7846999999999999e-05 2024-09-13T09:12:20.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.814 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.814 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770479+0000", 2024-09-13T09:12:20.814 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1586999999999998e-05 2024-09-13T09:12:20.814 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.814 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.814 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.814 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:249 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.814 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.761134+0000", 2024-09-13T09:12:20.814 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.00518167, 2024-09-13T09:12:20.814 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0094378090000000001, 2024-09-13T09:12:20.814 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.814 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.815 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.815 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.815 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.815 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 249 2024-09-13T09:12:20.815 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.815 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.815 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.815 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.815 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761134+0000", 2024-09-13T09:12:20.815 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.815 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.815 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.816 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.816 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761134+0000", 2024-09-13T09:12:20.816 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.816 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.816 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.816 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761134+0000", 2024-09-13T09:12:20.816 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.816 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.816 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.816 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761136+0000", 2024-09-13T09:12:20.817 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5849999999999998e-06 2024-09-13T09:12:20.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.817 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.817 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761144+0000", 2024-09-13T09:12:20.817 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5569999999999998e-06 2024-09-13T09:12:20.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.817 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.817 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761150+0000", 2024-09-13T09:12:20.817 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7649999999999996e-06 2024-09-13T09:12:20.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.817 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.818 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.769175+0000", 2024-09-13T09:12:20.818 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.008024415 2024-09-13T09:12:20.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.818 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.818 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.769267+0000", 2024-09-13T09:12:20.818 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1902000000000002e-05 2024-09-13T09:12:20.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.818 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.818 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.769664+0000", 2024-09-13T09:12:20.818 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00039739899999999999 2024-09-13T09:12:20.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.818 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.819 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770488+0000", 2024-09-13T09:12:20.819 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00082345699999999999 2024-09-13T09:12:20.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.819 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.819 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770518+0000", 2024-09-13T09:12:20.819 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0576e-05 2024-09-13T09:12:20.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.819 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.819 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770553+0000", 2024-09-13T09:12:20.819 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4907999999999998e-05 2024-09-13T09:12:20.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.819 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.820 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.770572+0000", 2024-09-13T09:12:20.820 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8635999999999999e-05 2024-09-13T09:12:20.820 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.820 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.820 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.820 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.820 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:679 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.820 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.971666+0000", 2024-09-13T09:12:20.820 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.79465006100000002, 2024-09-13T09:12:20.820 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071801060000000003, 2024-09-13T09:12:20.820 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.820 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.821 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.821 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.821 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.821 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 679 2024-09-13T09:12:20.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.821 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.821 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.821 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971666+0000", 2024-09-13T09:12:20.821 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.821 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.822 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971666+0000", 2024-09-13T09:12:20.822 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.822 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.822 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.822 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971665+0000", 2024-09-13T09:12:20.822 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.822 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.822 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.822 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971670+0000", 2024-09-13T09:12:20.822 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9869999999999999e-06 2024-09-13T09:12:20.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.822 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.822 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.823 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971679+0000", 2024-09-13T09:12:20.823 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1980000000000007e-06 2024-09-13T09:12:20.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.823 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.823 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.823 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971685+0000", 2024-09-13T09:12:20.823 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8919999999999997e-06 2024-09-13T09:12:20.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.823 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.823 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.823 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.975976+0000", 2024-09-13T09:12:20.823 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042908979999999996 2024-09-13T09:12:20.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.824 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.824 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.976069+0000", 2024-09-13T09:12:20.824 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2437999999999998e-05 2024-09-13T09:12:20.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.824 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.824 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.976412+0000", 2024-09-13T09:12:20.824 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034306499999999998 2024-09-13T09:12:20.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.824 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.824 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.978748+0000", 2024-09-13T09:12:20.825 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023360569999999999 2024-09-13T09:12:20.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.825 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.825 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.978757+0000", 2024-09-13T09:12:20.825 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6220000000000004e-06 2024-09-13T09:12:20.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.825 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.825 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.978805+0000", 2024-09-13T09:12:20.825 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7589000000000001e-05 2024-09-13T09:12:20.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.825 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.826 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.978846+0000", 2024-09-13T09:12:20.826 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0828999999999998e-05 2024-09-13T09:12:20.826 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.826 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.826 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.826 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.826 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:680 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.826 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.971761+0000", 2024-09-13T09:12:20.826 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.79455478499999999, 2024-09-13T09:12:20.826 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0081350060000000002, 2024-09-13T09:12:20.826 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.826 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.826 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.827 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.827 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.827 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 680 2024-09-13T09:12:20.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.827 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.827 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.827 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.827 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971761+0000", 2024-09-13T09:12:20.827 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.827 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.827 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.827 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971761+0000", 2024-09-13T09:12:20.828 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.828 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.828 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971760+0000", 2024-09-13T09:12:20.828 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.828 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.828 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971767+0000", 2024-09-13T09:12:20.828 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7660000000000001e-06 2024-09-13T09:12:20.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.829 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.829 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971776+0000", 2024-09-13T09:12:20.829 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7809999999999994e-06 2024-09-13T09:12:20.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.829 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.829 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971781+0000", 2024-09-13T09:12:20.829 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7370000000000003e-06 2024-09-13T09:12:20.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.829 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.829 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.976912+0000", 2024-09-13T09:12:20.829 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051300920000000002 2024-09-13T09:12:20.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.830 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.830 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.977025+0000", 2024-09-13T09:12:20.830 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000113214 2024-09-13T09:12:20.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.830 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.830 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.977377+0000", 2024-09-13T09:12:20.830 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035218000000000001 2024-09-13T09:12:20.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.830 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.830 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.978910+0000", 2024-09-13T09:12:20.830 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015328480000000001 2024-09-13T09:12:20.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.831 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.831 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.831 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.979795+0000", 2024-09-13T09:12:20.831 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00088546199999999995 2024-09-13T09:12:20.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.831 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.831 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.831 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.979850+0000", 2024-09-13T09:12:20.831 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4557e-05 2024-09-13T09:12:20.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.831 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.831 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.831 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.979896+0000", 2024-09-13T09:12:20.832 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6035000000000002e-05 2024-09-13T09:12:20.832 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.832 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.832 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.832 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.832 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:681 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.832 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.971790+0000", 2024-09-13T09:12:20.832 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.79452557499999998, 2024-09-13T09:12:20.832 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0092003099999999997, 2024-09-13T09:12:20.832 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.832 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.833 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.833 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.833 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.833 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 681 2024-09-13T09:12:20.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.833 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.833 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.833 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.833 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971790+0000", 2024-09-13T09:12:20.833 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.833 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.833 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.834 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971790+0000", 2024-09-13T09:12:20.834 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.834 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.834 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.834 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971790+0000", 2024-09-13T09:12:20.834 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.834 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.834 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.834 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971793+0000", 2024-09-13T09:12:20.834 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4649999999999999e-06 2024-09-13T09:12:20.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.835 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.835 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971800+0000", 2024-09-13T09:12:20.835 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1099999999999997e-06 2024-09-13T09:12:20.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.835 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.835 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.971807+0000", 2024-09-13T09:12:20.835 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6359999999999997e-06 2024-09-13T09:12:20.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.836 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.836 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.977846+0000", 2024-09-13T09:12:20.836 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0060396770000000002 2024-09-13T09:12:20.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.836 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.836 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.836 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.977935+0000", 2024-09-13T09:12:20.836 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8537999999999998e-05 2024-09-13T09:12:20.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.836 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.836 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.836 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.978341+0000", 2024-09-13T09:12:20.837 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00040591800000000001 2024-09-13T09:12:20.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.837 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.837 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.837 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.979905+0000", 2024-09-13T09:12:20.837 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001564516 2024-09-13T09:12:20.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.837 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.837 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.837 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.980906+0000", 2024-09-13T09:12:20.837 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010001949999999999 2024-09-13T09:12:20.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.837 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.838 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.838 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.980954+0000", 2024-09-13T09:12:20.838 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8751000000000001e-05 2024-09-13T09:12:20.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.838 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.838 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.838 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.980990+0000", 2024-09-13T09:12:20.838 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6001000000000003e-05 2024-09-13T09:12:20.838 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.838 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.838 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.838 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.839 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:693 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.839 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.975971+0000", 2024-09-13T09:12:20.839 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.79034453400000004, 2024-09-13T09:12:20.839 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0077270489999999997, 2024-09-13T09:12:20.839 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.839 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.839 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.839 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.839 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.839 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 693 2024-09-13T09:12:20.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.839 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.839 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.840 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.840 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.975971+0000", 2024-09-13T09:12:20.840 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.840 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.840 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.840 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.975971+0000", 2024-09-13T09:12:20.840 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.840 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.840 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.840 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.975970+0000", 2024-09-13T09:12:20.841 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:20.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.841 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.841 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.841 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.975976+0000", 2024-09-13T09:12:20.841 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4489999999999996e-06 2024-09-13T09:12:20.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.841 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.841 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.841 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.975988+0000", 2024-09-13T09:12:20.841 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1595e-05 2024-09-13T09:12:20.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.841 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.842 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.842 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.975995+0000", 2024-09-13T09:12:20.842 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5129999999999999e-06 2024-09-13T09:12:20.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.842 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.842 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.842 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.978935+0000", 2024-09-13T09:12:20.842 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029395060000000002 2024-09-13T09:12:20.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.842 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.842 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.842 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.979012+0000", 2024-09-13T09:12:20.842 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6647e-05 2024-09-13T09:12:20.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.843 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.843 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.979407+0000", 2024-09-13T09:12:20.843 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00039534800000000001 2024-09-13T09:12:20.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.843 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.843 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.980833+0000", 2024-09-13T09:12:20.843 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0014259100000000001 2024-09-13T09:12:20.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.843 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.843 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.983612+0000", 2024-09-13T09:12:20.843 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002779288 2024-09-13T09:12:20.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.844 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.844 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.844 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.983660+0000", 2024-09-13T09:12:20.844 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7388000000000001e-05 2024-09-13T09:12:20.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.844 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.844 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.844 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.983698+0000", 2024-09-13T09:12:20.844 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8621999999999997e-05 2024-09-13T09:12:20.844 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.844 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.844 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.845 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.845 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:911 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.845 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.077485+0000", 2024-09-13T09:12:20.845 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.68883074300000002, 2024-09-13T09:12:20.845 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0069909940000000004, 2024-09-13T09:12:20.845 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.845 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.845 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.845 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.845 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.845 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 911 2024-09-13T09:12:20.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.846 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.846 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.846 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.846 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.077485+0000", 2024-09-13T09:12:20.846 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.846 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.846 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.846 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.077485+0000", 2024-09-13T09:12:20.846 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.846 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.846 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.847 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.077485+0000", 2024-09-13T09:12:20.847 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.847 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.847 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.847 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.077489+0000", 2024-09-13T09:12:20.847 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8010000000000003e-06 2024-09-13T09:12:20.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.847 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.847 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.847 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.077495+0000", 2024-09-13T09:12:20.847 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8390000000000003e-06 2024-09-13T09:12:20.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.848 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.848 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.077501+0000", 2024-09-13T09:12:20.848 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1020000000000002e-06 2024-09-13T09:12:20.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.848 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.848 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.080099+0000", 2024-09-13T09:12:20.848 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025976240000000002 2024-09-13T09:12:20.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.848 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.848 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.080190+0000", 2024-09-13T09:12:20.849 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0771000000000002e-05 2024-09-13T09:12:20.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.849 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.849 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.849 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.080459+0000", 2024-09-13T09:12:20.849 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00026980100000000001 2024-09-13T09:12:20.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.849 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.849 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.849 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.081996+0000", 2024-09-13T09:12:20.849 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015369229999999999 2024-09-13T09:12:20.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.849 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.850 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.850 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.084408+0000", 2024-09-13T09:12:20.850 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0024116839999999999 2024-09-13T09:12:20.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.850 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.850 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.084440+0000", 2024-09-13T09:12:20.850 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2175999999999998e-05 2024-09-13T09:12:20.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.850 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.850 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.084476+0000", 2024-09-13T09:12:20.850 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5608000000000002e-05 2024-09-13T09:12:20.850 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.851 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.851 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.851 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.851 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.851 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:912 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.851 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.077506+0000", 2024-09-13T09:12:20.851 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.68880923900000002, 2024-09-13T09:12:20.851 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007045234, 2024-09-13T09:12:20.851 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.851 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.851 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.851 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.851 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.852 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 912 2024-09-13T09:12:20.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.852 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.852 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.852 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.852 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.077506+0000", 2024-09-13T09:12:20.852 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.852 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.852 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.852 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.077506+0000", 2024-09-13T09:12:20.852 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.852 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.853 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.853 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.077506+0000", 2024-09-13T09:12:20.853 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.853 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.853 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.077509+0000", 2024-09-13T09:12:20.853 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2940000000000001e-06 2024-09-13T09:12:20.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.853 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.853 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.077516+0000", 2024-09-13T09:12:20.853 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8059999999999997e-06 2024-09-13T09:12:20.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.077521+0000", 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3800000000000002e-06 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.080722+0000", 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0032007699999999999 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.080791+0000", 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8387999999999996e-05 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.855 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.855 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.081024+0000", 2024-09-13T09:12:20.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000233546 2024-09-13T09:12:20.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.855 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.855 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.082071+0000", 2024-09-13T09:12:20.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010467639999999999 2024-09-13T09:12:20.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.855 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.855 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.084504+0000", 2024-09-13T09:12:20.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0024334169999999998 2024-09-13T09:12:20.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.856 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.856 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.856 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.084531+0000", 2024-09-13T09:12:20.856 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6239000000000001e-05 2024-09-13T09:12:20.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.856 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.856 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.856 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.084552+0000", 2024-09-13T09:12:20.856 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1009000000000002e-05 2024-09-13T09:12:20.856 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.856 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.856 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.857 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:1038 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.857 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.134588+0000", 2024-09-13T09:12:20.857 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.63172726300000004, 2024-09-13T09:12:20.857 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0078177660000000003, 2024-09-13T09:12:20.857 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.857 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.857 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.857 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.857 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.857 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1038 2024-09-13T09:12:20.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.857 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.858 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134588+0000", 2024-09-13T09:12:20.858 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.858 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134588+0000", 2024-09-13T09:12:20.858 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.858 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134588+0000", 2024-09-13T09:12:20.859 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.859 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.859 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134591+0000", 2024-09-13T09:12:20.859 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4939999999999999e-06 2024-09-13T09:12:20.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.859 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.859 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134601+0000", 2024-09-13T09:12:20.859 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5650000000000007e-06 2024-09-13T09:12:20.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.859 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.859 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134614+0000", 2024-09-13T09:12:20.860 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3276e-05 2024-09-13T09:12:20.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.860 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.860 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.860 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.137367+0000", 2024-09-13T09:12:20.860 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027524699999999999 2024-09-13T09:12:20.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.860 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.860 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.860 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.137449+0000", 2024-09-13T09:12:20.860 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2112000000000003e-05 2024-09-13T09:12:20.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.137751+0000", 2024-09-13T09:12:20.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00030206200000000002 2024-09-13T09:12:20.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140406+0000", 2024-09-13T09:12:20.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026551750000000001 2024-09-13T09:12:20.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142342+0000", 2024-09-13T09:12:20.862 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00193647 2024-09-13T09:12:20.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142370+0000", 2024-09-13T09:12:20.862 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7948000000000001e-05 2024-09-13T09:12:20.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142406+0000", 2024-09-13T09:12:20.862 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5719999999999997e-05 2024-09-13T09:12:20.862 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.862 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.862 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.863 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.863 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:1039 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.863 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.134628+0000", 2024-09-13T09:12:20.863 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.63168740499999998, 2024-09-13T09:12:20.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0078825490000000008, 2024-09-13T09:12:20.863 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.863 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.863 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.863 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.863 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.863 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1039 2024-09-13T09:12:20.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.863 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.864 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.864 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134628+0000", 2024-09-13T09:12:20.864 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.864 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.864 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134628+0000", 2024-09-13T09:12:20.864 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.864 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.864 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134628+0000", 2024-09-13T09:12:20.864 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.865 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.865 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134632+0000", 2024-09-13T09:12:20.865 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1099999999999996e-06 2024-09-13T09:12:20.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.865 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.865 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134639+0000", 2024-09-13T09:12:20.865 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8210000000000002e-06 2024-09-13T09:12:20.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.865 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.865 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134643+0000", 2024-09-13T09:12:20.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6169999999999998e-06 2024-09-13T09:12:20.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.866 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.138092+0000", 2024-09-13T09:12:20.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034487060000000002 2024-09-13T09:12:20.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.866 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.138143+0000", 2024-09-13T09:12:20.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1674e-05 2024-09-13T09:12:20.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.138363+0000", 2024-09-13T09:12:20.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00021914100000000001 2024-09-13T09:12:20.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140410+0000", 2024-09-13T09:12:20.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020475699999999999 2024-09-13T09:12:20.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142437+0000", 2024-09-13T09:12:20.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020263400000000002 2024-09-13T09:12:20.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.868 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.868 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.868 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142472+0000", 2024-09-13T09:12:20.868 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5006000000000003e-05 2024-09-13T09:12:20.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.868 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.868 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.868 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142511+0000", 2024-09-13T09:12:20.868 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9263e-05 2024-09-13T09:12:20.868 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.868 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.868 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.869 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.869 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:1040 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.869 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.134710+0000", 2024-09-13T09:12:20.869 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.63160533500000005, 2024-09-13T09:12:20.869 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0078853889999999996, 2024-09-13T09:12:20.869 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.869 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.869 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.869 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.869 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.869 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1040 2024-09-13T09:12:20.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.869 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134710+0000", 2024-09-13T09:12:20.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134710+0000", 2024-09-13T09:12:20.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134710+0000", 2024-09-13T09:12:20.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.871 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134713+0000", 2024-09-13T09:12:20.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0809999999999998e-06 2024-09-13T09:12:20.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.871 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134719+0000", 2024-09-13T09:12:20.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2419999999999997e-06 2024-09-13T09:12:20.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.871 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134723+0000", 2024-09-13T09:12:20.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7330000000000001e-06 2024-09-13T09:12:20.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.138608+0000", 2024-09-13T09:12:20.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0038850930000000001 2024-09-13T09:12:20.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.138657+0000", 2024-09-13T09:12:20.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8736000000000002e-05 2024-09-13T09:12:20.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.138851+0000", 2024-09-13T09:12:20.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00019411 2024-09-13T09:12:20.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.873 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.873 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140424+0000", 2024-09-13T09:12:20.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001573208 2024-09-13T09:12:20.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.873 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.873 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142539+0000", 2024-09-13T09:12:20.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002114618 2024-09-13T09:12:20.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.873 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142563+0000", 2024-09-13T09:12:20.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4247e-05 2024-09-13T09:12:20.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142596+0000", 2024-09-13T09:12:20.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2811999999999997e-05 2024-09-13T09:12:20.874 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.874 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.874 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.874 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:1041 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.874 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.134735+0000", 2024-09-13T09:12:20.875 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.63158087399999996, 2024-09-13T09:12:20.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010418536000000001, 2024-09-13T09:12:20.875 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.875 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.875 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.875 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.875 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.875 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1041 2024-09-13T09:12:20.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.875 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.875 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134735+0000", 2024-09-13T09:12:20.876 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.876 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.876 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134735+0000", 2024-09-13T09:12:20.876 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.876 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.876 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134734+0000", 2024-09-13T09:12:20.876 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.876 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.876 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134737+0000", 2024-09-13T09:12:20.877 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.514e-06 2024-09-13T09:12:20.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.877 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.877 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134742+0000", 2024-09-13T09:12:20.877 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4739999999999996e-06 2024-09-13T09:12:20.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.877 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.877 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.134746+0000", 2024-09-13T09:12:20.877 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0910000000000003e-06 2024-09-13T09:12:20.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.877 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.139120+0000", 2024-09-13T09:12:20.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0043730640000000003 2024-09-13T09:12:20.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.878 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.139167+0000", 2024-09-13T09:12:20.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7315999999999999e-05 2024-09-13T09:12:20.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.878 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.139361+0000", 2024-09-13T09:12:20.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000194463 2024-09-13T09:12:20.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.878 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140434+0000", 2024-09-13T09:12:20.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010729750000000001 2024-09-13T09:12:20.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145106+0000", 2024-09-13T09:12:20.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046721100000000002 2024-09-13T09:12:20.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145133+0000", 2024-09-13T09:12:20.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6398999999999999e-05 2024-09-13T09:12:20.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145153+0000", 2024-09-13T09:12:20.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0497e-05 2024-09-13T09:12:20.880 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.880 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.880 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.880 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:1081 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:20.880 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.148294+0000", 2024-09-13T09:12:20.880 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.61802176099999995, 2024-09-13T09:12:20.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0079458270000000008, 2024-09-13T09:12:20.880 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.880 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.880 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.880 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.881 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.881 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1081 2024-09-13T09:12:20.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.881 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.881 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.881 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148294+0000", 2024-09-13T09:12:20.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.881 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.881 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148294+0000", 2024-09-13T09:12:20.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148293+0000", 2024-09-13T09:12:20.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148296+0000", 2024-09-13T09:12:20.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1140000000000002e-06 2024-09-13T09:12:20.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148309+0000", 2024-09-13T09:12:20.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2896e-05 2024-09-13T09:12:20.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148318+0000", 2024-09-13T09:12:20.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1300000000000001e-06 2024-09-13T09:12:20.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.152294+0000", 2024-09-13T09:12:20.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0039761220000000003 2024-09-13T09:12:20.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.152381+0000", 2024-09-13T09:12:20.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7839999999999999e-05 2024-09-13T09:12:20.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.152771+0000", 2024-09-13T09:12:20.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00038955800000000003 2024-09-13T09:12:20.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.155187+0000", 2024-09-13T09:12:20.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0024164080000000001 2024-09-13T09:12:20.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.156159+0000", 2024-09-13T09:12:20.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00097118900000000001 2024-09-13T09:12:20.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.156204+0000", 2024-09-13T09:12:20.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5241999999999998e-05 2024-09-13T09:12:20.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.156240+0000", 2024-09-13T09:12:20.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5818999999999997e-05 2024-09-13T09:12:20.885 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.885 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.885 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.886 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:1659 3.2 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:12:20.886 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.417311+0000", 2024-09-13T09:12:20.886 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34900462100000001, 2024-09-13T09:12:20.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073749760000000001, 2024-09-13T09:12:20.886 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:20.886 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:20.886 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:20.886 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:20.886 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:20.886 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1659 2024-09-13T09:12:20.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.886 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:20.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:20.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.417311+0000", 2024-09-13T09:12:20.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:20.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.417311+0000", 2024-09-13T09:12:20.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:20.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:20.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.417311+0000", 2024-09-13T09:12:20.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:20.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:20.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.417313+0000", 2024-09-13T09:12:20.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2079999999999999e-06 2024-09-13T09:12:20.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:20.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.417318+0000", 2024-09-13T09:12:20.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6979999999999997e-06 2024-09-13T09:12:20.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:20.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.417320+0000", 2024-09-13T09:12:20.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8940000000000002e-06 2024-09-13T09:12:20.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:20.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.417989+0000", 2024-09-13T09:12:20.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00066836700000000001 2024-09-13T09:12:20.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:20.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.418072+0000", 2024-09-13T09:12:20.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3537999999999999e-05 2024-09-13T09:12:20.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:20.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.418339+0000", 2024-09-13T09:12:20.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00026675400000000001 2024-09-13T09:12:20.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:20.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.420985+0000", 2024-09-13T09:12:20.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026456320000000002 2024-09-13T09:12:20.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:20.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.424558+0000", 2024-09-13T09:12:20.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0035727509999999999 2024-09-13T09:12:20.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:20.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.424648+0000", 2024-09-13T09:12:20.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0079000000000003e-05 2024-09-13T09:12:20.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:20.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:20.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:20.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.424686+0000", 2024-09-13T09:12:20.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8408000000000002e-05 2024-09-13T09:12:20.891 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.891 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.891 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.891 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:20.891 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:20.891 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:20.893 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:20.894 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:20.892+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:20.994 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:20.993+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:21.004 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:12:21.041 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:21.041 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:21.050 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:21.127 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:21.127 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:21.126+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:21.143 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:21.144 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:21.160 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:21.160 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:21.177 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:21.177 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:21.197 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:21.198 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:21.198 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:21.197+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:21.199 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:21.199 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:21.208 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:21.211 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:21.211 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:21.299 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:21.298+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:21.317 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:21.318 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:21.326 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:21.401 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:12:21.402 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:21.400+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:21.445 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:21.445 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:21.454 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:21.468 INFO:teuthology.orchestra.run.smithi157.stdout:leaving image image_client.0-clone1 intact 2024-09-13T09:12:21.502 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:21.501+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:21.556 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:21.557 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:21.565 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:21.606 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:21.605+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:21.668 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:21.668 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:21.677 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:21.707 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:21.707+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:21.781 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:21.781 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:21.790 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:21.809 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-13T09:12:21.810 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:21.809+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:21.896 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:21.896 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:21.906 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:21.950 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:21.949+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:22.011 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:22.009+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:22.020 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:22.020 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:22.030 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:22.113 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:22.112+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:22.173 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:22.173 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:22.181 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:22.212 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:22.211+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:22.220 INFO:teuthology.orchestra.run.smithi157.stdout:492 write 0xaae3e00 thru 0xaaec577 (0x8778 bytes) 2024-09-13T09:12:22.226 INFO:teuthology.orchestra.run.smithi157.stdout:494 write 0x554e600 thru 0x555abf3 (0xc5f4 bytes) 2024-09-13T09:12:22.226 INFO:teuthology.orchestra.run.smithi157.stdout:496 punch from 0x8a82a00 to 0x8a87a00, (0x5000 bytes) 2024-09-13T09:12:22.254 INFO:teuthology.orchestra.run.smithi157.stdout:498 write 0xbb1ea00 thru 0xbb2a6d7 (0xbcd8 bytes) 2024-09-13T09:12:22.274 INFO:teuthology.orchestra.run.smithi157.stdout:499 trunc from 0xbb2a800 to 0x37fe00 2024-09-13T09:12:22.313 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:22.312+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:22.332 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:22.332 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:22.334 INFO:teuthology.orchestra.run.smithi157.stdout:501 read 0x19e400 thru 0x1aad47 (0xc948 bytes) 2024-09-13T09:12:22.338 INFO:teuthology.orchestra.run.smithi157.stdout:502 read 0x24a400 thru 0x259074 (0xec75 bytes) 2024-09-13T09:12:22.341 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:22.364 INFO:teuthology.orchestra.run.smithi157.stdout:503 write 0xe5bd200 thru 0xe5c364b (0x644c bytes) 2024-09-13T09:12:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:504 write 0x4b41e00 thru 0x4b48836 (0x6a37 bytes) 2024-09-13T09:12:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:506 write 0xc088200 thru 0xc08d149 (0x4f4a bytes) 2024-09-13T09:12:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:507 punch from 0xdf3da00 to 0xdf4be00, (0xe400 bytes) 2024-09-13T09:12:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:508 read 0x27dc800 thru 0x27eb46a (0xec6b bytes) 2024-09-13T09:12:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:509 punch from 0x8d36400 to 0x8d3ca00, (0x6600 bytes) 2024-09-13T09:12:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:510 punch from 0x7d1a400 to 0x7d1e800, (0x4400 bytes) 2024-09-13T09:12:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:512 punch from 0x36d0800 to 0x36d0a00, (0x200 bytes) 2024-09-13T09:12:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:514 punch from 0x2b95600 to 0x2b99000, (0x3a00 bytes) 2024-09-13T09:12:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:515 punch from 0xddf6600 to 0xde00a00, (0xa400 bytes) 2024-09-13T09:12:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:516 write 0x92cec00 thru 0x92d7540 (0x8941 bytes) 2024-09-13T09:12:22.376 INFO:teuthology.orchestra.run.smithi157.stdout:518 trunc from 0xe5c3800 to 0x8b73600 2024-09-13T09:12:22.396 INFO:teuthology.orchestra.run.smithi157.stdout:521 punch from 0x3814c00 to 0x3817c00, (0x3000 bytes) 2024-09-13T09:12:22.403 INFO:teuthology.orchestra.run.smithi157.stdout:526 punch from 0x6961c00 to 0x696e000, (0xc400 bytes) 2024-09-13T09:12:22.403 INFO:teuthology.orchestra.run.smithi157.stdout:527 write 0x80ccc00 thru 0x80cf316 (0x2717 bytes) 2024-09-13T09:12:22.413 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-09-13T09:12:22.413 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:22.412+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:22.425 INFO:teuthology.orchestra.run.smithi157.stdout:529 write 0xa20d400 thru 0xa21b3d5 (0xdfd6 bytes) 2024-09-13T09:12:22.431 INFO:teuthology.orchestra.run.smithi157.stdout:531 read 0x8b5ac00 thru 0x8b63a43 (0x8e44 bytes) 2024-09-13T09:12:22.432 INFO:teuthology.orchestra.run.smithi157.stdout:532 read 0x52c5c00 thru 0x52c7f0f (0x2310 bytes) 2024-09-13T09:12:22.432 INFO:teuthology.orchestra.run.smithi157.stdout:539 punch from 0x2bcc600 to 0x2bd8000, (0xba00 bytes) 2024-09-13T09:12:22.434 INFO:teuthology.orchestra.run.smithi157.stdout:541 trunc from 0xa21b400 to 0xbd24800 2024-09-13T09:12:22.445 INFO:teuthology.orchestra.run.smithi157.stdout:545 read 0x6c9b400 thru 0x6c9d7d9 (0x23da bytes) 2024-09-13T09:12:22.449 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:22.450 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:22.453 INFO:teuthology.orchestra.run.smithi157.stdout:548 read 0x472b600 thru 0x473a1b8 (0xebb9 bytes) 2024-09-13T09:12:22.453 INFO:teuthology.orchestra.run.smithi157.stdout:549 clone 3 order 24 su 131072 sc 11 2024-09-13T09:12:22.459 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:22.515 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:22.513+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:22.575 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:22.575 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:22.583 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:22.616 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:22.615+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:22.689 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:22.690 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:22.698 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:22.742 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:22.741+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:22.804 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:22.805 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:22.813 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:22.855 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:22.854+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:22.918 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:22.917+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:22.961 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:22.961 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:22.970 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:23.055 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:23.054+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:23.116 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:23.116 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:23.118 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:23.117+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:23.125 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:23.220 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:23.218+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:23.223 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:12:23.272 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:23.272 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:23.281 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:23.321 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:23.320+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:23.348 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:23.348 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:23.365 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:23.365 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:23.381 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:23.381 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:23.391 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:23.391 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:23.391 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:23.391 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:23.398 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:23.398 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:23.400 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:23.418 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:23.418 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:23.421 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:23.419+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:23.437 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:23.438 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:23.498 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone2 from 0xbd24800 (overlap 0x37fe00) to 0x0 2024-09-13T09:12:23.518 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:23.518 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:23.518 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:23.519 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:23.519 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:23.522 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:23.521+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:23.528 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:23.622 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:23.621+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:23.679 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:23.679 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:23.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:23.680 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:23.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.680 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4400.0:1493 3.1e 3:7fc6f2cf:::rbd_data.11238d3f93be.0000000000000029:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:23.680 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.136878+0000", 2024-09-13T09:12:23.680 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.5403391019999999, 2024-09-13T09:12:23.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003791745, 2024-09-13T09:12:23.680 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.680 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:23.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4400", 2024-09-13T09:12:23.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:23.680 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1493 2024-09-13T09:12:23.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.681 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.136878+0000", 2024-09-13T09:12:23.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.136878+0000", 2024-09-13T09:12:23.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.136876+0000", 2024-09-13T09:12:23.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:23.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.136882+0000", 2024-09-13T09:12:23.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:12:23.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.136892+0000", 2024-09-13T09:12:23.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3610000000000007e-06 2024-09-13T09:12:23.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.136897+0000", 2024-09-13T09:12:23.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9189999999999998e-06 2024-09-13T09:12:23.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.136941+0000", 2024-09-13T09:12:23.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4613999999999997e-05 2024-09-13T09:12:23.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.137028+0000", 2024-09-13T09:12:23.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6693999999999994e-05 2024-09-13T09:12:23.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:23.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.137154+0000", 2024-09-13T09:12:23.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012551899999999999 2024-09-13T09:12:23.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:23.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.137818+0000", 2024-09-13T09:12:23.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000664765 2024-09-13T09:12:23.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:23.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.140610+0000", 2024-09-13T09:12:23.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002791548 2024-09-13T09:12:23.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.140648+0000", 2024-09-13T09:12:23.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7932000000000002e-05 2024-09-13T09:12:23.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.140669+0000", 2024-09-13T09:12:23.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1642999999999999e-05 2024-09-13T09:12:23.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.686 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.686 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:226 3.1e 3:7da6a7d6:::rbd_data.1123ffac6bc2.00000000000000cb:head [set-alloc-hint object_size 1048576 write_size 1048576,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:23.686 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.757050+0000", 2024-09-13T09:12:23.686 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.920166944, 2024-09-13T09:12:23.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0040435899999999997, 2024-09-13T09:12:23.686 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.686 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.686 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:23.686 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:23.686 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:23.687 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 226 2024-09-13T09:12:23.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.687 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.757050+0000", 2024-09-13T09:12:23.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.757050+0000", 2024-09-13T09:12:23.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.757049+0000", 2024-09-13T09:12:23.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:23.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.757145+0000", 2024-09-13T09:12:23.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6051999999999994e-05 2024-09-13T09:12:23.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.757159+0000", 2024-09-13T09:12:23.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4687e-05 2024-09-13T09:12:23.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.757165+0000", 2024-09-13T09:12:23.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2580000000000003e-06 2024-09-13T09:12:23.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.757202+0000", 2024-09-13T09:12:23.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7725999999999998e-05 2024-09-13T09:12:23.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.757301+0000", 2024-09-13T09:12:23.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8944999999999995e-05 2024-09-13T09:12:23.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:23.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.757410+0000", 2024-09-13T09:12:23.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010895600000000001 2024-09-13T09:12:23.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:23.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.758245+0000", 2024-09-13T09:12:23.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00083496799999999995 2024-09-13T09:12:23.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:23.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761034+0000", 2024-09-13T09:12:23.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027885900000000001 2024-09-13T09:12:23.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761071+0000", 2024-09-13T09:12:23.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7753999999999998e-05 2024-09-13T09:12:23.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.761093+0000", 2024-09-13T09:12:23.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1963999999999998e-05 2024-09-13T09:12:23.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.692 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.692 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:285 3.7 3:e76cef2e:::rbd_data.1123ffac6bc2.00000000000000c2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:23.692 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.778209+0000", 2024-09-13T09:12:23.692 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.899007385, 2024-09-13T09:12:23.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051374489999999997, 2024-09-13T09:12:23.692 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.692 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.692 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:23.692 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:23.693 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:23.693 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 285 2024-09-13T09:12:23.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.693 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778209+0000", 2024-09-13T09:12:23.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778209+0000", 2024-09-13T09:12:23.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778209+0000", 2024-09-13T09:12:23.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:23.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778213+0000", 2024-09-13T09:12:23.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.301e-06 2024-09-13T09:12:23.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778226+0000", 2024-09-13T09:12:23.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2923e-05 2024-09-13T09:12:23.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778233+0000", 2024-09-13T09:12:23.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5739999999999998e-06 2024-09-13T09:12:23.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778279+0000", 2024-09-13T09:12:23.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5300000000000003e-05 2024-09-13T09:12:23.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778352+0000", 2024-09-13T09:12:23.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2960999999999995e-05 2024-09-13T09:12:23.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:23.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778461+0000", 2024-09-13T09:12:23.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010903700000000001 2024-09-13T09:12:23.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:23.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.780559+0000", 2024-09-13T09:12:23.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020987150000000001 2024-09-13T09:12:23.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:23.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783266+0000", 2024-09-13T09:12:23.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002707096 2024-09-13T09:12:23.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783307+0000", 2024-09-13T09:12:23.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0043999999999998e-05 2024-09-13T09:12:23.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783347+0000", 2024-09-13T09:12:23.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0333000000000002e-05 2024-09-13T09:12:23.698 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.698 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.698 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.698 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:288 3.1e 3:7da6a7d6:::rbd_data.1123ffac6bc2.00000000000000cb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:23.699 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.778243+0000", 2024-09-13T09:12:23.699 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.898974038, 2024-09-13T09:12:23.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050784439999999997, 2024-09-13T09:12:23.699 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.699 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.699 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:23.699 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:23.699 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:23.699 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 288 2024-09-13T09:12:23.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.699 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778243+0000", 2024-09-13T09:12:23.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778243+0000", 2024-09-13T09:12:23.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778242+0000", 2024-09-13T09:12:23.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:23.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778245+0000", 2024-09-13T09:12:23.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9909999999999998e-06 2024-09-13T09:12:23.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778264+0000", 2024-09-13T09:12:23.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8698999999999998e-05 2024-09-13T09:12:23.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778268+0000", 2024-09-13T09:12:23.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.002e-06 2024-09-13T09:12:23.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778302+0000", 2024-09-13T09:12:23.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3926e-05 2024-09-13T09:12:23.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778348+0000", 2024-09-13T09:12:23.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6458999999999997e-05 2024-09-13T09:12:23.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:23.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778427+0000", 2024-09-13T09:12:23.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8645000000000003e-05 2024-09-13T09:12:23.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:23.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.779006+0000", 2024-09-13T09:12:23.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00057902799999999999 2024-09-13T09:12:23.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:23.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783241+0000", 2024-09-13T09:12:23.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042353790000000001 2024-09-13T09:12:23.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783286+0000", 2024-09-13T09:12:23.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4537999999999999e-05 2024-09-13T09:12:23.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783321+0000", 2024-09-13T09:12:23.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5308999999999997e-05 2024-09-13T09:12:23.704 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.704 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.704 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.704 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:290 3.8 3:17d1c290:::rbd_data.1123ffac6bc2.00000000000000e6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:23.704 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.778368+0000", 2024-09-13T09:12:23.704 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.8988490630000001, 2024-09-13T09:12:23.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049790379999999999, 2024-09-13T09:12:23.705 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.705 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.705 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:23.705 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:23.705 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:23.705 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 290 2024-09-13T09:12:23.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.705 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778368+0000", 2024-09-13T09:12:23.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778368+0000", 2024-09-13T09:12:23.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778367+0000", 2024-09-13T09:12:23.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:23.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778371+0000", 2024-09-13T09:12:23.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.98e-06 2024-09-13T09:12:23.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778382+0000", 2024-09-13T09:12:23.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:12:23.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778388+0000", 2024-09-13T09:12:23.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3459999999999999e-06 2024-09-13T09:12:23.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778425+0000", 2024-09-13T09:12:23.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6721000000000003e-05 2024-09-13T09:12:23.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778486+0000", 2024-09-13T09:12:23.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1344000000000001e-05 2024-09-13T09:12:23.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:23.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778585+0000", 2024-09-13T09:12:23.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8061999999999997e-05 2024-09-13T09:12:23.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:23.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.780584+0000", 2024-09-13T09:12:23.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019999649999999998 2024-09-13T09:12:23.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:23.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783298+0000", 2024-09-13T09:12:23.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027135779999999999 2024-09-13T09:12:23.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783326+0000", 2024-09-13T09:12:23.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7727999999999999e-05 2024-09-13T09:12:23.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783347+0000", 2024-09-13T09:12:23.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0971e-05 2024-09-13T09:12:23.710 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.710 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.710 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.710 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:393 3.7 3:e1888e44:::rbd_data.1123ffac6bc2.00000000000000a4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:23.710 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.836837+0000", 2024-09-13T09:12:23.710 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.8403795249999999, 2024-09-13T09:12:23.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048048379999999996, 2024-09-13T09:12:23.711 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.711 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.711 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:23.711 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:23.711 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:23.711 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 393 2024-09-13T09:12:23.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.711 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836837+0000", 2024-09-13T09:12:23.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836837+0000", 2024-09-13T09:12:23.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836835+0000", 2024-09-13T09:12:23.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-09-13T09:12:23.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836844+0000", 2024-09-13T09:12:23.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2930000000000001e-06 2024-09-13T09:12:23.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836860+0000", 2024-09-13T09:12:23.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5772000000000001e-05 2024-09-13T09:12:23.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836868+0000", 2024-09-13T09:12:23.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0090000000000007e-06 2024-09-13T09:12:23.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836932+0000", 2024-09-13T09:12:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4336000000000002e-05 2024-09-13T09:12:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837025+0000", 2024-09-13T09:12:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3282999999999994e-05 2024-09-13T09:12:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837149+0000", 2024-09-13T09:12:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000123776 2024-09-13T09:12:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:23.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.838672+0000", 2024-09-13T09:12:23.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015235579999999999 2024-09-13T09:12:23.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:23.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.841574+0000", 2024-09-13T09:12:23.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029013899999999998 2024-09-13T09:12:23.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.841609+0000", 2024-09-13T09:12:23.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5278999999999999e-05 2024-09-13T09:12:23.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.841642+0000", 2024-09-13T09:12:23.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2947999999999997e-05 2024-09-13T09:12:23.717 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.717 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.717 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.717 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:394 3.0 e27/21 3:03bb2f6d:::rbd_data.1123ffac6bc2.00000000000000a5:head v 27'2, mlcod=0'0)", 2024-09-13T09:12:23.717 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.837093+0000", 2024-09-13T09:12:23.717 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.840123825, 2024-09-13T09:12:23.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0043767670000000002, 2024-09-13T09:12:23.717 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.717 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.717 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.718 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.718 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837093+0000", 2024-09-13T09:12:23.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.718 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.718 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837093+0000", 2024-09-13T09:12:23.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.718 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.718 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837092+0000", 2024-09-13T09:12:23.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:23.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837099+0000", 2024-09-13T09:12:23.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0570000000000004e-06 2024-09-13T09:12:23.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837111+0000", 2024-09-13T09:12:23.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1642e-05 2024-09-13T09:12:23.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837117+0000", 2024-09-13T09:12:23.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2519999999999996e-06 2024-09-13T09:12:23.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837159+0000", 2024-09-13T09:12:23.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1943999999999997e-05 2024-09-13T09:12:23.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837279+0000", 2024-09-13T09:12:23.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012011 2024-09-13T09:12:23.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.841434+0000", 2024-09-13T09:12:23.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0041544019999999997 2024-09-13T09:12:23.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.721 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.841470+0000", 2024-09-13T09:12:23.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6124999999999997e-05 2024-09-13T09:12:23.721 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.721 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.721 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.721 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:399 3.5 e27/21 3:a2e350d5:::rbd_data.1123ffac6bc2.00000000000000e2:head v 27'119, mlcod=27'117)", 2024-09-13T09:12:23.722 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.837182+0000", 2024-09-13T09:12:23.722 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.8400347899999998, 2024-09-13T09:12:23.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042512619999999996, 2024-09-13T09:12:23.722 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.722 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.722 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837182+0000", 2024-09-13T09:12:23.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837182+0000", 2024-09-13T09:12:23.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837181+0000", 2024-09-13T09:12:23.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:23.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837187+0000", 2024-09-13T09:12:23.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5650000000000001e-06 2024-09-13T09:12:23.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837216+0000", 2024-09-13T09:12:23.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9167e-05 2024-09-13T09:12:23.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837222+0000", 2024-09-13T09:12:23.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9100000000000002e-06 2024-09-13T09:12:23.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837259+0000", 2024-09-13T09:12:23.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6470999999999997e-05 2024-09-13T09:12:23.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837282+0000", 2024-09-13T09:12:23.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3314000000000001e-05 2024-09-13T09:12:23.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.841395+0000", 2024-09-13T09:12:23.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0041134580000000004 2024-09-13T09:12:23.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.841433+0000", 2024-09-13T09:12:23.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7972000000000002e-05 2024-09-13T09:12:23.726 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.726 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.726 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.726 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:451 3.18 3:1c88c91a:::rbd_data.1123ffac6bc2.000000000000002d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:23.726 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.865164+0000", 2024-09-13T09:12:23.726 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.8120525020000002, 2024-09-13T09:12:23.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051471370000000004, 2024-09-13T09:12:23.726 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.726 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.726 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:23.727 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:23.727 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:23.727 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 451 2024-09-13T09:12:23.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.727 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865164+0000", 2024-09-13T09:12:23.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865164+0000", 2024-09-13T09:12:23.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865163+0000", 2024-09-13T09:12:23.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:23.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865169+0000", 2024-09-13T09:12:23.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.798e-06 2024-09-13T09:12:23.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865185+0000", 2024-09-13T09:12:23.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5404e-05 2024-09-13T09:12:23.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865193+0000", 2024-09-13T09:12:23.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:12:23.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865240+0000", 2024-09-13T09:12:23.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7293000000000003e-05 2024-09-13T09:12:23.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865331+0000", 2024-09-13T09:12:23.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0672000000000002e-05 2024-09-13T09:12:23.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:23.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865410+0000", 2024-09-13T09:12:23.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9319000000000004e-05 2024-09-13T09:12:23.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:23.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.867480+0000", 2024-09-13T09:12:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020703430000000001 2024-09-13T09:12:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870227+0000", 2024-09-13T09:12:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00274705 2024-09-13T09:12:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870272+0000", 2024-09-13T09:12:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4554000000000001e-05 2024-09-13T09:12:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870311+0000", 2024-09-13T09:12:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9536000000000002e-05 2024-09-13T09:12:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:453 3.7 3:e75bf08e:::rbd_data.1123ffac6bc2.000000000000002e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.865211+0000", 2024-09-13T09:12:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.8120061110000001, 2024-09-13T09:12:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051785149999999999, 2024-09-13T09:12:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.732 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 453 2024-09-13T09:12:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865211+0000", 2024-09-13T09:12:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865211+0000", 2024-09-13T09:12:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865210+0000", 2024-09-13T09:12:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865213+0000", 2024-09-13T09:12:23.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:12:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865222+0000", 2024-09-13T09:12:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8380000000000008e-06 2024-09-13T09:12:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865228+0000", 2024-09-13T09:12:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9900000000000002e-06 2024-09-13T09:12:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865276+0000", 2024-09-13T09:12:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7407999999999997e-05 2024-09-13T09:12:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865364+0000", 2024-09-13T09:12:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8412000000000007e-05 2024-09-13T09:12:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865471+0000", 2024-09-13T09:12:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000106609 2024-09-13T09:12:23.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.869879+0000", 2024-09-13T09:12:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044085640000000002 2024-09-13T09:12:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870308+0000", 2024-09-13T09:12:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00042895900000000002 2024-09-13T09:12:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870353+0000", 2024-09-13T09:12:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4749999999999997e-05 2024-09-13T09:12:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870389+0000", 2024-09-13T09:12:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6269999999999997e-05 2024-09-13T09:12:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:459 3.13 e27/22 3:cd3e5265:::rbd_data.1123ffac6bc2.0000000000000040:head v 27'6, mlcod=27'4)", 2024-09-13T09:12:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.865629+0000", 2024-09-13T09:12:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.8115873649999998, 2024-09-13T09:12:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042945450000000003, 2024-09-13T09:12:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865629+0000", 2024-09-13T09:12:23.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865629+0000", 2024-09-13T09:12:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865629+0000", 2024-09-13T09:12:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865636+0000", 2024-09-13T09:12:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4200000000000001e-06 2024-09-13T09:12:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865647+0000", 2024-09-13T09:12:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0404999999999999e-05 2024-09-13T09:12:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865654+0000", 2024-09-13T09:12:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.537e-06 2024-09-13T09:12:23.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865698+0000", 2024-09-13T09:12:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3390000000000003e-05 2024-09-13T09:12:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865717+0000", 2024-09-13T09:12:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9395999999999999e-05 2024-09-13T09:12:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.869892+0000", 2024-09-13T09:12:23.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0041746659999999996 2024-09-13T09:12:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.869924+0000", 2024-09-13T09:12:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2363999999999997e-05 2024-09-13T09:12:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:457 3.4 e27/21 3:20969f55:::rbd_data.1123ffac6bc2.0000000000000035:head v 27'2, mlcod=0'0)", 2024-09-13T09:12:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.865972+0000", 2024-09-13T09:12:23.743 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.8112445020000001, 2024-09-13T09:12:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0039767919999999998, 2024-09-13T09:12:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865972+0000", 2024-09-13T09:12:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865972+0000", 2024-09-13T09:12:23.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865971+0000", 2024-09-13T09:12:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865979+0000", 2024-09-13T09:12:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7389999999999999e-06 2024-09-13T09:12:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865993+0000", 2024-09-13T09:12:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3730999999999999e-05 2024-09-13T09:12:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866000+0000", 2024-09-13T09:12:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9240000000000001e-06 2024-09-13T09:12:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866045+0000", 2024-09-13T09:12:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5374999999999998e-05 2024-09-13T09:12:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.866068+0000", 2024-09-13T09:12:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2413e-05 2024-09-13T09:12:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.869900+0000", 2024-09-13T09:12:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0038327169999999998 2024-09-13T09:12:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.869949+0000", 2024-09-13T09:12:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8723000000000001e-05 2024-09-13T09:12:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.747 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:710 3.d 3:b2fa2321:::rbd_data.1123ffac6bc2.0000000000000088:head [set-alloc-hint object_size 1048576 write_size 1048576,write 393216~24064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.980009+0000", 2024-09-13T09:12:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.697207718, 2024-09-13T09:12:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037187510000000002, 2024-09-13T09:12:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 710 2024-09-13T09:12:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.980009+0000", 2024-09-13T09:12:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.980009+0000", 2024-09-13T09:12:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.749 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.980008+0000", 2024-09-13T09:12:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.980029+0000", 2024-09-13T09:12:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0968e-05 2024-09-13T09:12:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.980044+0000", 2024-09-13T09:12:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5515e-05 2024-09-13T09:12:23.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.980054+0000", 2024-09-13T09:12:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2089999999999994e-06 2024-09-13T09:12:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.980121+0000", 2024-09-13T09:12:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7323999999999994e-05 2024-09-13T09:12:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.980262+0000", 2024-09-13T09:12:23.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014135999999999999 2024-09-13T09:12:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.980339+0000", 2024-09-13T09:12:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7039000000000006e-05 2024-09-13T09:12:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.980914+0000", 2024-09-13T09:12:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00057421900000000005 2024-09-13T09:12:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.983636+0000", 2024-09-13T09:12:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027224800000000002 2024-09-13T09:12:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.983686+0000", 2024-09-13T09:12:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9733e-05 2024-09-13T09:12:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.983728+0000", 2024-09-13T09:12:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2027999999999998e-05 2024-09-13T09:12:23.753 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:1052 3.7 3:e7c11647:::rbd_data.1123ffac6bc2.0000000000000090:head [set-alloc-hint object_size 1048576 write_size 1048576,write 503808~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.140204+0000", 2024-09-13T09:12:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.5370131759999999, 2024-09-13T09:12:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055287460000000002, 2024-09-13T09:12:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:23.754 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1052 2024-09-13T09:12:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140204+0000", 2024-09-13T09:12:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140204+0000", 2024-09-13T09:12:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140202+0000", 2024-09-13T09:12:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-09-13T09:12:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140242+0000", 2024-09-13T09:12:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0512000000000001e-05 2024-09-13T09:12:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140263+0000", 2024-09-13T09:12:23.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1268e-05 2024-09-13T09:12:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140277+0000", 2024-09-13T09:12:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3346000000000001e-05 2024-09-13T09:12:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140333+0000", 2024-09-13T09:12:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5695000000000001e-05 2024-09-13T09:12:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140461+0000", 2024-09-13T09:12:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012853800000000001 2024-09-13T09:12:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140645+0000", 2024-09-13T09:12:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000183463 2024-09-13T09:12:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145465+0000", 2024-09-13T09:12:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048209289999999998 2024-09-13T09:12:23.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:23.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145639+0000", 2024-09-13T09:12:23.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017306799999999999 2024-09-13T09:12:23.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.760 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145696+0000", 2024-09-13T09:12:23.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7599000000000002e-05 2024-09-13T09:12:23.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.760 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.760 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145732+0000", 2024-09-13T09:12:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6254000000000002e-05 2024-09-13T09:12:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:1055 3.18 3:192ac1d1:::rbd_data.1123ffac6bc2.0000000000000091:head [set-alloc-hint object_size 1048576 write_size 1048576,write 393216~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.140290+0000", 2024-09-13T09:12:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.5369263040000001, 2024-09-13T09:12:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.005447454, 2024-09-13T09:12:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.761 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:23.762 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:23.762 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:23.762 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1055 2024-09-13T09:12:23.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.762 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.762 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.762 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140290+0000", 2024-09-13T09:12:23.762 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.762 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.762 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140290+0000", 2024-09-13T09:12:23.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.763 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140290+0000", 2024-09-13T09:12:23.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:23.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.763 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140311+0000", 2024-09-13T09:12:23.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1484e-05 2024-09-13T09:12:23.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.764 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140320+0000", 2024-09-13T09:12:23.764 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1500000000000005e-06 2024-09-13T09:12:23.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.764 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.764 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140324+0000", 2024-09-13T09:12:23.764 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:12:23.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.764 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.764 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140375+0000", 2024-09-13T09:12:23.764 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1082000000000003e-05 2024-09-13T09:12:23.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.765 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.765 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140504+0000", 2024-09-13T09:12:23.765 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012903599999999999 2024-09-13T09:12:23.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.765 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:23.765 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140608+0000", 2024-09-13T09:12:23.765 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000103669 2024-09-13T09:12:23.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.765 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:23.766 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142875+0000", 2024-09-13T09:12:23.766 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002267388 2024-09-13T09:12:23.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.766 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:23.766 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145639+0000", 2024-09-13T09:12:23.766 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027635369999999999 2024-09-13T09:12:23.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.766 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145700+0000", 2024-09-13T09:12:23.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1453999999999998e-05 2024-09-13T09:12:23.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.767 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145738+0000", 2024-09-13T09:12:23.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7478999999999998e-05 2024-09-13T09:12:23.767 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.767 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.767 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.767 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:1069 3.5 e27/21 3:a4a542ae:::rbd_data.1123ffac6bc2.0000000000000036:head v 27'123, mlcod=27'121)", 2024-09-13T09:12:23.768 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.142984+0000", 2024-09-13T09:12:23.768 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.5342330159999999, 2024-09-13T09:12:23.768 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052211899999999997, 2024-09-13T09:12:23.768 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.768 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.768 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.768 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.768 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.768 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142984+0000", 2024-09-13T09:12:23.768 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.768 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142984+0000", 2024-09-13T09:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142983+0000", 2024-09-13T09:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142996+0000", 2024-09-13T09:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3361e-05 2024-09-13T09:12:23.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143012+0000", 2024-09-13T09:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5574999999999999e-05 2024-09-13T09:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143019+0000", 2024-09-13T09:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7290000000000003e-06 2024-09-13T09:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143066+0000", 2024-09-13T09:12:23.770 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7250999999999999e-05 2024-09-13T09:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143088+0000", 2024-09-13T09:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2450999999999999e-05 2024-09-13T09:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148171+0000", 2024-09-13T09:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050823190000000001 2024-09-13T09:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148205+0000", 2024-09-13T09:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4350000000000001e-05 2024-09-13T09:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:1099 3.7 3:e7c11647:::rbd_data.1123ffac6bc2.0000000000000090:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.161109+0000", 2024-09-13T09:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.5161072989999997, 2024-09-13T09:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037516759999999998, 2024-09-13T09:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.772 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1099 2024-09-13T09:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161109+0000", 2024-09-13T09:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.773 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161109+0000", 2024-09-13T09:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161108+0000", 2024-09-13T09:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161115+0000", 2024-09-13T09:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.774 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161132+0000", 2024-09-13T09:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6518e-05 2024-09-13T09:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161143+0000", 2024-09-13T09:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1097999999999999e-05 2024-09-13T09:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161187+0000", 2024-09-13T09:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4401999999999999e-05 2024-09-13T09:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161268+0000", 2024-09-13T09:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0829999999999997e-05 2024-09-13T09:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161371+0000", 2024-09-13T09:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000103138 2024-09-13T09:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.164140+0000", 2024-09-13T09:12:23.776 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027687670000000001 2024-09-13T09:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.164766+0000", 2024-09-13T09:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00062610300000000003 2024-09-13T09:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.164822+0000", 2024-09-13T09:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5965999999999999e-05 2024-09-13T09:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.777 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.164861+0000", 2024-09-13T09:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9088999999999998e-05 2024-09-13T09:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:1100 3.18 3:192ac1d1:::rbd_data.1123ffac6bc2.0000000000000091:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.161156+0000", 2024-09-13T09:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.5160610380000001, 2024-09-13T09:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0038418139999999998, 2024-09-13T09:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:23.778 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1100 2024-09-13T09:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161156+0000", 2024-09-13T09:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161156+0000", 2024-09-13T09:12:23.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161155+0000", 2024-09-13T09:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161158+0000", 2024-09-13T09:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.095e-06 2024-09-13T09:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.780 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161166+0000", 2024-09-13T09:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161172+0000", 2024-09-13T09:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161214+0000", 2024-09-13T09:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2150999999999997e-05 2024-09-13T09:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.781 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161293+0000", 2024-09-13T09:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8776999999999995e-05 2024-09-13T09:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.161401+0000", 2024-09-13T09:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000107562 2024-09-13T09:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.162132+0000", 2024-09-13T09:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00073154800000000005 2024-09-13T09:12:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.164926+0000", 2024-09-13T09:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027938080000000001 2024-09-13T09:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.164959+0000", 2024-09-13T09:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2660999999999998e-05 2024-09-13T09:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.164998+0000", 2024-09-13T09:12:23.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8855e-05 2024-09-13T09:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:1144 3.8 3:1232303b:::rbd_data.1123ffac6bc2.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 494080~30208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.176352+0000", 2024-09-13T09:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.5008648959999999, 2024-09-13T09:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002753967, 2024-09-13T09:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:23.784 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1144 2024-09-13T09:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.176352+0000", 2024-09-13T09:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.176352+0000", 2024-09-13T09:12:23.785 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.176350+0000", 2024-09-13T09:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.176380+0000", 2024-09-13T09:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9918999999999999e-05 2024-09-13T09:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.786 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.176400+0000", 2024-09-13T09:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9375999999999999e-05 2024-09-13T09:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.176410+0000", 2024-09-13T09:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0852999999999999e-05 2024-09-13T09:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.176469+0000", 2024-09-13T09:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8233000000000003e-05 2024-09-13T09:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.176612+0000", 2024-09-13T09:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014315500000000001 2024-09-13T09:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.176784+0000", 2024-09-13T09:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000172571 2024-09-13T09:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.178834+0000", 2024-09-13T09:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020499670000000002 2024-09-13T09:12:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.179052+0000", 2024-09-13T09:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000217269 2024-09-13T09:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.179085+0000", 2024-09-13T09:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.789 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.179106+0000", 2024-09-13T09:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0967000000000001e-05 2024-09-13T09:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4513.0:820 3.8 3:1239702a:::rbd_data.11238839e366.0000000000000006:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:23.512749+0000", 2024-09-13T09:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.16446771399999999, 2024-09-13T09:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027776039999999999, 2024-09-13T09:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:23.790 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4513", 2024-09-13T09:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/432888818", 2024-09-13T09:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 820 2024-09-13T09:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:23.512749+0000", 2024-09-13T09:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:23.512749+0000", 2024-09-13T09:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:23.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:23.512748+0000", 2024-09-13T09:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:23.512753+0000", 2024-09-13T09:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2599999999999999e-06 2024-09-13T09:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:23.512762+0000", 2024-09-13T09:12:23.792 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8830000000000001e-06 2024-09-13T09:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:23.512767+0000", 2024-09-13T09:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9740000000000001e-06 2024-09-13T09:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:23.512799+0000", 2024-09-13T09:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1723000000000001e-05 2024-09-13T09:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:23.512886+0000", 2024-09-13T09:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6846000000000002e-05 2024-09-13T09:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:23.513074+0000", 2024-09-13T09:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000188001 2024-09-13T09:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:23.514321+0000", 2024-09-13T09:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001247377 2024-09-13T09:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:23.515467+0000", 2024-09-13T09:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001145758 2024-09-13T09:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:23.515492+0000", 2024-09-13T09:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5284999999999999e-05 2024-09-13T09:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:23.515527+0000", 2024-09-13T09:12:23.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4239000000000002e-05 2024-09-13T09:12:23.796 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.796 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.796 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.796 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:23.796 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:23.796 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:23.797 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:23.798 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:23.797+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:23.820 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-13T09:12:23.897 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:23.896+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:23.911 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:23.911 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:23.911 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:23.911 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:23.921 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:23.998 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:23.997+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:24.072 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:24.072 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:24.073 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:24.081 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:24.099 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:24.097+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:24.186 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:24.186 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:24.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:24.187 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:24.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.187 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:24.187 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:11:59.289661+0000", 2024-09-13T09:12:24.187 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 24.894311053999999, 2024-09-13T09:12:24.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.93386499199999995, 2024-09-13T09:12:24.187 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.187 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.187 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1 2024-09-13T09:12:24.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.188 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289661+0000", 2024-09-13T09:12:24.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289661+0000", 2024-09-13T09:12:24.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289659+0000", 2024-09-13T09:12:24.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:24.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289669+0000", 2024-09-13T09:12:24.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:12:24.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289681+0000", 2024-09-13T09:12:24.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2316e-05 2024-09-13T09:12:24.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289693+0000", 2024-09-13T09:12:24.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2549e-05 2024-09-13T09:12:24.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289744+0000", 2024-09-13T09:12:24.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0365999999999999e-05 2024-09-13T09:12:24.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:24.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289752+0000", 2024-09-13T09:12:24.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5590000000000006e-06 2024-09-13T09:12:24.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.218301+0000", 2024-09-13T09:12:24.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.928549085 2024-09-13T09:12:24.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.218447+0000", 2024-09-13T09:12:24.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014537400000000001 2024-09-13T09:12:24.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.218615+0000", 2024-09-13T09:12:24.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000168195 2024-09-13T09:12:24.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.219277+0000", 2024-09-13T09:12:24.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066202600000000002 2024-09-13T09:12:24.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.223461+0000", 2024-09-13T09:12:24.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004184152 2024-09-13T09:12:24.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.223500+0000", 2024-09-13T09:12:24.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8733000000000003e-05 2024-09-13T09:12:24.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.223526+0000", 2024-09-13T09:12:24.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5612000000000001e-05 2024-09-13T09:12:24.193 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.193 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.193 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.194 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:24.194 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:24.194 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.936594548999999, 2024-09-13T09:12:24.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079358310000000008, 2024-09-13T09:12:24.194 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.194 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.194 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.194 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.194 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.194 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35 2024-09-13T09:12:24.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.194 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:24.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:24.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:24.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:24.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247410+0000", 2024-09-13T09:12:24.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3053999999999999e-05 2024-09-13T09:12:24.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247420+0000", 2024-09-13T09:12:24.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7189999999999996e-06 2024-09-13T09:12:24.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247429+0000", 2024-09-13T09:12:24.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5340000000000005e-06 2024-09-13T09:12:24.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.249500+0000", 2024-09-13T09:12:24.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020708990000000002 2024-09-13T09:12:24.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.249551+0000", 2024-09-13T09:12:24.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0762e-05 2024-09-13T09:12:24.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.249649+0000", 2024-09-13T09:12:24.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.768e-05 2024-09-13T09:12:24.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.251260+0000", 2024-09-13T09:12:24.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001611101 2024-09-13T09:12:24.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255178+0000", 2024-09-13T09:12:24.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039180930000000001 2024-09-13T09:12:24.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255256+0000", 2024-09-13T09:12:24.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8572000000000006e-05 2024-09-13T09:12:24.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255313+0000", 2024-09-13T09:12:24.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6625000000000001e-05 2024-09-13T09:12:24.199 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.199 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.199 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.199 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:24.199 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:24.200 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.936537371, 2024-09-13T09:12:24.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079428840000000007, 2024-09-13T09:12:24.200 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.200 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.200 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.200 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.200 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.200 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36 2024-09-13T09:12:24.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.200 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:24.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:24.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:24.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247438+0000", 2024-09-13T09:12:24.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7409999999999998e-06 2024-09-13T09:12:24.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247442+0000", 2024-09-13T09:12:24.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:12:24.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247444+0000", 2024-09-13T09:12:24.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8880000000000001e-06 2024-09-13T09:12:24.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250381+0000", 2024-09-13T09:12:24.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0029366800000000001 2024-09-13T09:12:24.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250446+0000", 2024-09-13T09:12:24.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5173999999999996e-05 2024-09-13T09:12:24.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250546+0000", 2024-09-13T09:12:24.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100142 2024-09-13T09:12:24.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.251514+0000", 2024-09-13T09:12:24.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00096772000000000004 2024-09-13T09:12:24.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255336+0000", 2024-09-13T09:12:24.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038214719999999998 2024-09-13T09:12:24.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.205 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255358+0000", 2024-09-13T09:12:24.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2141e-05 2024-09-13T09:12:24.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.205 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.205 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255377+0000", 2024-09-13T09:12:24.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9398999999999998e-05 2024-09-13T09:12:24.205 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.205 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.205 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.205 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:24.205 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:24.206 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.936522769, 2024-09-13T09:12:24.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079696799999999998, 2024-09-13T09:12:24.206 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.206 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.206 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37 2024-09-13T09:12:24.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.206 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:24.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:24.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:24.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:24.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247451+0000", 2024-09-13T09:12:24.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6000000000000001e-06 2024-09-13T09:12:24.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247463+0000", 2024-09-13T09:12:24.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1609e-05 2024-09-13T09:12:24.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247473+0000", 2024-09-13T09:12:24.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0536999999999999e-05 2024-09-13T09:12:24.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250802+0000", 2024-09-13T09:12:24.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033287270000000001 2024-09-13T09:12:24.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250853+0000", 2024-09-13T09:12:24.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1329000000000003e-05 2024-09-13T09:12:24.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250935+0000", 2024-09-13T09:12:24.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1396999999999993e-05 2024-09-13T09:12:24.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.251796+0000", 2024-09-13T09:12:24.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00086075099999999999 2024-09-13T09:12:24.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255382+0000", 2024-09-13T09:12:24.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003586334 2024-09-13T09:12:24.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255402+0000", 2024-09-13T09:12:24.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0392000000000001e-05 2024-09-13T09:12:24.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255419+0000", 2024-09-13T09:12:24.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6229e-05 2024-09-13T09:12:24.211 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.211 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.211 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.211 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:24.211 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:24.211 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.908588675000001, 2024-09-13T09:12:24.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077389640000000001, 2024-09-13T09:12:24.212 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.212 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.212 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.212 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.212 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.212 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 56 2024-09-13T09:12:24.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.212 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:24.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:24.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:24.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275385+0000", 2024-09-13T09:12:24.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:12:24.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275388+0000", 2024-09-13T09:12:24.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:12:24.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275396+0000", 2024-09-13T09:12:24.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0099999999999995e-06 2024-09-13T09:12:24.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.276666+0000", 2024-09-13T09:12:24.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00126968 2024-09-13T09:12:24.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.276710+0000", 2024-09-13T09:12:24.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4141999999999999e-05 2024-09-13T09:12:24.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.276785+0000", 2024-09-13T09:12:24.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4777000000000006e-05 2024-09-13T09:12:24.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.282050+0000", 2024-09-13T09:12:24.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0052647320000000003 2024-09-13T09:12:24.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283071+0000", 2024-09-13T09:12:24.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001021196 2024-09-13T09:12:24.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283102+0000", 2024-09-13T09:12:24.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0849999999999998e-05 2024-09-13T09:12:24.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283122+0000", 2024-09-13T09:12:24.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0190000000000002e-05 2024-09-13T09:12:24.217 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.217 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.217 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.217 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:58 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:24.217 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:24.217 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.900605160000001, 2024-09-13T09:12:24.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064599870000000004, 2024-09-13T09:12:24.217 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.218 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.218 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.218 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.218 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.218 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 58 2024-09-13T09:12:24.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.218 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:24.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:24.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:24.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283398+0000", 2024-09-13T09:12:24.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1646000000000002e-05 2024-09-13T09:12:24.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283403+0000", 2024-09-13T09:12:24.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0540000000000002e-06 2024-09-13T09:12:24.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283406+0000", 2024-09-13T09:12:24.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3110000000000001e-06 2024-09-13T09:12:24.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283841+0000", 2024-09-13T09:12:24.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000434549 2024-09-13T09:12:24.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283900+0000", 2024-09-13T09:12:24.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9153000000000001e-05 2024-09-13T09:12:24.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283975+0000", 2024-09-13T09:12:24.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4967000000000002e-05 2024-09-13T09:12:24.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.285844+0000", 2024-09-13T09:12:24.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001869435 2024-09-13T09:12:24.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289769+0000", 2024-09-13T09:12:24.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039245230000000001 2024-09-13T09:12:24.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289804+0000", 2024-09-13T09:12:24.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4999000000000002e-05 2024-09-13T09:12:24.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289826+0000", 2024-09-13T09:12:24.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2857000000000002e-05 2024-09-13T09:12:24.223 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.223 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.223 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.223 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:59 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:24.223 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.283413+0000", 2024-09-13T09:12:24.223 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.900558815, 2024-09-13T09:12:24.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064987589999999998, 2024-09-13T09:12:24.223 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.223 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.224 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.224 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.224 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.224 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 59 2024-09-13T09:12:24.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.224 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283413+0000", 2024-09-13T09:12:24.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283413+0000", 2024-09-13T09:12:24.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283412+0000", 2024-09-13T09:12:24.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283416+0000", 2024-09-13T09:12:24.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:12:24.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283420+0000", 2024-09-13T09:12:24.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6789999999999996e-06 2024-09-13T09:12:24.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283423+0000", 2024-09-13T09:12:24.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.672e-06 2024-09-13T09:12:24.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.284235+0000", 2024-09-13T09:12:24.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00081158900000000004 2024-09-13T09:12:24.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.284282+0000", 2024-09-13T09:12:24.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7213999999999999e-05 2024-09-13T09:12:24.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.284359+0000", 2024-09-13T09:12:24.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6814e-05 2024-09-13T09:12:24.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.285868+0000", 2024-09-13T09:12:24.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015097310000000001 2024-09-13T09:12:24.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289836+0000", 2024-09-13T09:12:24.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003967613 2024-09-13T09:12:24.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289877+0000", 2024-09-13T09:12:24.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1519e-05 2024-09-13T09:12:24.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289912+0000", 2024-09-13T09:12:24.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4088999999999998e-05 2024-09-13T09:12:24.229 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.229 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.229 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.229 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:24.229 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.298116+0000", 2024-09-13T09:12:24.229 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.885856011000001, 2024-09-13T09:12:24.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063667139999999999, 2024-09-13T09:12:24.229 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.229 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.229 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.229 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.229 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.230 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 69 2024-09-13T09:12:24.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.230 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298116+0000", 2024-09-13T09:12:24.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298116+0000", 2024-09-13T09:12:24.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298115+0000", 2024-09-13T09:12:24.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298193+0000", 2024-09-13T09:12:24.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7441999999999995e-05 2024-09-13T09:12:24.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298213+0000", 2024-09-13T09:12:24.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0829999999999999e-05 2024-09-13T09:12:24.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298224+0000", 2024-09-13T09:12:24.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0282000000000001e-05 2024-09-13T09:12:24.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299016+0000", 2024-09-13T09:12:24.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00079248999999999997 2024-09-13T09:12:24.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299098+0000", 2024-09-13T09:12:24.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1607999999999995e-05 2024-09-13T09:12:24.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299223+0000", 2024-09-13T09:12:24.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000124838 2024-09-13T09:12:24.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.300536+0000", 2024-09-13T09:12:24.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001312843 2024-09-13T09:12:24.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304428+0000", 2024-09-13T09:12:24.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038928000000000001 2024-09-13T09:12:24.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304463+0000", 2024-09-13T09:12:24.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4214999999999997e-05 2024-09-13T09:12:24.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304482+0000", 2024-09-13T09:12:24.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9800999999999999e-05 2024-09-13T09:12:24.234 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.235 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.235 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.235 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:24.235 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.298237+0000", 2024-09-13T09:12:24.235 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.885735043, 2024-09-13T09:12:24.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063589789999999998, 2024-09-13T09:12:24.235 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.235 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.236 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 70 2024-09-13T09:12:24.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.236 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298237+0000", 2024-09-13T09:12:24.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298237+0000", 2024-09-13T09:12:24.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298236+0000", 2024-09-13T09:12:24.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:24.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298242+0000", 2024-09-13T09:12:24.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5889999999999999e-06 2024-09-13T09:12:24.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298252+0000", 2024-09-13T09:12:24.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6539999999999994e-06 2024-09-13T09:12:24.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298257+0000", 2024-09-13T09:12:24.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4240000000000001e-06 2024-09-13T09:12:24.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299735+0000", 2024-09-13T09:12:24.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0014772120000000001 2024-09-13T09:12:24.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299851+0000", 2024-09-13T09:12:24.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000115998 2024-09-13T09:12:24.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299975+0000", 2024-09-13T09:12:24.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000124805 2024-09-13T09:12:24.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.300830+0000", 2024-09-13T09:12:24.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0008543 2024-09-13T09:12:24.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304519+0000", 2024-09-13T09:12:24.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036896849999999998 2024-09-13T09:12:24.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304560+0000", 2024-09-13T09:12:24.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1009e-05 2024-09-13T09:12:24.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304596+0000", 2024-09-13T09:12:24.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5114999999999998e-05 2024-09-13T09:12:24.241 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.241 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.241 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.241 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:85 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:24.241 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:24.241 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.871784643000002, 2024-09-13T09:12:24.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068319399999999999, 2024-09-13T09:12:24.241 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.241 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.241 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.242 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.242 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.242 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 85 2024-09-13T09:12:24.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.242 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:24.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:24.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:24.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312190+0000", 2024-09-13T09:12:24.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2389999999999998e-06 2024-09-13T09:12:24.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312198+0000", 2024-09-13T09:12:24.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8599999999999993e-06 2024-09-13T09:12:24.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312201+0000", 2024-09-13T09:12:24.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.399e-06 2024-09-13T09:12:24.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.313614+0000", 2024-09-13T09:12:24.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001413369 2024-09-13T09:12:24.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.313702+0000", 2024-09-13T09:12:24.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7556e-05 2024-09-13T09:12:24.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.313843+0000", 2024-09-13T09:12:24.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000141015 2024-09-13T09:12:24.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318968+0000", 2024-09-13T09:12:24.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0051251680000000003 2024-09-13T09:12:24.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318973+0000", 2024-09-13T09:12:24.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9520000000000002e-06 2024-09-13T09:12:24.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318999+0000", 2024-09-13T09:12:24.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5706000000000001e-05 2024-09-13T09:12:24.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319019+0000", 2024-09-13T09:12:24.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0024e-05 2024-09-13T09:12:24.247 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.247 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.247 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.247 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:24.247 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:24.247 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.871766622999999, 2024-09-13T09:12:24.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069390709999999998, 2024-09-13T09:12:24.247 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.247 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.247 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.247 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.248 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.248 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 86 2024-09-13T09:12:24.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.248 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:24.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:24.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:24.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312207+0000", 2024-09-13T09:12:24.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.243e-06 2024-09-13T09:12:24.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312211+0000", 2024-09-13T09:12:24.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1219999999999997e-06 2024-09-13T09:12:24.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312215+0000", 2024-09-13T09:12:24.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4680000000000001e-06 2024-09-13T09:12:24.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314378+0000", 2024-09-13T09:12:24.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021634309999999999 2024-09-13T09:12:24.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314426+0000", 2024-09-13T09:12:24.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7963000000000001e-05 2024-09-13T09:12:24.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314507+0000", 2024-09-13T09:12:24.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0619999999999997e-05 2024-09-13T09:12:24.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319024+0000", 2024-09-13T09:12:24.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045178140000000002 2024-09-13T09:12:24.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319065+0000", 2024-09-13T09:12:24.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0216999999999998e-05 2024-09-13T09:12:24.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319108+0000", 2024-09-13T09:12:24.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3618000000000001e-05 2024-09-13T09:12:24.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319144+0000", 2024-09-13T09:12:24.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5858999999999997e-05 2024-09-13T09:12:24.253 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.253 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.253 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.253 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:24.253 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:24.253 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.871752245, 2024-09-13T09:12:24.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011377922, 2024-09-13T09:12:24.253 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.253 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.253 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.253 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.253 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.254 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 87 2024-09-13T09:12:24.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.254 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:24.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:24.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:24.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:24.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312249+0000", 2024-09-13T09:12:24.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9377e-05 2024-09-13T09:12:24.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312255+0000", 2024-09-13T09:12:24.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0220000000000001e-06 2024-09-13T09:12:24.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312259+0000", 2024-09-13T09:12:24.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-13T09:12:24.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314845+0000", 2024-09-13T09:12:24.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0025858750000000001 2024-09-13T09:12:24.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314891+0000", 2024-09-13T09:12:24.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6347999999999997e-05 2024-09-13T09:12:24.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314968+0000", 2024-09-13T09:12:24.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7428999999999993e-05 2024-09-13T09:12:24.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319174+0000", 2024-09-13T09:12:24.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042056999999999997 2024-09-13T09:12:24.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323544+0000", 2024-09-13T09:12:24.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043701119999999998 2024-09-13T09:12:24.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323575+0000", 2024-09-13T09:12:24.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1285000000000003e-05 2024-09-13T09:12:24.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323597+0000", 2024-09-13T09:12:24.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1891000000000001e-05 2024-09-13T09:12:24.260 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.260 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.260 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.260 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:24.260 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:24.260 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.871709488, 2024-09-13T09:12:24.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011622512, 2024-09-13T09:12:24.261 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.261 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.261 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.261 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.261 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.261 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 88 2024-09-13T09:12:24.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.261 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:24.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:24.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:24.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312265+0000", 2024-09-13T09:12:24.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8380000000000002e-06 2024-09-13T09:12:24.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312268+0000", 2024-09-13T09:12:24.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2310000000000001e-06 2024-09-13T09:12:24.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312271+0000", 2024-09-13T09:12:24.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.551e-06 2024-09-13T09:12:24.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318458+0000", 2024-09-13T09:12:24.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061866710000000004 2024-09-13T09:12:24.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318540+0000", 2024-09-13T09:12:24.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1845e-05 2024-09-13T09:12:24.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318668+0000", 2024-09-13T09:12:24.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012830900000000001 2024-09-13T09:12:24.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319598+0000", 2024-09-13T09:12:24.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000929377 2024-09-13T09:12:24.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323840+0000", 2024-09-13T09:12:24.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042420180000000002 2024-09-13T09:12:24.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323866+0000", 2024-09-13T09:12:24.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6262999999999999e-05 2024-09-13T09:12:24.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323885+0000", 2024-09-13T09:12:24.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8675e-05 2024-09-13T09:12:24.266 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.266 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.266 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.267 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:24.267 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:24.267 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.711331494, 2024-09-13T09:12:24.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0062525000000000002, 2024-09-13T09:12:24.267 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.267 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.267 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.267 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.267 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.267 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113 2024-09-13T09:12:24.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.268 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:24.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:24.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:24.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472644+0000", 2024-09-13T09:12:24.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-13T09:12:24.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472650+0000", 2024-09-13T09:12:24.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:12:24.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472655+0000", 2024-09-13T09:12:24.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:12:24.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477258+0000", 2024-09-13T09:12:24.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004603762 2024-09-13T09:12:24.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477303+0000", 2024-09-13T09:12:24.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5031999999999998e-05 2024-09-13T09:12:24.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477375+0000", 2024-09-13T09:12:24.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1186000000000006e-05 2024-09-13T09:12:24.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478772+0000", 2024-09-13T09:12:24.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00139704 2024-09-13T09:12:24.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478817+0000", 2024-09-13T09:12:24.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5120999999999997e-05 2024-09-13T09:12:24.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478859+0000", 2024-09-13T09:12:24.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2567000000000001e-05 2024-09-13T09:12:24.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478893+0000", 2024-09-13T09:12:24.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3210000000000002e-05 2024-09-13T09:12:24.272 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.273 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.273 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.273 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:24.273 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:24.273 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.711308186, 2024-09-13T09:12:24.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063268899999999999, 2024-09-13T09:12:24.273 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.273 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.273 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.273 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.273 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.274 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 114 2024-09-13T09:12:24.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.274 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:24.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:24.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:24.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472667+0000", 2024-09-13T09:12:24.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9219999999999999e-06 2024-09-13T09:12:24.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472673+0000", 2024-09-13T09:12:24.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.63e-06 2024-09-13T09:12:24.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472678+0000", 2024-09-13T09:12:24.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6689999999999997e-06 2024-09-13T09:12:24.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477608+0000", 2024-09-13T09:12:24.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0049302870000000002 2024-09-13T09:12:24.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477685+0000", 2024-09-13T09:12:24.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6922000000000007e-05 2024-09-13T09:12:24.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477804+0000", 2024-09-13T09:12:24.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011865499999999999 2024-09-13T09:12:24.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478935+0000", 2024-09-13T09:12:24.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0011305099999999999 2024-09-13T09:12:24.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478942+0000", 2024-09-13T09:12:24.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3969999999999997e-06 2024-09-13T09:12:24.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478970+0000", 2024-09-13T09:12:24.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8447000000000001e-05 2024-09-13T09:12:24.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478990+0000", 2024-09-13T09:12:24.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9933e-05 2024-09-13T09:12:24.278 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.278 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.278 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.279 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:24.279 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472870+0000", 2024-09-13T09:12:24.279 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.711101768999999, 2024-09-13T09:12:24.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068457680000000003, 2024-09-13T09:12:24.279 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.279 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.279 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.279 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.279 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.279 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 115 2024-09-13T09:12:24.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.279 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472870+0000", 2024-09-13T09:12:24.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472870+0000", 2024-09-13T09:12:24.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472869+0000", 2024-09-13T09:12:24.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:24.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472933+0000", 2024-09-13T09:12:24.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3448000000000006e-05 2024-09-13T09:12:24.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472943+0000", 2024-09-13T09:12:24.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0234999999999999e-05 2024-09-13T09:12:24.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472948+0000", 2024-09-13T09:12:24.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:12:24.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478311+0000", 2024-09-13T09:12:24.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053631759999999999 2024-09-13T09:12:24.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478364+0000", 2024-09-13T09:12:24.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3174000000000003e-05 2024-09-13T09:12:24.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478441+0000", 2024-09-13T09:12:24.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6211000000000006e-05 2024-09-13T09:12:24.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479649+0000", 2024-09-13T09:12:24.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001208319 2024-09-13T09:12:24.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479673+0000", 2024-09-13T09:12:24.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4060999999999999e-05 2024-09-13T09:12:24.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479697+0000", 2024-09-13T09:12:24.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3958999999999999e-05 2024-09-13T09:12:24.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479716+0000", 2024-09-13T09:12:24.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8661e-05 2024-09-13T09:12:24.284 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.284 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.284 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.285 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:24.285 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472955+0000", 2024-09-13T09:12:24.285 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.711016713999999, 2024-09-13T09:12:24.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069091580000000003, 2024-09-13T09:12:24.285 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.285 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.285 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.285 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.285 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.285 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 116 2024-09-13T09:12:24.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.285 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472955+0000", 2024-09-13T09:12:24.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472955+0000", 2024-09-13T09:12:24.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472954+0000", 2024-09-13T09:12:24.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472959+0000", 2024-09-13T09:12:24.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1309999999999999e-06 2024-09-13T09:12:24.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472964+0000", 2024-09-13T09:12:24.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6659999999999996e-06 2024-09-13T09:12:24.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472968+0000", 2024-09-13T09:12:24.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.495e-06 2024-09-13T09:12:24.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479021+0000", 2024-09-13T09:12:24.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060531990000000004 2024-09-13T09:12:24.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479062+0000", 2024-09-13T09:12:24.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1436000000000001e-05 2024-09-13T09:12:24.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479131+0000", 2024-09-13T09:12:24.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8533000000000003e-05 2024-09-13T09:12:24.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479727+0000", 2024-09-13T09:12:24.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00059612 2024-09-13T09:12:24.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479827+0000", 2024-09-13T09:12:24.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9940999999999997e-05 2024-09-13T09:12:24.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479848+0000", 2024-09-13T09:12:24.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0877999999999999e-05 2024-09-13T09:12:24.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479864+0000", 2024-09-13T09:12:24.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6175999999999999e-05 2024-09-13T09:12:24.290 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.291 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.291 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.291 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:24.291 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472991+0000", 2024-09-13T09:12:24.291 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.710981107999999, 2024-09-13T09:12:24.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071846899999999997, 2024-09-13T09:12:24.291 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.291 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.291 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.291 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.291 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.292 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 117 2024-09-13T09:12:24.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.292 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472991+0000", 2024-09-13T09:12:24.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472991+0000", 2024-09-13T09:12:24.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472990+0000", 2024-09-13T09:12:24.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:24.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472994+0000", 2024-09-13T09:12:24.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4870000000000004e-06 2024-09-13T09:12:24.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.473001+0000", 2024-09-13T09:12:24.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:12:24.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.473005+0000", 2024-09-13T09:12:24.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-13T09:12:24.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479356+0000", 2024-09-13T09:12:24.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063509869999999998 2024-09-13T09:12:24.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479407+0000", 2024-09-13T09:12:24.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0349999999999997e-05 2024-09-13T09:12:24.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479488+0000", 2024-09-13T09:12:24.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0922999999999998e-05 2024-09-13T09:12:24.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479942+0000", 2024-09-13T09:12:24.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045411799999999999 2024-09-13T09:12:24.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.480136+0000", 2024-09-13T09:12:24.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000193909 2024-09-13T09:12:24.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.480158+0000", 2024-09-13T09:12:24.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2427999999999999e-05 2024-09-13T09:12:24.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.480175+0000", 2024-09-13T09:12:24.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7071e-05 2024-09-13T09:12:24.297 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.297 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.297 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.297 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:24.297 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:24.297 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.331633394000001, 2024-09-13T09:12:24.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066097600000000001, 2024-09-13T09:12:24.297 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.297 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.297 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.298 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.298 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.298 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 140 2024-09-13T09:12:24.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.298 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:24.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:24.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:24.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:24.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852387+0000", 2024-09-13T09:12:24.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9191999999999999e-05 2024-09-13T09:12:24.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852392+0000", 2024-09-13T09:12:24.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7299999999999996e-06 2024-09-13T09:12:24.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852395+0000", 2024-09-13T09:12:24.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9979999999999999e-06 2024-09-13T09:12:24.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857443+0000", 2024-09-13T09:12:24.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005048075 2024-09-13T09:12:24.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857503+0000", 2024-09-13T09:12:24.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9577999999999998e-05 2024-09-13T09:12:24.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857571+0000", 2024-09-13T09:12:24.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8320999999999999e-05 2024-09-13T09:12:24.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858400+0000", 2024-09-13T09:12:24.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00082945100000000004 2024-09-13T09:12:24.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858910+0000", 2024-09-13T09:12:24.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050916999999999998 2024-09-13T09:12:24.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858931+0000", 2024-09-13T09:12:24.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1742e-05 2024-09-13T09:12:24.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858948+0000", 2024-09-13T09:12:24.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6714999999999998e-05 2024-09-13T09:12:24.303 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.303 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.303 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.303 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:24.303 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.852399+0000", 2024-09-13T09:12:24.303 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.331573082999999, 2024-09-13T09:12:24.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068470969999999999, 2024-09-13T09:12:24.303 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.303 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.303 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.304 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:24.304 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:24.304 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 141 2024-09-13T09:12:24.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.304 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852399+0000", 2024-09-13T09:12:24.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852399+0000", 2024-09-13T09:12:24.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852398+0000", 2024-09-13T09:12:24.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852400+0000", 2024-09-13T09:12:24.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:12:24.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852404+0000", 2024-09-13T09:12:24.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:12:24.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852406+0000", 2024-09-13T09:12:24.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4559999999999999e-06 2024-09-13T09:12:24.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857928+0000", 2024-09-13T09:12:24.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005521593 2024-09-13T09:12:24.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857968+0000", 2024-09-13T09:12:24.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0586000000000001e-05 2024-09-13T09:12:24.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858029+0000", 2024-09-13T09:12:24.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1140000000000001e-05 2024-09-13T09:12:24.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859187+0000", 2024-09-13T09:12:24.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001157969 2024-09-13T09:12:24.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859198+0000", 2024-09-13T09:12:24.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1028e-05 2024-09-13T09:12:24.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859227+0000", 2024-09-13T09:12:24.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8370000000000001e-05 2024-09-13T09:12:24.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859246+0000", 2024-09-13T09:12:24.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8961e-05 2024-09-13T09:12:24.308 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.309 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.309 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.309 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.309 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.309 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:24.309 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-13T09:12:24.310 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T09:12:24.311 INFO:teuthology.orchestra.run.smithi157.stdout:550 read 0x6ad6800 thru 0x6ad88a0 (0x20a1 bytes) 2024-09-13T09:12:24.311 INFO:teuthology.orchestra.run.smithi157.stdout:551 trunc from 0xbd24800 to 0x6b24000 2024-09-13T09:12:24.312 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:24.312 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:24.308+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:24.359 INFO:teuthology.orchestra.run.smithi157.stdout:557 trunc from 0x6b24000 to 0xcca200 2024-09-13T09:12:24.388 INFO:teuthology.orchestra.run.smithi157.stdout:558 clone 4 order 21 su 32768 sc 8 2024-09-13T09:12:24.411 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:24.410+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:24.428 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:12:24.457 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:24.457 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:24.457 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:24.458 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:24.466 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:24.507 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone3 from 0xcca200 (overlap 0xcca200) to 0x51da00 2024-09-13T09:12:24.511 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:24.509+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:24.574 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:24.574 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:24.574 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:24.574 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:24.574 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:24.582 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:24.612 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:24.611+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:24.712 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:24.711+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:24.728 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:24.728 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:24.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:24.728 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:24.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.728 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:12:24.728 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:15.368781+0000", 2024-09-13T09:12:24.729 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3574731979999992, 2024-09-13T09:12:24.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038167510000000002, 2024-09-13T09:12:24.729 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.729 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.729 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.729 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:12:24.729 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:12:24.729 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 7 2024-09-13T09:12:24.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.729 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368781+0000", 2024-09-13T09:12:24.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368781+0000", 2024-09-13T09:12:24.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368778+0000", 2024-09-13T09:12:24.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999971 2024-09-13T09:12:24.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368796+0000", 2024-09-13T09:12:24.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8148999999999999e-05 2024-09-13T09:12:24.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368828+0000", 2024-09-13T09:12:24.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2066000000000001e-05 2024-09-13T09:12:24.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368846+0000", 2024-09-13T09:12:24.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7759999999999999e-05 2024-09-13T09:12:24.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368958+0000", 2024-09-13T09:12:24.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112074 2024-09-13T09:12:24.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:24.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.368969+0000", 2024-09-13T09:12:24.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.094e-05 2024-09-13T09:12:24.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.371161+0000", 2024-09-13T09:12:24.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021919869999999998 2024-09-13T09:12:24.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.371274+0000", 2024-09-13T09:12:24.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011297400000000001 2024-09-13T09:12:24.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:24.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.371386+0000", 2024-09-13T09:12:24.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112348 2024-09-13T09:12:24.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372324+0000", 2024-09-13T09:12:24.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00093778700000000004 2024-09-13T09:12:24.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372516+0000", 2024-09-13T09:12:24.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019215700000000001 2024-09-13T09:12:24.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372556+0000", 2024-09-13T09:12:24.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9919999999999997e-05 2024-09-13T09:12:24.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:15.372598+0000", 2024-09-13T09:12:24.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1495000000000002e-05 2024-09-13T09:12:24.735 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.735 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.735 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.735 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:12:24.735 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:16.378424+0000", 2024-09-13T09:12:24.736 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.3478303329999992, 2024-09-13T09:12:24.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061107690000000003, 2024-09-13T09:12:24.736 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.736 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.736 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.736 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:12:24.736 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:12:24.736 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9 2024-09-13T09:12:24.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.736 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378424+0000", 2024-09-13T09:12:24.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378424+0000", 2024-09-13T09:12:24.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378422+0000", 2024-09-13T09:12:24.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:24.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378429+0000", 2024-09-13T09:12:24.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6100000000000002e-06 2024-09-13T09:12:24.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378441+0000", 2024-09-13T09:12:24.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1581e-05 2024-09-13T09:12:24.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378449+0000", 2024-09-13T09:12:24.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2679999999999994e-06 2024-09-13T09:12:24.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378496+0000", 2024-09-13T09:12:24.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7286000000000001e-05 2024-09-13T09:12:24.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:24.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378501+0000", 2024-09-13T09:12:24.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0980000000000001e-06 2024-09-13T09:12:24.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.380081+0000", 2024-09-13T09:12:24.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015801879999999999 2024-09-13T09:12:24.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.380177+0000", 2024-09-13T09:12:24.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5311999999999998e-05 2024-09-13T09:12:24.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:24.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.380299+0000", 2024-09-13T09:12:24.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000121921 2024-09-13T09:12:24.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.381082+0000", 2024-09-13T09:12:24.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00078348100000000004 2024-09-13T09:12:24.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384464+0000", 2024-09-13T09:12:24.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033814349999999999 2024-09-13T09:12:24.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384495+0000", 2024-09-13T09:12:24.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1554999999999999e-05 2024-09-13T09:12:24.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384534+0000", 2024-09-13T09:12:24.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9320999999999998e-05 2024-09-13T09:12:24.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.742 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.743 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1499 3.11 3:8ebbb892:::rbd_data.11238d3f93be.0000000000000036:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:24.743 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.136984+0000", 2024-09-13T09:12:24.743 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.589270484, 2024-09-13T09:12:24.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064184419999999999, 2024-09-13T09:12:24.743 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.743 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:24.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:24.743 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1499 2024-09-13T09:12:24.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.744 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.136984+0000", 2024-09-13T09:12:24.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.136984+0000", 2024-09-13T09:12:24.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.136983+0000", 2024-09-13T09:12:24.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.136987+0000", 2024-09-13T09:12:24.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9659999999999998e-06 2024-09-13T09:12:24.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.137001+0000", 2024-09-13T09:12:24.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3832000000000001e-05 2024-09-13T09:12:24.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.137007+0000", 2024-09-13T09:12:24.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8549999999999998e-06 2024-09-13T09:12:24.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.137060+0000", 2024-09-13T09:12:24.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3171999999999998e-05 2024-09-13T09:12:24.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.137141+0000", 2024-09-13T09:12:24.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1116000000000001e-05 2024-09-13T09:12:24.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.137246+0000", 2024-09-13T09:12:24.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010518099999999999 2024-09-13T09:12:24.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.138787+0000", 2024-09-13T09:12:24.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015407789999999999 2024-09-13T09:12:24.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.143355+0000", 2024-09-13T09:12:24.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045682359999999998 2024-09-13T09:12:24.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.143381+0000", 2024-09-13T09:12:24.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5585999999999999e-05 2024-09-13T09:12:24.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.143402+0000", 2024-09-13T09:12:24.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1154000000000001e-05 2024-09-13T09:12:24.748 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.748 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.749 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.749 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4483.0:291 3.3 3:c3e921d9:::rbd_data.1123ffac6bc2.00000000000000ed:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:24.749 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.778379+0000", 2024-09-13T09:12:24.749 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.9478749500000001, 2024-09-13T09:12:24.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0048318249999999997, 2024-09-13T09:12:24.749 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.749 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.749 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.749 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4483", 2024-09-13T09:12:24.749 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:24.749 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 291 2024-09-13T09:12:24.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.750 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778379+0000", 2024-09-13T09:12:24.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778379+0000", 2024-09-13T09:12:24.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778379+0000", 2024-09-13T09:12:24.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778382+0000", 2024-09-13T09:12:24.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1709999999999998e-06 2024-09-13T09:12:24.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778388+0000", 2024-09-13T09:12:24.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6529999999999997e-06 2024-09-13T09:12:24.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778394+0000", 2024-09-13T09:12:24.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-06 2024-09-13T09:12:24.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778444+0000", 2024-09-13T09:12:24.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0303000000000003e-05 2024-09-13T09:12:24.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778524+0000", 2024-09-13T09:12:24.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0187999999999999e-05 2024-09-13T09:12:24.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778589+0000", 2024-09-13T09:12:24.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4488999999999999e-05 2024-09-13T09:12:24.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.779159+0000", 2024-09-13T09:12:24.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00057035500000000004 2024-09-13T09:12:24.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.783165+0000", 2024-09-13T09:12:24.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0040055769999999997 2024-09-13T09:12:24.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.783189+0000", 2024-09-13T09:12:24.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4573999999999999e-05 2024-09-13T09:12:24.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.783211+0000", 2024-09-13T09:12:24.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1715000000000001e-05 2024-09-13T09:12:24.755 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.755 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.755 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.755 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4483.0:1045 3.3 3:c48fd897:::rbd_data.1123ffac6bc2.00000000000000a1:head [set-alloc-hint object_size 1048576 write_size 1048576,write 77824~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:24.755 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:20.135620+0000", 2024-09-13T09:12:24.755 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.590634219, 2024-09-13T09:12:24.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043927250000000001, 2024-09-13T09:12:24.755 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.755 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.755 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.755 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4483", 2024-09-13T09:12:24.755 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:24.756 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1045 2024-09-13T09:12:24.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.756 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.135620+0000", 2024-09-13T09:12:24.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.135620+0000", 2024-09-13T09:12:24.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.135619+0000", 2024-09-13T09:12:24.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:24.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.135684+0000", 2024-09-13T09:12:24.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4641999999999995e-05 2024-09-13T09:12:24.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.135693+0000", 2024-09-13T09:12:24.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7750000000000004e-06 2024-09-13T09:12:24.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.135700+0000", 2024-09-13T09:12:24.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3239999999999999e-06 2024-09-13T09:12:24.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.135746+0000", 2024-09-13T09:12:24.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6448e-05 2024-09-13T09:12:24.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.135883+0000", 2024-09-13T09:12:24.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013641599999999999 2024-09-13T09:12:24.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.136025+0000", 2024-09-13T09:12:24.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014257899999999999 2024-09-13T09:12:24.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.136598+0000", 2024-09-13T09:12:24.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0005731 2024-09-13T09:12:24.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.139962+0000", 2024-09-13T09:12:24.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033636989999999999 2024-09-13T09:12:24.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.139991+0000", 2024-09-13T09:12:24.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8615999999999998e-05 2024-09-13T09:12:24.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140013+0000", 2024-09-13T09:12:24.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1874000000000001e-05 2024-09-13T09:12:24.761 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.761 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.761 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.761 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4483.0:1054 3.3 3:c48fd897:::rbd_data.1123ffac6bc2.00000000000000a1:head [write 159744~27648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:24.761 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:20.140196+0000", 2024-09-13T09:12:24.761 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.5860582780000003, 2024-09-13T09:12:24.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0047523959999999999, 2024-09-13T09:12:24.761 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.761 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.761 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.761 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4483", 2024-09-13T09:12:24.762 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:24.762 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1054 2024-09-13T09:12:24.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.762 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140196+0000", 2024-09-13T09:12:24.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140196+0000", 2024-09-13T09:12:24.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.763 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140195+0000", 2024-09-13T09:12:24.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140216+0000", 2024-09-13T09:12:24.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0672000000000001e-05 2024-09-13T09:12:24.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140223+0000", 2024-09-13T09:12:24.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5050000000000001e-06 2024-09-13T09:12:24.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140228+0000", 2024-09-13T09:12:24.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4950000000000002e-06 2024-09-13T09:12:24.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140259+0000", 2024-09-13T09:12:24.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0964999999999999e-05 2024-09-13T09:12:24.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140304+0000", 2024-09-13T09:12:24.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4922000000000001e-05 2024-09-13T09:12:24.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:24.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140377+0000", 2024-09-13T09:12:24.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3819000000000006e-05 2024-09-13T09:12:24.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.143410+0000", 2024-09-13T09:12:24.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0030321290000000002 2024-09-13T09:12:24.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.144868+0000", 2024-09-13T09:12:24.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001458097 2024-09-13T09:12:24.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.144914+0000", 2024-09-13T09:12:24.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6715000000000003e-05 2024-09-13T09:12:24.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.144948+0000", 2024-09-13T09:12:24.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3760000000000002e-05 2024-09-13T09:12:24.767 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.767 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.767 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.768 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4483.0:1105 3.a 3:5043962a:::rbd_data.1123ffac6bc2.00000000000000b7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:24.768 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:20.161199+0000", 2024-09-13T09:12:24.768 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.5650546290000005, 2024-09-13T09:12:24.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004111216, 2024-09-13T09:12:24.768 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.768 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.768 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.768 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4483", 2024-09-13T09:12:24.768 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:24.768 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1105 2024-09-13T09:12:24.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.769 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.161199+0000", 2024-09-13T09:12:24.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.161199+0000", 2024-09-13T09:12:24.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.161199+0000", 2024-09-13T09:12:24.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.161201+0000", 2024-09-13T09:12:24.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1150000000000001e-06 2024-09-13T09:12:24.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.161207+0000", 2024-09-13T09:12:24.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.156e-06 2024-09-13T09:12:24.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.161212+0000", 2024-09-13T09:12:24.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9110000000000001e-06 2024-09-13T09:12:24.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.161786+0000", 2024-09-13T09:12:24.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00057375000000000004 2024-09-13T09:12:24.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.161856+0000", 2024-09-13T09:12:24.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9894999999999994e-05 2024-09-13T09:12:24.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:24.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.161955+0000", 2024-09-13T09:12:24.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9543999999999994e-05 2024-09-13T09:12:24.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.164829+0000", 2024-09-13T09:12:24.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0028739299999999998 2024-09-13T09:12:24.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.165263+0000", 2024-09-13T09:12:24.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000433767 2024-09-13T09:12:24.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.165284+0000", 2024-09-13T09:12:24.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0523e-05 2024-09-13T09:12:24.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.165311+0000", 2024-09-13T09:12:24.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7016e-05 2024-09-13T09:12:24.774 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.774 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.774 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.774 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4483.0:1320 3.14 3:2b7a8de5:::rbd_data.1123ffac6bc2.00000000000000a9:head [set-alloc-hint object_size 1048576 write_size 1048576,write 790528~61440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:12:24.774 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:20.271531+0000", 2024-09-13T09:12:24.774 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.4547228370000003, 2024-09-13T09:12:24.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0049277319999999998, 2024-09-13T09:12:24.774 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.774 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4483", 2024-09-13T09:12:24.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:24.775 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1320 2024-09-13T09:12:24.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.775 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271531+0000", 2024-09-13T09:12:24.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271531+0000", 2024-09-13T09:12:24.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271530+0000", 2024-09-13T09:12:24.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:24.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271586+0000", 2024-09-13T09:12:24.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6017999999999998e-05 2024-09-13T09:12:24.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271596+0000", 2024-09-13T09:12:24.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8949999999999993e-06 2024-09-13T09:12:24.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271601+0000", 2024-09-13T09:12:24.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9760000000000003e-06 2024-09-13T09:12:24.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271643+0000", 2024-09-13T09:12:24.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1934000000000002e-05 2024-09-13T09:12:24.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271776+0000", 2024-09-13T09:12:24.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013290599999999999 2024-09-13T09:12:24.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:24.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271911+0000", 2024-09-13T09:12:24.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000134566 2024-09-13T09:12:24.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:24.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.276207+0000", 2024-09-13T09:12:24.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042967819999999999 2024-09-13T09:12:24.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:24.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.276367+0000", 2024-09-13T09:12:24.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00015928200000000001 2024-09-13T09:12:24.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.276419+0000", 2024-09-13T09:12:24.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2580000000000001e-05 2024-09-13T09:12:24.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.276459+0000", 2024-09-13T09:12:24.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9573999999999998e-05 2024-09-13T09:12:24.780 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.780 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.780 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.780 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4483.0:1322 3.1c e28/21 3:3b3c2393:::rbd_data.1123ffac6bc2.0000000000000055:head v 28'96, mlcod=27'94)", 2024-09-13T09:12:24.780 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:20.272115+0000", 2024-09-13T09:12:24.780 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.4541392159999997, 2024-09-13T09:12:24.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043149870000000002, 2024-09-13T09:12:24.780 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.781 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:24.781 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.272115+0000", 2024-09-13T09:12:24.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.272115+0000", 2024-09-13T09:12:24.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.272115+0000", 2024-09-13T09:12:24.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:24.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.272127+0000", 2024-09-13T09:12:24.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.207e-05 2024-09-13T09:12:24.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.272134+0000", 2024-09-13T09:12:24.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5889999999999997e-06 2024-09-13T09:12:24.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.272139+0000", 2024-09-13T09:12:24.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.386e-06 2024-09-13T09:12:24.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.272170+0000", 2024-09-13T09:12:24.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0927999999999999e-05 2024-09-13T09:12:24.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.272184+0000", 2024-09-13T09:12:24.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4039999999999999e-05 2024-09-13T09:12:24.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:24.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.276409+0000", 2024-09-13T09:12:24.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042249159999999996 2024-09-13T09:12:24.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.276430+0000", 2024-09-13T09:12:24.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1257999999999999e-05 2024-09-13T09:12:24.784 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.784 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.784 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.785 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:441 3.14 3:2dfb6081:::rbd_data.11238839e366.0000000000000001:6 [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:24.785 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.283321+0000", 2024-09-13T09:12:24.785 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.44293292099999998, 2024-09-13T09:12:24.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038695859999999999, 2024-09-13T09:12:24.785 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.785 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:24.785 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.785 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:24.785 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:24.785 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 441 2024-09-13T09:12:24.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.786 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283321+0000", 2024-09-13T09:12:24.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283321+0000", 2024-09-13T09:12:24.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283321+0000", 2024-09-13T09:12:24.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283323+0000", 2024-09-13T09:12:24.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1129999999999999e-06 2024-09-13T09:12:24.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283329+0000", 2024-09-13T09:12:24.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1310000000000003e-06 2024-09-13T09:12:24.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283335+0000", 2024-09-13T09:12:24.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-13T09:12:24.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287086+0000", 2024-09-13T09:12:24.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0037511139999999998 2024-09-13T09:12:24.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287140+0000", 2024-09-13T09:12:24.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3879999999999999e-05 2024-09-13T09:12:24.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287191+0000", 2024-09-13T09:12:24.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1125000000000003e-05 2024-09-13T09:12:24.789 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.789 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.789 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.789 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:442 3.14 3:2dfb6081:::rbd_data.11238839e366.0000000000000001:6 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:24.789 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.283339+0000", 2024-09-13T09:12:24.790 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.44291539499999999, 2024-09-13T09:12:24.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039746900000000003, 2024-09-13T09:12:24.790 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.790 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:24.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:24.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:24.790 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 442 2024-09-13T09:12:24.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.790 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283339+0000", 2024-09-13T09:12:24.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283339+0000", 2024-09-13T09:12:24.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283338+0000", 2024-09-13T09:12:24.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283340+0000", 2024-09-13T09:12:24.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0480000000000001e-06 2024-09-13T09:12:24.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283346+0000", 2024-09-13T09:12:24.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1900000000000003e-06 2024-09-13T09:12:24.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283351+0000", 2024-09-13T09:12:24.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3689999999999998e-06 2024-09-13T09:12:24.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287211+0000", 2024-09-13T09:12:24.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038601550000000001 2024-09-13T09:12:24.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287251+0000", 2024-09-13T09:12:24.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0364999999999997e-05 2024-09-13T09:12:24.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287313+0000", 2024-09-13T09:12:24.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1932000000000003e-05 2024-09-13T09:12:24.794 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.794 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.794 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.794 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:443 3.14 3:2dfb6081:::rbd_data.11238839e366.0000000000000001:6 [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:24.794 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.283355+0000", 2024-09-13T09:12:24.794 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.44289922300000001, 2024-09-13T09:12:24.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004192823, 2024-09-13T09:12:24.794 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.794 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:24.794 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.794 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:24.795 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:24.795 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 443 2024-09-13T09:12:24.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.795 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283355+0000", 2024-09-13T09:12:24.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283355+0000", 2024-09-13T09:12:24.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283355+0000", 2024-09-13T09:12:24.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283356+0000", 2024-09-13T09:12:24.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9319999999999999e-06 2024-09-13T09:12:24.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283362+0000", 2024-09-13T09:12:24.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7969999999999997e-06 2024-09-13T09:12:24.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283367+0000", 2024-09-13T09:12:24.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3789999999999999e-06 2024-09-13T09:12:24.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287353+0000", 2024-09-13T09:12:24.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039868200000000003 2024-09-13T09:12:24.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287438+0000", 2024-09-13T09:12:24.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4196999999999994e-05 2024-09-13T09:12:24.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287548+0000", 2024-09-13T09:12:24.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000109991 2024-09-13T09:12:24.798 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.798 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.798 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.798 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:444 3.14 3:2dfb6081:::rbd_data.11238839e366.0000000000000001:6 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:24.799 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.283371+0000", 2024-09-13T09:12:24.799 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.44288338300000002, 2024-09-13T09:12:24.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043761690000000001, 2024-09-13T09:12:24.799 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.799 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:24.799 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.799 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:24.799 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:24.799 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 444 2024-09-13T09:12:24.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.799 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283371+0000", 2024-09-13T09:12:24.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283371+0000", 2024-09-13T09:12:24.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283370+0000", 2024-09-13T09:12:24.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283372+0000", 2024-09-13T09:12:24.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.905e-06 2024-09-13T09:12:24.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283378+0000", 2024-09-13T09:12:24.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8300000000000001e-06 2024-09-13T09:12:24.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283382+0000", 2024-09-13T09:12:24.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.143e-06 2024-09-13T09:12:24.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287585+0000", 2024-09-13T09:12:24.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042028029999999997 2024-09-13T09:12:24.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287660+0000", 2024-09-13T09:12:24.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5339000000000005e-05 2024-09-13T09:12:24.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287747+0000", 2024-09-13T09:12:24.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6489999999999994e-05 2024-09-13T09:12:24.803 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.803 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.803 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.803 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:445 3.14 3:2dfb6081:::rbd_data.11238839e366.0000000000000001:6 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:24.803 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.283411+0000", 2024-09-13T09:12:24.803 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.44284326299999999, 2024-09-13T09:12:24.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0044893329999999999, 2024-09-13T09:12:24.803 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.804 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:24.804 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.804 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:24.804 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:24.804 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 445 2024-09-13T09:12:24.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.804 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283411+0000", 2024-09-13T09:12:24.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283411+0000", 2024-09-13T09:12:24.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283410+0000", 2024-09-13T09:12:24.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283413+0000", 2024-09-13T09:12:24.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0800000000000002e-06 2024-09-13T09:12:24.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283419+0000", 2024-09-13T09:12:24.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9340000000000003e-06 2024-09-13T09:12:24.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283424+0000", 2024-09-13T09:12:24.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5600000000000004e-06 2024-09-13T09:12:24.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287773+0000", 2024-09-13T09:12:24.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004349571 2024-09-13T09:12:24.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287840+0000", 2024-09-13T09:12:24.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6424e-05 2024-09-13T09:12:24.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287900+0000", 2024-09-13T09:12:24.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0340000000000002e-05 2024-09-13T09:12:24.807 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.808 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.808 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.808 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:486 3.9 3:901933f8:::rbd_data.11238839e366.0000000000000005:6 [sparse-read 8650752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:24.808 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.283675+0000", 2024-09-13T09:12:24.808 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.44257918099999999, 2024-09-13T09:12:24.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038391699999999998, 2024-09-13T09:12:24.808 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.808 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:24.808 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.808 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:24.808 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:24.808 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 486 2024-09-13T09:12:24.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.809 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283675+0000", 2024-09-13T09:12:24.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283675+0000", 2024-09-13T09:12:24.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283674+0000", 2024-09-13T09:12:24.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283677+0000", 2024-09-13T09:12:24.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5600000000000001e-06 2024-09-13T09:12:24.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283683+0000", 2024-09-13T09:12:24.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:12:24.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283687+0000", 2024-09-13T09:12:24.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:12:24.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287313+0000", 2024-09-13T09:12:24.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003625727 2024-09-13T09:12:24.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287410+0000", 2024-09-13T09:12:24.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7495000000000001e-05 2024-09-13T09:12:24.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287514+0000", 2024-09-13T09:12:24.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010354 2024-09-13T09:12:24.812 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.812 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.812 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.812 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:487 3.9 3:901933f8:::rbd_data.11238839e366.0000000000000005:6 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:24.812 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.283692+0000", 2024-09-13T09:12:24.813 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.44256235999999999, 2024-09-13T09:12:24.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039542889999999997, 2024-09-13T09:12:24.813 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.813 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:24.813 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.813 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:24.813 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:24.813 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 487 2024-09-13T09:12:24.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.813 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283692+0000", 2024-09-13T09:12:24.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283692+0000", 2024-09-13T09:12:24.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283691+0000", 2024-09-13T09:12:24.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283693+0000", 2024-09-13T09:12:24.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8449999999999999e-06 2024-09-13T09:12:24.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283699+0000", 2024-09-13T09:12:24.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.519e-06 2024-09-13T09:12:24.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283703+0000", 2024-09-13T09:12:24.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2860000000000001e-06 2024-09-13T09:12:24.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287552+0000", 2024-09-13T09:12:24.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038490120000000002 2024-09-13T09:12:24.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287594+0000", 2024-09-13T09:12:24.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2206000000000002e-05 2024-09-13T09:12:24.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287646+0000", 2024-09-13T09:12:24.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1764000000000001e-05 2024-09-13T09:12:24.817 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.817 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.817 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.817 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:488 3.9 3:901933f8:::rbd_data.11238839e366.0000000000000005:6 [sparse-read 11010048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:24.817 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.283708+0000", 2024-09-13T09:12:24.817 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.44254582399999998, 2024-09-13T09:12:24.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0041373590000000002, 2024-09-13T09:12:24.817 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.817 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:24.817 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.818 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:24.818 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:24.818 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 488 2024-09-13T09:12:24.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.818 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283708+0000", 2024-09-13T09:12:24.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283708+0000", 2024-09-13T09:12:24.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283708+0000", 2024-09-13T09:12:24.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283710+0000", 2024-09-13T09:12:24.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9860000000000001e-06 2024-09-13T09:12:24.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283715+0000", 2024-09-13T09:12:24.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4790000000000004e-06 2024-09-13T09:12:24.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283719+0000", 2024-09-13T09:12:24.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.658e-06 2024-09-13T09:12:24.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287681+0000", 2024-09-13T09:12:24.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039617380000000002 2024-09-13T09:12:24.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287754+0000", 2024-09-13T09:12:24.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3632000000000002e-05 2024-09-13T09:12:24.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287846+0000", 2024-09-13T09:12:24.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1198000000000003e-05 2024-09-13T09:12:24.821 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.821 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.821 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.821 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:489 3.9 3:901933f8:::rbd_data.11238839e366.0000000000000005:6 [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:24.822 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.283723+0000", 2024-09-13T09:12:24.822 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.44253063599999998, 2024-09-13T09:12:24.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043255059999999998, 2024-09-13T09:12:24.822 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.822 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:24.822 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.822 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:24.822 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:24.822 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 489 2024-09-13T09:12:24.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.822 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283723+0000", 2024-09-13T09:12:24.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283723+0000", 2024-09-13T09:12:24.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283723+0000", 2024-09-13T09:12:24.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283725+0000", 2024-09-13T09:12:24.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.863e-06 2024-09-13T09:12:24.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283730+0000", 2024-09-13T09:12:24.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5079999999999996e-06 2024-09-13T09:12:24.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283735+0000", 2024-09-13T09:12:24.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1690000000000002e-06 2024-09-13T09:12:24.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287882+0000", 2024-09-13T09:12:24.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0041472490000000004 2024-09-13T09:12:24.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287960+0000", 2024-09-13T09:12:24.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8646000000000006e-05 2024-09-13T09:12:24.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.288049+0000", 2024-09-13T09:12:24.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8399999999999994e-05 2024-09-13T09:12:24.826 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.826 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.826 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.826 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:490 3.9 3:901933f8:::rbd_data.11238839e366.0000000000000005:6 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:24.826 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.283739+0000", 2024-09-13T09:12:24.826 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.44251517499999998, 2024-09-13T09:12:24.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045407370000000004, 2024-09-13T09:12:24.826 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.826 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:24.826 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.826 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:24.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:24.827 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 490 2024-09-13T09:12:24.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.827 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283739+0000", 2024-09-13T09:12:24.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283739+0000", 2024-09-13T09:12:24.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283739+0000", 2024-09-13T09:12:24.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283740+0000", 2024-09-13T09:12:24.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8980000000000001e-06 2024-09-13T09:12:24.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283746+0000", 2024-09-13T09:12:24.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3970000000000002e-06 2024-09-13T09:12:24.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283750+0000", 2024-09-13T09:12:24.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9040000000000002e-06 2024-09-13T09:12:24.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.288085+0000", 2024-09-13T09:12:24.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043348659999999997 2024-09-13T09:12:24.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.288171+0000", 2024-09-13T09:12:24.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6200999999999997e-05 2024-09-13T09:12:24.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.288280+0000", 2024-09-13T09:12:24.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000108795 2024-09-13T09:12:24.830 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.830 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.830 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.830 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:491 3.9 3:901933f8:::rbd_data.11238839e366.0000000000000005:6 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:24.830 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.283754+0000", 2024-09-13T09:12:24.831 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.442499906, 2024-09-13T09:12:24.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004723054, 2024-09-13T09:12:24.831 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:24.831 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:24.831 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:24.831 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:24.831 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:24.831 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 491 2024-09-13T09:12:24.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.831 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:24.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:24.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283754+0000", 2024-09-13T09:12:24.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:24.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283754+0000", 2024-09-13T09:12:24.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:24.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:24.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283754+0000", 2024-09-13T09:12:24.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:24.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:24.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283756+0000", 2024-09-13T09:12:24.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.905e-06 2024-09-13T09:12:24.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:24.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283761+0000", 2024-09-13T09:12:24.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.383e-06 2024-09-13T09:12:24.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:24.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283765+0000", 2024-09-13T09:12:24.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1829999999999996e-06 2024-09-13T09:12:24.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:24.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.288303+0000", 2024-09-13T09:12:24.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045381680000000004 2024-09-13T09:12:24.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:24.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.288384+0000", 2024-09-13T09:12:24.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0315999999999995e-05 2024-09-13T09:12:24.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:24.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:24.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:24.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.288477+0000", 2024-09-13T09:12:24.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3405000000000004e-05 2024-09-13T09:12:24.834 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.835 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.835 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.835 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:24.835 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:24.835 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:24.836 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-13T09:12:24.837 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:24.837 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:24.834+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:24.882 INFO:teuthology.orchestra.run.smithi157.stdout:560 punch from 0x213a00 to 0x215c00, (0x2200 bytes) 2024-09-13T09:12:24.895 INFO:teuthology.orchestra.run.smithi157.stdout:562 read 0x8fde00 thru 0x90d51f (0xf720 bytes) 2024-09-13T09:12:24.898 INFO:teuthology.orchestra.run.smithi157.stdout:564 trunc from 0xcca200 to 0x22cea00 2024-09-13T09:12:24.907 INFO:teuthology.orchestra.run.smithi157.stdout:565 read 0xb8a600 thru 0xb97b6c (0xd56d bytes) 2024-09-13T09:12:24.915 INFO:teuthology.orchestra.run.smithi157.stdout:566 trunc from 0x22cea00 to 0x10ea000 2024-09-13T09:12:24.924 INFO:teuthology.orchestra.run.smithi157.stdout:567 punch from 0x5e7c00 to 0x5ede00, (0x6200 bytes) 2024-09-13T09:12:24.931 INFO:teuthology.orchestra.run.smithi157.stdout:570 punch from 0x1089a00 to 0x1095e00, (0xc400 bytes) 2024-09-13T09:12:24.932 INFO:teuthology.orchestra.run.smithi157.stdout:571 trunc from 0x10ea000 to 0x2fda00 2024-09-13T09:12:24.937 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:24.935+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:24.973 INFO:teuthology.orchestra.run.smithi157.stdout:573 read 0x2ca800 thru 0x2d664b (0xbe4c bytes) 2024-09-13T09:12:24.988 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:24.988 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:24.988 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:24.988 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:24.997 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:25.004 INFO:teuthology.orchestra.run.smithi157.stdout:575 write 0xc76f400 thru 0xc77ac1c (0xb81d bytes) 2024-09-13T09:12:25.010 INFO:teuthology.orchestra.run.smithi157.stdout:576 trunc from 0xc77ae00 to 0x7372e00 2024-09-13T09:12:25.035 INFO:teuthology.orchestra.run.smithi157.stdout:577 trunc from 0x7372e00 to 0x545be00 2024-09-13T09:12:25.039 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:25.037+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:25.063 INFO:teuthology.orchestra.run.smithi157.stdout:579 trunc from 0x545be00 to 0xd304a00 2024-09-13T09:12:25.074 INFO:teuthology.orchestra.run.smithi157.stdout:582 punch from 0x7d4600 to 0x7e3a00, (0xf400 bytes) 2024-09-13T09:12:25.079 INFO:teuthology.orchestra.run.smithi157.stdout:584 write 0x60a800 thru 0x614257 (0x9a58 bytes) 2024-09-13T09:12:25.081 INFO:teuthology.orchestra.run.smithi157.stdout:586 punch from 0xc087800 to 0xc094800, (0xd000 bytes) 2024-09-13T09:12:25.084 INFO:teuthology.orchestra.run.smithi157.stdout:587 trunc from 0xd304a00 to 0xac74200 2024-09-13T09:12:25.101 INFO:teuthology.orchestra.run.smithi157.stdout:588 flatten 2024-09-13T09:12:25.113 INFO:teuthology.orchestra.run.smithi157.stdout:590 read 0x7569000 thru 0x757533b (0xc33c bytes) 2024-09-13T09:12:25.118 INFO:teuthology.orchestra.run.smithi157.stdout:595 trunc from 0xac74200 to 0xe99be00 2024-09-13T09:12:25.127 INFO:teuthology.orchestra.run.smithi157.stdout:597 read 0x56a2a00 thru 0x56a8a1c (0x601d bytes) 2024-09-13T09:12:25.134 INFO:teuthology.orchestra.run.smithi157.stdout:598 write 0x878f600 thru 0x8794f47 (0x5948 bytes) 2024-09-13T09:12:25.135 INFO:teuthology.orchestra.run.smithi157.stdout:599 read 0x7ef1200 thru 0x7ef2c2a (0x1a2b bytes) 2024-09-13T09:12:25.135 INFO:teuthology.orchestra.run.smithi157.stdout:600 read 0x3b4c000 thru 0x3b52c34 (0x6c35 bytes) 2024-09-13T09:12:25.135 INFO:teuthology.orchestra.run.smithi157.stdout:602 read 0x6290600 thru 0x629caf6 (0xc4f7 bytes) 2024-09-13T09:12:25.135 INFO:teuthology.orchestra.run.smithi157.stdout:603 write 0x5154400 thru 0x5163523 (0xf124 bytes) 2024-09-13T09:12:25.135 INFO:teuthology.orchestra.run.smithi157.stdout:605 read 0x44c6800 thru 0x44d2d81 (0xc582 bytes) 2024-09-13T09:12:25.138 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:25.136+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:25.142 INFO:teuthology.orchestra.run.smithi157.stdout:606 trunc from 0xe99be00 to 0x7f39400 2024-09-13T09:12:25.144 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:25.144 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:25.144 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:25.144 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:25.145 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:25.154 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:25.165 INFO:teuthology.orchestra.run.smithi157.stdout:607 write 0x5c7c200 thru 0x5c8a8fb (0xe6fc bytes) 2024-09-13T09:12:25.172 INFO:teuthology.orchestra.run.smithi157.stdout:609 read 0x1d3f000 thru 0x1d467e6 (0x77e7 bytes) 2024-09-13T09:12:25.172 INFO:teuthology.orchestra.run.smithi157.stdout:612 read 0x27f9c00 thru 0x27fba8e (0x1e8f bytes) 2024-09-13T09:12:25.172 INFO:teuthology.orchestra.run.smithi157.stdout:613 write 0xfad600 thru 0xfb116f (0x3b70 bytes) 2024-09-13T09:12:25.205 INFO:teuthology.orchestra.run.smithi157.stdout:615 write 0xea8b000 thru 0xea95163 (0xa164 bytes) 2024-09-13T09:12:25.205 INFO:teuthology.orchestra.run.smithi157.stdout:619 read 0x63ca800 thru 0x63d5649 (0xae4a bytes) 2024-09-13T09:12:25.206 INFO:teuthology.orchestra.run.smithi157.stdout:622 read 0x83a000 thru 0x83f5d4 (0x55d5 bytes) 2024-09-13T09:12:25.206 INFO:teuthology.orchestra.run.smithi157.stdout:624 read 0x75ffc00 thru 0x760a8a3 (0xaca4 bytes) 2024-09-13T09:12:25.213 INFO:teuthology.orchestra.run.smithi157.stdout:625 trunc from 0xea95200 to 0x838f800 2024-09-13T09:12:25.235 INFO:teuthology.orchestra.run.smithi157.stdout:630 punch from 0x7274a00 to 0x727de00, (0x9400 bytes) 2024-09-13T09:12:25.239 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:25.238+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:25.242 INFO:teuthology.orchestra.run.smithi157.stdout:631 read 0x5a29400 thru 0x5a37296 (0xde97 bytes) 2024-09-13T09:12:25.242 INFO:teuthology.orchestra.run.smithi157.stdout:633 punch from 0x18e7c00 to 0x18f5c00, (0xe000 bytes) 2024-09-13T09:12:25.250 INFO:teuthology.orchestra.run.smithi157.stdout:635 trunc from 0x838f800 to 0xe67a000 2024-09-13T09:12:25.262 INFO:teuthology.orchestra.run.smithi157.stdout:636 punch from 0x61cea00 to 0x61da400, (0xba00 bytes) 2024-09-13T09:12:25.264 INFO:teuthology.orchestra.run.smithi157.stdout:638 punch from 0x2af8c00 to 0x2b08200, (0xf600 bytes) 2024-09-13T09:12:25.273 INFO:teuthology.orchestra.run.smithi157.stdout:639 trunc from 0xe67a000 to 0x6bd4a00 2024-09-13T09:12:25.294 INFO:teuthology.orchestra.run.smithi157.stdout:640 trunc from 0x6bd4a00 to 0xb8f4400 2024-09-13T09:12:25.303 INFO:teuthology.orchestra.run.smithi157.stdout:642 trunc from 0xb8f4400 to 0xbc12e00 2024-09-13T09:12:25.306 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:25.306 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:25.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:25.306 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:25.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.306 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4221.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:12:25.306 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:04.320224+0000", 2024-09-13T09:12:25.307 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.983736631999999, 2024-09-13T09:12:25.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010683929, 2024-09-13T09:12:25.307 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.307 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.307 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.307 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4221", 2024-09-13T09:12:25.307 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.44:0/558773684", 2024-09-13T09:12:25.307 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1 2024-09-13T09:12:25.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.307 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320224+0000", 2024-09-13T09:12:25.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320224+0000", 2024-09-13T09:12:25.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320220+0000", 2024-09-13T09:12:25.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999967 2024-09-13T09:12:25.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320230+0000", 2024-09-13T09:12:25.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0275000000000001e-05 2024-09-13T09:12:25.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320241+0000", 2024-09-13T09:12:25.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0154e-05 2024-09-13T09:12:25.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320256+0000", 2024-09-13T09:12:25.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5068999999999999e-05 2024-09-13T09:12:25.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320328+0000", 2024-09-13T09:12:25.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2507000000000002e-05 2024-09-13T09:12:25.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:25.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320336+0000", 2024-09-13T09:12:25.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:12:25.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.322475+0000", 2024-09-13T09:12:25.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021399430000000001 2024-09-13T09:12:25.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:25.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.322484+0000", 2024-09-13T09:12:25.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5779999999999999e-06 2024-09-13T09:12:25.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.325065+0000", 2024-09-13T09:12:25.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002581323 2024-09-13T09:12:25.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.325211+0000", 2024-09-13T09:12:25.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014534899999999999 2024-09-13T09:12:25.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:25.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.325374+0000", 2024-09-13T09:12:25.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000162898 2024-09-13T09:12:25.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.326528+0000", 2024-09-13T09:12:25.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001154415 2024-09-13T09:12:25.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.330811+0000", 2024-09-13T09:12:25.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042825090000000003 2024-09-13T09:12:25.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.330862+0000", 2024-09-13T09:12:25.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1430000000000001e-05 2024-09-13T09:12:25.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.330908+0000", 2024-09-13T09:12:25.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5636000000000001e-05 2024-09-13T09:12:25.314 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.314 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.314 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.314 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:12:25.314 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:16.385493+0000", 2024-09-13T09:12:25.314 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.9184668259999995, 2024-09-13T09:12:25.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063914480000000001, 2024-09-13T09:12:25.314 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.314 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.314 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.314 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:12:25.315 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:12:25.315 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 10 2024-09-13T09:12:25.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.315 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385493+0000", 2024-09-13T09:12:25.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385493+0000", 2024-09-13T09:12:25.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385492+0000", 2024-09-13T09:12:25.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:25.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385498+0000", 2024-09-13T09:12:25.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2219999999999999e-06 2024-09-13T09:12:25.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385509+0000", 2024-09-13T09:12:25.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1207e-05 2024-09-13T09:12:25.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385519+0000", 2024-09-13T09:12:25.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8560000000000002e-06 2024-09-13T09:12:25.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385560+0000", 2024-09-13T09:12:25.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1223000000000002e-05 2024-09-13T09:12:25.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:25.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385565+0000", 2024-09-13T09:12:25.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9409999999999998e-06 2024-09-13T09:12:25.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388159+0000", 2024-09-13T09:12:25.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0025934 2024-09-13T09:12:25.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388261+0000", 2024-09-13T09:12:25.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102799 2024-09-13T09:12:25.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:25.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388406+0000", 2024-09-13T09:12:25.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000144857 2024-09-13T09:12:25.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.389373+0000", 2024-09-13T09:12:25.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00096632900000000004 2024-09-13T09:12:25.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.391797+0000", 2024-09-13T09:12:25.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0024246580000000001 2024-09-13T09:12:25.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.391847+0000", 2024-09-13T09:12:25.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9811999999999997e-05 2024-09-13T09:12:25.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.391885+0000", 2024-09-13T09:12:25.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7796999999999997e-05 2024-09-13T09:12:25.320 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.320 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.320 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.321 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1459 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:25.321 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.121753+0000", 2024-09-13T09:12:25.321 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.1822077699999998, 2024-09-13T09:12:25.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065126389999999998, 2024-09-13T09:12:25.321 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.321 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.321 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.321 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:25.321 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:25.321 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1459 2024-09-13T09:12:25.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.322 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121753+0000", 2024-09-13T09:12:25.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121753+0000", 2024-09-13T09:12:25.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121752+0000", 2024-09-13T09:12:25.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:25.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121754+0000", 2024-09-13T09:12:25.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9750000000000001e-06 2024-09-13T09:12:25.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121759+0000", 2024-09-13T09:12:25.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0239999999999997e-06 2024-09-13T09:12:25.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121762+0000", 2024-09-13T09:12:25.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8049999999999998e-06 2024-09-13T09:12:25.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122595+0000", 2024-09-13T09:12:25.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00083291799999999998 2024-09-13T09:12:25.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122697+0000", 2024-09-13T09:12:25.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101727 2024-09-13T09:12:25.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:25.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122961+0000", 2024-09-13T09:12:25.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026407399999999999 2024-09-13T09:12:25.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124336+0000", 2024-09-13T09:12:25.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0013751270000000001 2024-09-13T09:12:25.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128157+0000", 2024-09-13T09:12:25.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038212419999999999 2024-09-13T09:12:25.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128205+0000", 2024-09-13T09:12:25.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7722999999999997e-05 2024-09-13T09:12:25.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128265+0000", 2024-09-13T09:12:25.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.033e-05 2024-09-13T09:12:25.326 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.327 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.327 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.327 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1460 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:25.327 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.121842+0000", 2024-09-13T09:12:25.327 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.1821180269999996, 2024-09-13T09:12:25.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065154749999999997, 2024-09-13T09:12:25.327 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.327 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.327 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.327 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:25.327 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:25.327 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1460 2024-09-13T09:12:25.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.328 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121842+0000", 2024-09-13T09:12:25.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121842+0000", 2024-09-13T09:12:25.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121842+0000", 2024-09-13T09:12:25.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:25.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121845+0000", 2024-09-13T09:12:25.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:12:25.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121849+0000", 2024-09-13T09:12:25.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2540000000000003e-06 2024-09-13T09:12:25.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121852+0000", 2024-09-13T09:12:25.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8710000000000001e-06 2024-09-13T09:12:25.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.123228+0000", 2024-09-13T09:12:25.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001376592 2024-09-13T09:12:25.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.123280+0000", 2024-09-13T09:12:25.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1975999999999998e-05 2024-09-13T09:12:25.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:25.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.123509+0000", 2024-09-13T09:12:25.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022916000000000001 2024-09-13T09:12:25.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.125491+0000", 2024-09-13T09:12:25.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0019817379999999998 2024-09-13T09:12:25.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128275+0000", 2024-09-13T09:12:25.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002784201 2024-09-13T09:12:25.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128328+0000", 2024-09-13T09:12:25.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3006e-05 2024-09-13T09:12:25.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128358+0000", 2024-09-13T09:12:25.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9397e-05 2024-09-13T09:12:25.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.333 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.333 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1462 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:25.333 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.122137+0000", 2024-09-13T09:12:25.333 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.1818237160000002, 2024-09-13T09:12:25.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063313809999999996, 2024-09-13T09:12:25.333 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.333 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1462 2024-09-13T09:12:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122137+0000", 2024-09-13T09:12:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122137+0000", 2024-09-13T09:12:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122136+0000", 2024-09-13T09:12:25.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:25.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122140+0000", 2024-09-13T09:12:25.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.425e-06 2024-09-13T09:12:25.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122148+0000", 2024-09-13T09:12:25.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1170000000000004e-06 2024-09-13T09:12:25.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122154+0000", 2024-09-13T09:12:25.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5990000000000001e-06 2024-09-13T09:12:25.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.123777+0000", 2024-09-13T09:12:25.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00162301 2024-09-13T09:12:25.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.123826+0000", 2024-09-13T09:12:25.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9051999999999997e-05 2024-09-13T09:12:25.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:25.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124038+0000", 2024-09-13T09:12:25.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021218499999999999 2024-09-13T09:12:25.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.125699+0000", 2024-09-13T09:12:25.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0016603589999999999 2024-09-13T09:12:25.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128373+0000", 2024-09-13T09:12:25.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026739770000000001 2024-09-13T09:12:25.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128426+0000", 2024-09-13T09:12:25.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3643000000000001e-05 2024-09-13T09:12:25.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128468+0000", 2024-09-13T09:12:25.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1826000000000002e-05 2024-09-13T09:12:25.339 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.339 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.339 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.339 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1463 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:25.339 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.122162+0000", 2024-09-13T09:12:25.339 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.1817984709999996, 2024-09-13T09:12:25.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064746990000000004, 2024-09-13T09:12:25.339 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.339 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.339 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.340 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:25.340 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:25.340 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1463 2024-09-13T09:12:25.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.340 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122162+0000", 2024-09-13T09:12:25.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122162+0000", 2024-09-13T09:12:25.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122161+0000", 2024-09-13T09:12:25.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:25.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122165+0000", 2024-09-13T09:12:25.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5209999999999998e-06 2024-09-13T09:12:25.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122172+0000", 2024-09-13T09:12:25.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2540000000000003e-06 2024-09-13T09:12:25.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122177+0000", 2024-09-13T09:12:25.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-13T09:12:25.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124358+0000", 2024-09-13T09:12:25.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021809350000000002 2024-09-13T09:12:25.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124407+0000", 2024-09-13T09:12:25.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9469999999999999e-05 2024-09-13T09:12:25.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:25.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124633+0000", 2024-09-13T09:12:25.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022538300000000001 2024-09-13T09:12:25.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.126674+0000", 2024-09-13T09:12:25.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020412540000000002 2024-09-13T09:12:25.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128475+0000", 2024-09-13T09:12:25.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001801137 2024-09-13T09:12:25.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128598+0000", 2024-09-13T09:12:25.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012318600000000001 2024-09-13T09:12:25.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128637+0000", 2024-09-13T09:12:25.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8219000000000001e-05 2024-09-13T09:12:25.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.345 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.345 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1465 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:25.345 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.122262+0000", 2024-09-13T09:12:25.346 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.1816978569999996, 2024-09-13T09:12:25.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006503043, 2024-09-13T09:12:25.346 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.346 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:25.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:25.346 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1465 2024-09-13T09:12:25.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.346 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122262+0000", 2024-09-13T09:12:25.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122262+0000", 2024-09-13T09:12:25.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122261+0000", 2024-09-13T09:12:25.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:25.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122268+0000", 2024-09-13T09:12:25.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1720000000000004e-06 2024-09-13T09:12:25.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122277+0000", 2024-09-13T09:12:25.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7459999999999995e-06 2024-09-13T09:12:25.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122284+0000", 2024-09-13T09:12:25.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.511e-06 2024-09-13T09:12:25.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124897+0000", 2024-09-13T09:12:25.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026131420000000002 2024-09-13T09:12:25.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124947+0000", 2024-09-13T09:12:25.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9543000000000003e-05 2024-09-13T09:12:25.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:25.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.125149+0000", 2024-09-13T09:12:25.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000202709 2024-09-13T09:12:25.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.126698+0000", 2024-09-13T09:12:25.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015487649999999999 2024-09-13T09:12:25.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128645+0000", 2024-09-13T09:12:25.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001946632 2024-09-13T09:12:25.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128684+0000", 2024-09-13T09:12:25.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9860000000000001e-05 2024-09-13T09:12:25.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128766+0000", 2024-09-13T09:12:25.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1025000000000004e-05 2024-09-13T09:12:25.351 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.351 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.351 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.351 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4483.0:1067 3.f 3:f00ae4d0:::rbd_data.1123ffac6bc2.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 897024~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:25.351 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:20.142567+0000", 2024-09-13T09:12:25.351 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.1613934459999999, 2024-09-13T09:12:25.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059237839999999996, 2024-09-13T09:12:25.351 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.352 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.352 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.352 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4483", 2024-09-13T09:12:25.352 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:25.352 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1067 2024-09-13T09:12:25.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.352 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142567+0000", 2024-09-13T09:12:25.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142567+0000", 2024-09-13T09:12:25.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142566+0000", 2024-09-13T09:12:25.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:25.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142578+0000", 2024-09-13T09:12:25.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2668e-05 2024-09-13T09:12:25.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142588+0000", 2024-09-13T09:12:25.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5980000000000003e-06 2024-09-13T09:12:25.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142595+0000", 2024-09-13T09:12:25.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0199999999999997e-06 2024-09-13T09:12:25.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142629+0000", 2024-09-13T09:12:25.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4351999999999999e-05 2024-09-13T09:12:25.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142712+0000", 2024-09-13T09:12:25.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2913999999999999e-05 2024-09-13T09:12:25.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:25.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142830+0000", 2024-09-13T09:12:25.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011835 2024-09-13T09:12:25.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.147783+0000", 2024-09-13T09:12:25.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0049524180000000001 2024-09-13T09:12:25.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.148393+0000", 2024-09-13T09:12:25.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00061000699999999998 2024-09-13T09:12:25.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.148456+0000", 2024-09-13T09:12:25.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2996999999999993e-05 2024-09-13T09:12:25.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.148491+0000", 2024-09-13T09:12:25.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4814000000000002e-05 2024-09-13T09:12:25.357 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.357 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.357 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.357 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4483.0:1068 3.c 3:313e6343:::rbd_data.1123ffac6bc2.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 786432~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:25.357 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:20.142607+0000", 2024-09-13T09:12:25.357 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.1613529409999996, 2024-09-13T09:12:25.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0058839460000000001, 2024-09-13T09:12:25.357 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.358 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.358 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.358 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4483", 2024-09-13T09:12:25.358 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:25.358 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1068 2024-09-13T09:12:25.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.358 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142607+0000", 2024-09-13T09:12:25.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142607+0000", 2024-09-13T09:12:25.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142607+0000", 2024-09-13T09:12:25.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:25.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142617+0000", 2024-09-13T09:12:25.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0339e-05 2024-09-13T09:12:25.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142626+0000", 2024-09-13T09:12:25.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5839999999999995e-06 2024-09-13T09:12:25.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142631+0000", 2024-09-13T09:12:25.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4319999999999998e-06 2024-09-13T09:12:25.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142669+0000", 2024-09-13T09:12:25.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7571999999999999e-05 2024-09-13T09:12:25.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142759+0000", 2024-09-13T09:12:25.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0662999999999996e-05 2024-09-13T09:12:25.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:25.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142869+0000", 2024-09-13T09:12:25.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000109734 2024-09-13T09:12:25.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.147790+0000", 2024-09-13T09:12:25.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004921295 2024-09-13T09:12:25.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.148415+0000", 2024-09-13T09:12:25.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00062403000000000003 2024-09-13T09:12:25.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.148462+0000", 2024-09-13T09:12:25.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7933000000000003e-05 2024-09-13T09:12:25.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.148491+0000", 2024-09-13T09:12:25.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8873e-05 2024-09-13T09:12:25.363 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.363 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.363 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.363 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4483.0:1659 3.2 e28/21 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head v 28'145, mlcod=28'142)", 2024-09-13T09:12:25.363 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:20.418410+0000", 2024-09-13T09:12:25.363 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8855502790000003, 2024-09-13T09:12:25.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0058268540000000002, 2024-09-13T09:12:25.363 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.364 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.364 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418410+0000", 2024-09-13T09:12:25.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418410+0000", 2024-09-13T09:12:25.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418410+0000", 2024-09-13T09:12:25.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:25.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418413+0000", 2024-09-13T09:12:25.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7900000000000001e-06 2024-09-13T09:12:25.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418420+0000", 2024-09-13T09:12:25.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:12:25.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418423+0000", 2024-09-13T09:12:25.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7189999999999999e-06 2024-09-13T09:12:25.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418459+0000", 2024-09-13T09:12:25.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5822999999999999e-05 2024-09-13T09:12:25.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418479+0000", 2024-09-13T09:12:25.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9714000000000001e-05 2024-09-13T09:12:25.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.424202+0000", 2024-09-13T09:12:25.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057233350000000004 2024-09-13T09:12:25.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.424237+0000", 2024-09-13T09:12:25.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4691999999999999e-05 2024-09-13T09:12:25.368 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.368 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.368 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.368 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4546.0:2175 3.10 3:0eca404c:::rbd_data.1123ffac6bc2.0000000000000080:head [sparse-read 4096~126976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:25.368 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.048825+0000", 2024-09-13T09:12:25.368 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.2551352389999999, 2024-09-13T09:12:25.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026988175999999999, 2024-09-13T09:12:25.368 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.368 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:25.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.369 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4546", 2024-09-13T09:12:25.369 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2955085030", 2024-09-13T09:12:25.369 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2175 2024-09-13T09:12:25.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.369 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048825+0000", 2024-09-13T09:12:25.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048825+0000", 2024-09-13T09:12:25.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048825+0000", 2024-09-13T09:12:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048828+0000", 2024-09-13T09:12:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2160000000000002e-06 2024-09-13T09:12:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048837+0000", 2024-09-13T09:12:25.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6180000000000001e-06 2024-09-13T09:12:25.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048841+0000", 2024-09-13T09:12:25.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5389999999999999e-06 2024-09-13T09:12:25.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048952+0000", 2024-09-13T09:12:25.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011115 2024-09-13T09:12:25.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.075813+0000", 2024-09-13T09:12:25.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026861182000000001 2024-09-13T09:12:25.372 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.372 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.372 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.372 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4546.0:2178 3.13 3:c89627a6:::rbd_data.1123ffac6bc2.0000000000000082:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:25.372 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.049364+0000", 2024-09-13T09:12:25.372 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.2545962070000001, 2024-09-13T09:12:25.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026490942999999999, 2024-09-13T09:12:25.373 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.373 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:25.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4546", 2024-09-13T09:12:25.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2955085030", 2024-09-13T09:12:25.373 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2178 2024-09-13T09:12:25.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.373 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.049364+0000", 2024-09-13T09:12:25.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.049364+0000", 2024-09-13T09:12:25.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.049363+0000", 2024-09-13T09:12:25.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:25.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.049367+0000", 2024-09-13T09:12:25.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4960000000000001e-06 2024-09-13T09:12:25.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.075707+0000", 2024-09-13T09:12:25.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026339912 2024-09-13T09:12:25.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.075727+0000", 2024-09-13T09:12:25.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9898000000000002e-05 2024-09-13T09:12:25.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.075778+0000", 2024-09-13T09:12:25.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1617999999999999e-05 2024-09-13T09:12:25.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.075855+0000", 2024-09-13T09:12:25.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6644999999999995e-05 2024-09-13T09:12:25.376 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.376 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.376 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.376 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1551 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:25.376 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510403+0000", 2024-09-13T09:12:25.376 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.79355736600000004, 2024-09-13T09:12:25.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0058898470000000001, 2024-09-13T09:12:25.377 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.377 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.377 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.377 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:25.377 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:25.377 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1551 2024-09-13T09:12:25.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.377 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510403+0000", 2024-09-13T09:12:25.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510403+0000", 2024-09-13T09:12:25.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510403+0000", 2024-09-13T09:12:25.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:25.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510406+0000", 2024-09-13T09:12:25.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.219e-06 2024-09-13T09:12:25.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510409+0000", 2024-09-13T09:12:25.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5939999999999998e-06 2024-09-13T09:12:25.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510412+0000", 2024-09-13T09:12:25.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5229999999999998e-06 2024-09-13T09:12:25.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.512770+0000", 2024-09-13T09:12:25.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002358537 2024-09-13T09:12:25.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.512853+0000", 2024-09-13T09:12:25.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2608999999999995e-05 2024-09-13T09:12:25.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:25.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.513186+0000", 2024-09-13T09:12:25.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000332617 2024-09-13T09:12:25.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516155+0000", 2024-09-13T09:12:25.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0029687939999999999 2024-09-13T09:12:25.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516203+0000", 2024-09-13T09:12:25.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8265e-05 2024-09-13T09:12:25.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516253+0000", 2024-09-13T09:12:25.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0538999999999998e-05 2024-09-13T09:12:25.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516293+0000", 2024-09-13T09:12:25.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9490000000000003e-05 2024-09-13T09:12:25.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.382 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.382 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1552 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:25.382 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510415+0000", 2024-09-13T09:12:25.382 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.79354569399999997, 2024-09-13T09:12:25.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059682420000000003, 2024-09-13T09:12:25.383 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.383 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:25.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:25.383 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1552 2024-09-13T09:12:25.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.383 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510415+0000", 2024-09-13T09:12:25.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510415+0000", 2024-09-13T09:12:25.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510414+0000", 2024-09-13T09:12:25.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:25.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510416+0000", 2024-09-13T09:12:25.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:12:25.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510420+0000", 2024-09-13T09:12:25.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7000000000000002e-06 2024-09-13T09:12:25.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510423+0000", 2024-09-13T09:12:25.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6599999999999999e-06 2024-09-13T09:12:25.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.513672+0000", 2024-09-13T09:12:25.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0032489179999999999 2024-09-13T09:12:25.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.513761+0000", 2024-09-13T09:12:25.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9604999999999993e-05 2024-09-13T09:12:25.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:25.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.514081+0000", 2024-09-13T09:12:25.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00031942599999999999 2024-09-13T09:12:25.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516158+0000", 2024-09-13T09:12:25.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002077162 2024-09-13T09:12:25.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516344+0000", 2024-09-13T09:12:25.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018647800000000001 2024-09-13T09:12:25.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516366+0000", 2024-09-13T09:12:25.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1488000000000002e-05 2024-09-13T09:12:25.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516383+0000", 2024-09-13T09:12:25.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7030000000000001e-05 2024-09-13T09:12:25.388 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.388 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.388 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.388 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1553 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:25.388 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510425+0000", 2024-09-13T09:12:25.388 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.79353536099999999, 2024-09-13T09:12:25.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0089516189999999992, 2024-09-13T09:12:25.388 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.388 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:25.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:25.389 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1553 2024-09-13T09:12:25.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.389 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510425+0000", 2024-09-13T09:12:25.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510425+0000", 2024-09-13T09:12:25.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510425+0000", 2024-09-13T09:12:25.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:25.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510426+0000", 2024-09-13T09:12:25.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.72e-06 2024-09-13T09:12:25.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510434+0000", 2024-09-13T09:12:25.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7649999999999993e-06 2024-09-13T09:12:25.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510441+0000", 2024-09-13T09:12:25.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8449999999999997e-06 2024-09-13T09:12:25.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.514480+0000", 2024-09-13T09:12:25.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0040390199999999999 2024-09-13T09:12:25.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.514545+0000", 2024-09-13T09:12:25.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4834999999999998e-05 2024-09-13T09:12:25.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:25.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.514776+0000", 2024-09-13T09:12:25.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023143 2024-09-13T09:12:25.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519286+0000", 2024-09-13T09:12:25.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045099720000000001 2024-09-13T09:12:25.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519296+0000", 2024-09-13T09:12:25.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-06 2024-09-13T09:12:25.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519342+0000", 2024-09-13T09:12:25.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6436000000000001e-05 2024-09-13T09:12:25.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519377+0000", 2024-09-13T09:12:25.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4269e-05 2024-09-13T09:12:25.393 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.393 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.393 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1554 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510444+0000", 2024-09-13T09:12:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.79351654699999996, 2024-09-13T09:12:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0090383259999999993, 2024-09-13T09:12:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1554 2024-09-13T09:12:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510444+0000", 2024-09-13T09:12:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510444+0000", 2024-09-13T09:12:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510444+0000", 2024-09-13T09:12:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510445+0000", 2024-09-13T09:12:25.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.951e-06 2024-09-13T09:12:25.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510449+0000", 2024-09-13T09:12:25.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3569999999999998e-06 2024-09-13T09:12:25.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510456+0000", 2024-09-13T09:12:25.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6599999999999998e-06 2024-09-13T09:12:25.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.515200+0000", 2024-09-13T09:12:25.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0047443889999999999 2024-09-13T09:12:25.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.515286+0000", 2024-09-13T09:12:25.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5626999999999999e-05 2024-09-13T09:12:25.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:25.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.515685+0000", 2024-09-13T09:12:25.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039960899999999999 2024-09-13T09:12:25.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519386+0000", 2024-09-13T09:12:25.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0037010659999999998 2024-09-13T09:12:25.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519426+0000", 2024-09-13T09:12:25.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9929999999999999e-05 2024-09-13T09:12:25.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519458+0000", 2024-09-13T09:12:25.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2189e-05 2024-09-13T09:12:25.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519482+0000", 2024-09-13T09:12:25.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3779999999999999e-05 2024-09-13T09:12:25.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.399 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.399 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1555 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:25.399 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510460+0000", 2024-09-13T09:12:25.400 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.79350070699999997, 2024-09-13T09:12:25.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0091250399999999992, 2024-09-13T09:12:25.400 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.400 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:25.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:25.400 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1555 2024-09-13T09:12:25.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.400 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510460+0000", 2024-09-13T09:12:25.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510460+0000", 2024-09-13T09:12:25.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510459+0000", 2024-09-13T09:12:25.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:25.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510464+0000", 2024-09-13T09:12:25.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8709999999999996e-06 2024-09-13T09:12:25.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510469+0000", 2024-09-13T09:12:25.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:12:25.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510472+0000", 2024-09-13T09:12:25.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.813e-06 2024-09-13T09:12:25.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516420+0000", 2024-09-13T09:12:25.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059477749999999998 2024-09-13T09:12:25.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516523+0000", 2024-09-13T09:12:25.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103222 2024-09-13T09:12:25.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:25.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516877+0000", 2024-09-13T09:12:25.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00035438100000000002 2024-09-13T09:12:25.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519391+0000", 2024-09-13T09:12:25.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002514166 2024-09-13T09:12:25.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519520+0000", 2024-09-13T09:12:25.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000128582 2024-09-13T09:12:25.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519566+0000", 2024-09-13T09:12:25.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6224999999999999e-05 2024-09-13T09:12:25.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519585+0000", 2024-09-13T09:12:25.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8391000000000001e-05 2024-09-13T09:12:25.405 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.405 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.405 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.405 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1556 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:25.405 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510474+0000", 2024-09-13T09:12:25.405 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.79348588499999995, 2024-09-13T09:12:25.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0091831980000000001, 2024-09-13T09:12:25.406 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.406 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:25.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:25.406 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1556 2024-09-13T09:12:25.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.406 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510474+0000", 2024-09-13T09:12:25.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510474+0000", 2024-09-13T09:12:25.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510474+0000", 2024-09-13T09:12:25.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:25.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510476+0000", 2024-09-13T09:12:25.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1799999999999999e-06 2024-09-13T09:12:25.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510479+0000", 2024-09-13T09:12:25.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1159999999999999e-06 2024-09-13T09:12:25.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510482+0000", 2024-09-13T09:12:25.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:12:25.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.517292+0000", 2024-09-13T09:12:25.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068100239999999996 2024-09-13T09:12:25.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.517344+0000", 2024-09-13T09:12:25.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2092999999999997e-05 2024-09-13T09:12:25.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:25.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.517558+0000", 2024-09-13T09:12:25.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021441900000000001 2024-09-13T09:12:25.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519397+0000", 2024-09-13T09:12:25.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001838891 2024-09-13T09:12:25.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519617+0000", 2024-09-13T09:12:25.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021970400000000001 2024-09-13T09:12:25.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519639+0000", 2024-09-13T09:12:25.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1773999999999999e-05 2024-09-13T09:12:25.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519658+0000", 2024-09-13T09:12:25.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8787000000000002e-05 2024-09-13T09:12:25.411 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.411 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.411 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.411 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1557 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:25.411 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510490+0000", 2024-09-13T09:12:25.411 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.79347028900000005, 2024-09-13T09:12:25.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0092238319999999995, 2024-09-13T09:12:25.411 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.411 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.411 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.412 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:25.412 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:25.412 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1557 2024-09-13T09:12:25.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.412 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510490+0000", 2024-09-13T09:12:25.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510490+0000", 2024-09-13T09:12:25.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510490+0000", 2024-09-13T09:12:25.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:25.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510492+0000", 2024-09-13T09:12:25.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-13T09:12:25.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510495+0000", 2024-09-13T09:12:25.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5020000000000001e-06 2024-09-13T09:12:25.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510498+0000", 2024-09-13T09:12:25.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6809999999999998e-06 2024-09-13T09:12:25.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.517928+0000", 2024-09-13T09:12:25.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0074299259999999999 2024-09-13T09:12:25.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.518005+0000", 2024-09-13T09:12:25.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6642000000000002e-05 2024-09-13T09:12:25.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:25.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.518340+0000", 2024-09-13T09:12:25.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00033502400000000001 2024-09-13T09:12:25.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519592+0000", 2024-09-13T09:12:25.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001252647 2024-09-13T09:12:25.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519677+0000", 2024-09-13T09:12:25.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4751000000000002e-05 2024-09-13T09:12:25.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519698+0000", 2024-09-13T09:12:25.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0792999999999999e-05 2024-09-13T09:12:25.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519714+0000", 2024-09-13T09:12:25.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6008e-05 2024-09-13T09:12:25.417 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.417 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.417 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.417 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1558 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:25.417 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510501+0000", 2024-09-13T09:12:25.417 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.79345926, 2024-09-13T09:12:25.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0094031559999999993, 2024-09-13T09:12:25.417 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:25.417 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:25.417 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:25.418 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:25.418 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:25.418 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1558 2024-09-13T09:12:25.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.418 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:25.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:25.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510501+0000", 2024-09-13T09:12:25.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:25.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510501+0000", 2024-09-13T09:12:25.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:25.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:25.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510501+0000", 2024-09-13T09:12:25.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:25.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:25.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510503+0000", 2024-09-13T09:12:25.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.72e-06 2024-09-13T09:12:25.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:25.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510506+0000", 2024-09-13T09:12:25.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.286e-06 2024-09-13T09:12:25.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:25.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510508+0000", 2024-09-13T09:12:25.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-13T09:12:25.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:25.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.518739+0000", 2024-09-13T09:12:25.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0082309080000000003 2024-09-13T09:12:25.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:25.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.518787+0000", 2024-09-13T09:12:25.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8214999999999998e-05 2024-09-13T09:12:25.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:25.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519003+0000", 2024-09-13T09:12:25.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021570199999999999 2024-09-13T09:12:25.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:25.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519720+0000", 2024-09-13T09:12:25.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00071655099999999995 2024-09-13T09:12:25.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519868+0000", 2024-09-13T09:12:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014865000000000001 2024-09-13T09:12:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519888+0000", 2024-09-13T09:12:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0177e-05 2024-09-13T09:12:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:25.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519904+0000", 2024-09-13T09:12:25.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5767999999999999e-05 2024-09-13T09:12:25.423 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.423 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.423 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.423 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:25.423 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:25.423 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:25.424 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T09:12:25.425 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-09-13T09:12:25.425 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:25.423+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:25.426 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:25.427 INFO:teuthology.orchestra.run.smithi157.stdout:643 read 0xd2f800 thru 0xd36af9 (0x72fa bytes) 2024-09-13T09:12:25.427 INFO:teuthology.orchestra.run.smithi157.stdout:644 read 0x1185600 thru 0x118f728 (0xa129 bytes) 2024-09-13T09:12:25.427 INFO:teuthology.orchestra.run.smithi157.stdout:645 write 0x3abfc00 thru 0x3acee00 (0xf201 bytes) 2024-09-13T09:12:25.427 INFO:teuthology.orchestra.run.smithi157.stdout:646 write 0x4e5aa00 thru 0x4e6051f (0x5b20 bytes) 2024-09-13T09:12:25.428 INFO:teuthology.orchestra.run.smithi157.stdout:647 write 0x47b4a00 thru 0x47bd852 (0x8e53 bytes) 2024-09-13T09:12:25.428 INFO:teuthology.orchestra.run.smithi157.stdout:648 trunc from 0xbc12e00 to 0xe158e00 2024-09-13T09:12:25.428 INFO:teuthology.orchestra.run.smithi157.stdout:649 trunc from 0xe158e00 to 0xb0e3e00 2024-09-13T09:12:25.428 INFO:teuthology.orchestra.run.smithi157.stdout:650 write 0x1b11800 thru 0x1b1aadf (0x92e0 bytes) 2024-09-13T09:12:25.428 INFO:teuthology.orchestra.run.smithi157.stdout:651 write 0x3df0200 thru 0x3df9c49 (0x9a4a bytes) 2024-09-13T09:12:25.428 INFO:teuthology.orchestra.run.smithi157.stdout:652 read 0xa773e00 thru 0xa77f69c (0xb89d bytes) 2024-09-13T09:12:25.428 INFO:teuthology.orchestra.run.smithi157.stdout:653 trunc from 0xb0e3e00 to 0x1d3be00 2024-09-13T09:12:25.428 INFO:teuthology.orchestra.run.smithi157.stdout:656 write 0xcd3e00 thru 0xcda703 (0x6904 bytes) 2024-09-13T09:12:25.428 INFO:teuthology.orchestra.run.smithi157.stdout:659 read 0x1a51400 thru 0x1a59d29 (0x892a bytes) 2024-09-13T09:12:25.428 INFO:teuthology.orchestra.run.smithi157.stdout:660 punch from 0x100ee00 to 0x101b000, (0xc200 bytes) 2024-09-13T09:12:25.428 INFO:teuthology.orchestra.run.smithi157.stdout:663 trunc from 0x1d3be00 to 0x4bf6400 2024-09-13T09:12:25.435 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T09:12:25.437 INFO:teuthology.orchestra.run.smithi157.stdout:664 write 0x356bc00 thru 0x35752cc (0x96cd bytes) 2024-09-13T09:12:25.437 INFO:teuthology.orchestra.run.smithi157.stdout:665 read 0x1575000 thru 0x1577751 (0x2752 bytes) 2024-09-13T09:12:25.438 INFO:teuthology.orchestra.run.smithi157.stdout:667 trunc from 0x4bf6400 to 0x3d86e00 2024-09-13T09:12:25.449 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:12:25.463 INFO:teuthology.orchestra.run.smithi157.stdout:670 trunc from 0x3d86e00 to 0xd5ae000 2024-09-13T09:12:25.472 INFO:teuthology.orchestra.run.smithi157.stdout:677 read 0xbbe4400 thru 0xbbf41fc (0xfdfd bytes) 2024-09-13T09:12:25.480 INFO:teuthology.orchestra.run.smithi157.stdout:680 read 0xc5e7c00 thru 0xc5eb326 (0x3727 bytes) 2024-09-13T09:12:25.480 INFO:teuthology.orchestra.run.smithi157.stdout:681 punch from 0x2242600 to 0x2248e00, (0x6800 bytes) 2024-09-13T09:12:25.480 INFO:teuthology.orchestra.run.smithi157.stdout:682 read 0xcb31800 thru 0xcb3a75c (0x8f5d bytes) 2024-09-13T09:12:25.480 INFO:teuthology.orchestra.run.smithi157.stdout:683 punch from 0x582ee00 to 0x583ea00, (0xfc00 bytes) 2024-09-13T09:12:25.480 INFO:teuthology.orchestra.run.smithi157.stdout:685 write 0xbda9600 thru 0xbdace7e (0x387f bytes) 2024-09-13T09:12:25.502 INFO:teuthology.orchestra.run.smithi157.stdout:688 write 0xe412600 thru 0xe41abdf (0x85e0 bytes) 2024-09-13T09:12:25.503 INFO:teuthology.orchestra.run.smithi157.stdout:689 write 0x4d6aa00 thru 0x4d77bb3 (0xd1b4 bytes) 2024-09-13T09:12:25.503 INFO:teuthology.orchestra.run.smithi157.stdout:690 read 0xbccee00 thru 0xbcdaf72 (0xc173 bytes) 2024-09-13T09:12:25.506 INFO:teuthology.orchestra.run.smithi157.stdout:692 write 0x2410600 thru 0x241973e (0x913f bytes) 2024-09-13T09:12:25.525 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:25.523+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:25.528 INFO:teuthology.orchestra.run.smithi157.stdout:697 write 0xebade00 thru 0xebaf630 (0x1831 bytes) 2024-09-13T09:12:25.529 INFO:teuthology.orchestra.run.smithi157.stdout:698 write 0x6ff7800 thru 0x6ffb961 (0x4162 bytes) 2024-09-13T09:12:25.529 INFO:teuthology.orchestra.run.smithi157.stdout:699 read 0x1257000 thru 0x125a5b5 (0x35b6 bytes) 2024-09-13T09:12:25.533 INFO:teuthology.orchestra.run.smithi157.stdout:700 punch from 0xaa5400 to 0xaabe00, (0x6a00 bytes) 2024-09-13T09:12:25.541 INFO:teuthology.orchestra.run.smithi157.stdout:701 trunc from 0xebaf800 to 0x7e9fe00 2024-09-13T09:12:25.563 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:25.563 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:25.563 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:25.564 INFO:teuthology.orchestra.run.smithi157.stdout:704 read 0x254c00 thru 0x2622fc (0xd6fd bytes) 2024-09-13T09:12:25.568 INFO:teuthology.orchestra.run.smithi157.stdout:706 punch from 0x74dd200 to 0x74e3800, (0x6600 bytes) 2024-09-13T09:12:25.568 INFO:teuthology.orchestra.run.smithi157.stdout:709 punch from 0xcce800 to 0xcd0800, (0x2000 bytes) 2024-09-13T09:12:25.568 INFO:teuthology.orchestra.run.smithi157.stdout:710 punch from 0x508d800 to 0x508e200, (0xa00 bytes) 2024-09-13T09:12:25.569 INFO:teuthology.orchestra.run.smithi157.stdout:711 write 0x1133000 thru 0x11386bb (0x56bc bytes) 2024-09-13T09:12:25.569 INFO:teuthology.orchestra.run.smithi157.stdout:712 punch from 0x1053e00 to 0x105e800, (0xaa00 bytes) 2024-09-13T09:12:25.569 INFO:teuthology.orchestra.run.smithi157.stdout:713 read 0x6555000 thru 0x6563918 (0xe919 bytes) 2024-09-13T09:12:25.574 INFO:teuthology.orchestra.run.smithi157.stdout:717 trunc from 0x7e9fe00 to 0xc7ad800 2024-09-13T09:12:25.575 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:25.608 INFO:teuthology.orchestra.run.smithi157.stdout:718 write 0xe5a3200 thru 0xe5aac8d (0x7a8e bytes) 2024-09-13T09:12:25.609 INFO:teuthology.orchestra.run.smithi157.stdout:722 write 0x850f400 thru 0x851b0e0 (0xbce1 bytes) 2024-09-13T09:12:25.609 INFO:teuthology.orchestra.run.smithi157.stdout:723 read 0x637c400 thru 0x6380c27 (0x4828 bytes) 2024-09-13T09:12:25.609 INFO:teuthology.orchestra.run.smithi157.stdout:724 write 0x1e19400 thru 0x1e1cc66 (0x3867 bytes) 2024-09-13T09:12:25.613 INFO:teuthology.orchestra.run.smithi157.stdout:726 write 0x5b9200 thru 0x5be1b4 (0x4fb5 bytes) 2024-09-13T09:12:25.614 INFO:teuthology.orchestra.run.smithi157.stdout:728 write 0xc331000 thru 0xc33d723 (0xc724 bytes) 2024-09-13T09:12:25.614 INFO:teuthology.orchestra.run.smithi157.stdout:729 read 0x8130400 thru 0x8138853 (0x8454 bytes) 2024-09-13T09:12:25.614 INFO:teuthology.orchestra.run.smithi157.stdout:731 punch from 0x72fbc00 to 0x72fd800, (0x1c00 bytes) 2024-09-13T09:12:25.615 INFO:teuthology.orchestra.run.smithi157.stdout:732 read 0x294d000 thru 0x29552de (0x82df bytes) 2024-09-13T09:12:25.615 INFO:teuthology.orchestra.run.smithi157.stdout:736 write 0x5ef6000 thru 0x5efc77b (0x677c bytes) 2024-09-13T09:12:25.617 INFO:teuthology.orchestra.run.smithi157.stdout:737 write 0x889f200 thru 0x88a69b5 (0x77b6 bytes) 2024-09-13T09:12:25.617 INFO:teuthology.orchestra.run.smithi157.stdout:741 punch from 0xd5d4000 to 0xd5d8400, (0x4400 bytes) 2024-09-13T09:12:25.617 INFO:teuthology.orchestra.run.smithi157.stdout:745 read 0x66f5800 thru 0x6700416 (0xac17 bytes) 2024-09-13T09:12:25.618 INFO:teuthology.orchestra.run.smithi157.stdout:746 read 0x3914600 thru 0x3915188 (0xb89 bytes) 2024-09-13T09:12:25.618 INFO:teuthology.orchestra.run.smithi157.stdout:748 punch from 0xb3d9e00 to 0xb3dd800, (0x3a00 bytes) 2024-09-13T09:12:25.618 INFO:teuthology.orchestra.run.smithi157.stdout:750 write 0x6286200 thru 0x628a1a3 (0x3fa4 bytes) 2024-09-13T09:12:25.626 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:25.625+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:25.641 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:25.641 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:25.644 INFO:teuthology.orchestra.run.smithi157.stdout:753 write 0xebca200 thru 0xebd12ea (0x70eb bytes) 2024-09-13T09:12:25.646 INFO:teuthology.orchestra.run.smithi157.stdout:755 read 0xa903400 thru 0xa9079bc (0x45bd bytes) 2024-09-13T09:12:25.646 INFO:teuthology.orchestra.run.smithi157.stdout:757 punch from 0xd21da00 to 0xd224e00, (0x7400 bytes) 2024-09-13T09:12:25.650 INFO:teuthology.orchestra.run.smithi157.stdout:758 read 0x8fc6400 thru 0x8fc9a0f (0x3610 bytes) 2024-09-13T09:12:25.650 INFO:teuthology.orchestra.run.smithi157.stdout:760 write 0x668ba00 thru 0x66924a4 (0x6aa5 bytes) 2024-09-13T09:12:25.650 INFO:teuthology.orchestra.run.smithi157.stdout:761 write 0xe2aba00 thru 0xe2ba0b1 (0xe6b2 bytes) 2024-09-13T09:12:25.650 INFO:teuthology.orchestra.run.smithi157.stdout:764 write 0x794d600 thru 0x7955c55 (0x8656 bytes) 2024-09-13T09:12:25.659 INFO:teuthology.orchestra.run.smithi157.stdout:768 trunc from 0xebd1400 to 0x6baf800 2024-09-13T09:12:25.676 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:25.676 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:25.701 INFO:teuthology.orchestra.run.smithi157.stdout:772 trunc from 0x6baf800 to 0x47b7c00 2024-09-13T09:12:25.711 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:25.711 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:25.721 INFO:teuthology.orchestra.run.smithi157.stdout:774 read 0x28f6800 thru 0x28f7483 (0xc84 bytes) 2024-09-13T09:12:25.726 INFO:teuthology.orchestra.run.smithi157.stdout:779 read 0x3059c00 thru 0x30658ce (0xbccf bytes) 2024-09-13T09:12:25.726 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:25.725+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:25.730 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:25.730 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:25.743 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:25.744 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:25.744 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:25.746 INFO:teuthology.orchestra.run.smithi157.stdout:780 write 0xd3a6400 thru 0xd3b63ce (0xffcf bytes) 2024-09-13T09:12:25.753 INFO:teuthology.orchestra.run.smithi157.stdout:781 write 0x7930200 thru 0x793cda2 (0xcba3 bytes) 2024-09-13T09:12:25.753 INFO:teuthology.orchestra.run.smithi157.stdout:783 read 0xc9a8a00 thru 0xc9adc08 (0x5209 bytes) 2024-09-13T09:12:25.753 INFO:teuthology.orchestra.run.smithi157.stdout:784 write 0x8247400 thru 0x824d238 (0x5e39 bytes) 2024-09-13T09:12:25.753 INFO:teuthology.orchestra.run.smithi157.stdout:786 read 0x358f200 thru 0x35986a7 (0x94a8 bytes) 2024-09-13T09:12:25.753 INFO:teuthology.orchestra.run.smithi157.stdout:787 write 0x1b0e600 thru 0x1b1c9e9 (0xe3ea bytes) 2024-09-13T09:12:25.754 INFO:teuthology.orchestra.run.smithi157.stdout:788 read 0xb2ef800 thru 0xb2f0cf3 (0x14f4 bytes) 2024-09-13T09:12:25.767 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:25.767 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:25.775 INFO:teuthology.orchestra.run.smithi157.stdout:789 write 0xd516400 thru 0xd5179bb (0x15bc bytes) 2024-09-13T09:12:25.779 INFO:teuthology.orchestra.run.smithi157.stdout:791 punch from 0x33ea200 to 0x33f8400, (0xe200 bytes) 2024-09-13T09:12:25.780 INFO:teuthology.orchestra.run.smithi157.stdout:794 write 0x5e66e00 thru 0x5e70dd4 (0x9fd5 bytes) 2024-09-13T09:12:25.780 INFO:teuthology.orchestra.run.smithi157.stdout:795 write 0x5910c00 thru 0x59150e6 (0x44e7 bytes) 2024-09-13T09:12:25.788 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:25.788 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:25.791 INFO:teuthology.orchestra.run.smithi157.stdout:796 trunc from 0xd517a00 to 0x3e62200 2024-09-13T09:12:25.830 INFO:teuthology.orchestra.run.smithi157.stdout:798 trunc from 0x3e62200 to 0xcd56a00 2024-09-13T09:12:25.839 INFO:teuthology.orchestra.run.smithi157.stdout:799 punch from 0xba7f000 to 0xba87400, (0x8400 bytes) 2024-09-13T09:12:25.843 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:25.842+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:25.847 INFO:teuthology.orchestra.run.smithi157.stdout:801 read 0x5845c00 thru 0x5853fa7 (0xe3a8 bytes) 2024-09-13T09:12:25.847 INFO:teuthology.orchestra.run.smithi157.stdout:803 write 0xc8e0200 thru 0xc8e3491 (0x3292 bytes) 2024-09-13T09:12:25.847 INFO:teuthology.orchestra.run.smithi157.stdout:804 read 0xb15b000 thru 0xb15f6e5 (0x46e6 bytes) 2024-09-13T09:12:25.847 INFO:teuthology.orchestra.run.smithi157.stdout:805 read 0x870e600 thru 0x871255a (0x3f5b bytes) 2024-09-13T09:12:25.848 INFO:teuthology.orchestra.run.smithi157.stdout:809 read 0x15c9c00 thru 0x15cad13 (0x1114 bytes) 2024-09-13T09:12:25.848 INFO:teuthology.orchestra.run.smithi157.stdout:811 read 0x23f8000 thru 0x23fe36c (0x636d bytes) 2024-09-13T09:12:25.849 INFO:teuthology.orchestra.run.smithi157.stdout:814 trunc from 0xcd56a00 to 0xb5ae600 2024-09-13T09:12:25.873 INFO:teuthology.orchestra.run.smithi157.stdout:815 trunc from 0xb5ae600 to 0x7198a00 2024-09-13T09:12:25.876 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:25.876 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:25.887 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:25.888 INFO:teuthology.orchestra.run.smithi157.stdout:816 write 0x4260800 thru 0x426c0d5 (0xb8d6 bytes) 2024-09-13T09:12:25.889 INFO:teuthology.orchestra.run.smithi157.stdout:819 punch from 0xe7be00 to 0xe8ac00, (0xee00 bytes) 2024-09-13T09:12:25.889 INFO:teuthology.orchestra.run.smithi157.stdout:820 punch from 0x175fe00 to 0x176ca00, (0xcc00 bytes) 2024-09-13T09:12:25.889 INFO:teuthology.orchestra.run.smithi157.stdout:821 punch from 0x2a94e00 to 0x2a97e00, (0x3000 bytes) 2024-09-13T09:12:25.890 INFO:teuthology.orchestra.run.smithi157.stdout:822 read 0x22b800 thru 0x22ff35 (0x4736 bytes) 2024-09-13T09:12:25.891 INFO:teuthology.orchestra.run.smithi157.stdout:826 read 0x39ea800 thru 0x39eeb64 (0x4365 bytes) 2024-09-13T09:12:25.891 INFO:teuthology.orchestra.run.smithi157.stdout:828 read 0x3e29e00 thru 0x3e376db (0xd8dc bytes) 2024-09-13T09:12:25.892 INFO:teuthology.orchestra.run.smithi157.stdout:829 write 0x434ea00 thru 0x4354947 (0x5f48 bytes) 2024-09-13T09:12:25.892 INFO:teuthology.orchestra.run.smithi157.stdout:833 punch from 0x387a00 to 0x38d800, (0x5e00 bytes) 2024-09-13T09:12:25.892 INFO:teuthology.orchestra.run.smithi157.stdout:835 punch from 0x3deac00 to 0x3df5c00, (0xb000 bytes) 2024-09-13T09:12:25.892 INFO:teuthology.orchestra.run.smithi157.stdout:836 read 0x4beaa00 thru 0x4becf2f (0x2530 bytes) 2024-09-13T09:12:25.896 INFO:teuthology.orchestra.run.smithi157.stdout:837 trunc from 0x7198a00 to 0x3f20e00 2024-09-13T09:12:25.926 INFO:teuthology.orchestra.run.smithi157.stdout:839 write 0x9639600 thru 0x963c34d (0x2d4e bytes) 2024-09-13T09:12:25.928 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:25.927+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:25.935 INFO:teuthology.orchestra.run.smithi157.stdout:843 trunc from 0x963c400 to 0x5a53400 2024-09-13T09:12:25.950 INFO:teuthology.orchestra.run.smithi157.stdout:844 write 0x4379000 thru 0x43829a6 (0x99a7 bytes) 2024-09-13T09:12:25.962 INFO:teuthology.orchestra.run.smithi157.stdout:847 trunc from 0x5a53400 to 0xaf2f800 2024-09-13T09:12:25.977 INFO:teuthology.orchestra.run.smithi157.stdout:848 punch from 0x1dab200 to 0x1db6000, (0xae00 bytes) 2024-09-13T09:12:25.985 INFO:teuthology.orchestra.run.smithi157.stdout:849 read 0xc4d000 thru 0xc57111 (0xa112 bytes) 2024-09-13T09:12:25.986 INFO:teuthology.orchestra.run.smithi157.stdout:850 punch from 0x9b33600 to 0x9b41200, (0xdc00 bytes) 2024-09-13T09:12:25.986 INFO:teuthology.orchestra.run.smithi157.stdout:851 punch from 0x904ba00 to 0x904f400, (0x3a00 bytes) 2024-09-13T09:12:25.986 INFO:teuthology.orchestra.run.smithi157.stdout:853 write 0x535fa00 thru 0x5368092 (0x8693 bytes) 2024-09-13T09:12:25.986 INFO:teuthology.orchestra.run.smithi157.stdout:857 read 0x954fe00 thru 0x9558fba (0x91bb bytes) 2024-09-13T09:12:25.986 INFO:teuthology.orchestra.run.smithi157.stdout:859 write 0x6a5ba00 thru 0x6a6768e (0xbc8f bytes) 2024-09-13T09:12:25.986 INFO:teuthology.orchestra.run.smithi157.stdout:861 read 0x2577000 thru 0x257e7f8 (0x77f9 bytes) 2024-09-13T09:12:25.990 INFO:teuthology.orchestra.run.smithi157.stdout:863 trunc from 0xaf2f800 to 0xe213c00 2024-09-13T09:12:26.005 INFO:teuthology.orchestra.run.smithi157.stdout:866 write 0x98ac800 thru 0x98bb170 (0xe971 bytes) 2024-09-13T09:12:26.007 INFO:teuthology.orchestra.run.smithi157.stdout:868 write 0xa2b8200 thru 0xa2bec4c (0x6a4d bytes) 2024-09-13T09:12:26.007 INFO:teuthology.orchestra.run.smithi157.stdout:870 punch from 0x7a18e00 to 0x7a1ba00, (0x2c00 bytes) 2024-09-13T09:12:26.007 INFO:teuthology.orchestra.run.smithi157.stdout:872 write 0xa444200 thru 0xa448e7d (0x4c7e bytes) 2024-09-13T09:12:26.007 INFO:teuthology.orchestra.run.smithi157.stdout:873 read 0x829800 thru 0x82c4ea (0x2ceb bytes) 2024-09-13T09:12:26.008 INFO:teuthology.orchestra.run.smithi157.stdout:874 write 0xb9c9200 thru 0xb9d62c4 (0xd0c5 bytes) 2024-09-13T09:12:26.008 INFO:teuthology.orchestra.run.smithi157.stdout:877 write 0x2417a00 thru 0x24188e0 (0xee1 bytes) 2024-09-13T09:12:26.011 INFO:teuthology.orchestra.run.smithi157.stdout:878 write 0x6208000 thru 0x62148e0 (0xc8e1 bytes) 2024-09-13T09:12:26.012 INFO:teuthology.orchestra.run.smithi157.stdout:879 trunc from 0xe213c00 to 0xda0a800 2024-09-13T09:12:26.029 INFO:teuthology.orchestra.run.smithi157.stdout:881 punch from 0xa6cd000 to 0xa6d6a00, (0x9a00 bytes) 2024-09-13T09:12:26.029 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:26.028+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:26.037 INFO:teuthology.orchestra.run.smithi157.stdout:883 punch from 0x9224e00 to 0x9225000, (0x200 bytes) 2024-09-13T09:12:26.037 INFO:teuthology.orchestra.run.smithi157.stdout:884 trunc from 0xda0a800 to 0xe23fe00 2024-09-13T09:12:26.051 INFO:teuthology.orchestra.run.smithi157.stdout:887 punch from 0x6150600 to 0x6156c00, (0x6600 bytes) 2024-09-13T09:12:26.055 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:26.055 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:26.065 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:26.067 INFO:teuthology.orchestra.run.smithi157.stdout:888 trunc from 0xe23fe00 to 0x17be400 2024-09-13T09:12:26.090 INFO:teuthology.orchestra.run.smithi157.stdout:889 write 0x58400 thru 0x62dcf (0xa9d0 bytes) 2024-09-13T09:12:26.092 INFO:teuthology.orchestra.run.smithi157.stdout:890 read 0x6db600 thru 0x6dd017 (0x1a18 bytes) 2024-09-13T09:12:26.092 INFO:teuthology.orchestra.run.smithi157.stdout:895 write 0x497200 thru 0x4a6eb8 (0xfcb9 bytes) 2024-09-13T09:12:26.093 INFO:teuthology.orchestra.run.smithi157.stdout:896 read 0x1363400 thru 0x1366b41 (0x3742 bytes) 2024-09-13T09:12:26.093 INFO:teuthology.orchestra.run.smithi157.stdout:899 read 0xdc7800 thru 0xdce08c (0x688d bytes) 2024-09-13T09:12:26.110 INFO:teuthology.orchestra.run.smithi157.stdout:901 write 0x6dc6c00 thru 0x6dccc23 (0x6024 bytes) 2024-09-13T09:12:26.111 INFO:teuthology.orchestra.run.smithi157.stdout:902 read 0x3784800 thru 0x3790463 (0xbc64 bytes) 2024-09-13T09:12:26.111 INFO:teuthology.orchestra.run.smithi157.stdout:903 punch from 0x361dc00 to 0x362bc00, (0xe000 bytes) 2024-09-13T09:12:26.111 INFO:teuthology.orchestra.run.smithi157.stdout:906 read 0x5dd9c00 thru 0x5de978f (0xfb90 bytes) 2024-09-13T09:12:26.119 INFO:teuthology.orchestra.run.smithi157.stdout:907 trunc from 0x6dcce00 to 0xd8bae00 2024-09-13T09:12:26.128 INFO:teuthology.orchestra.run.smithi157.stdout:908 read 0x6a2a600 thru 0x6a30063 (0x5a64 bytes) 2024-09-13T09:12:26.128 INFO:teuthology.orchestra.run.smithi157.stdout:909 write 0xe0400 thru 0xea9e7 (0xa5e8 bytes) 2024-09-13T09:12:26.130 INFO:teuthology.orchestra.run.smithi157.stdout:910 read 0x5235800 thru 0x523cd8b (0x758c bytes) 2024-09-13T09:12:26.131 INFO:teuthology.orchestra.run.smithi157.stdout:911 read 0x9e9e800 thru 0x9eaa115 (0xb916 bytes) 2024-09-13T09:12:26.131 INFO:teuthology.orchestra.run.smithi157.stdout:912 read 0x24f3600 thru 0x25020ce (0xeacf bytes) 2024-09-13T09:12:26.131 INFO:teuthology.orchestra.run.smithi157.stdout:915 write 0xc170000 thru 0xc17aa49 (0xaa4a bytes) 2024-09-13T09:12:26.131 INFO:teuthology.orchestra.run.smithi157.stdout:916 punch from 0x99fe00 to 0x9a0c00, (0xe00 bytes) 2024-09-13T09:12:26.134 INFO:teuthology.orchestra.run.smithi157.stdout:917 trunc from 0xd8bae00 to 0x9eb6800 2024-09-13T09:12:26.151 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:26.150+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:26.163 INFO:teuthology.orchestra.run.smithi157.stdout:918 write 0xe16f600 thru 0xe17e6d2 (0xf0d3 bytes) 2024-09-13T09:12:26.181 INFO:teuthology.orchestra.run.smithi157.stdout:920 trunc from 0xe17e800 to 0x1479800 2024-09-13T09:12:26.215 INFO:teuthology.orchestra.run.smithi157.stdout:922 read 0x1ce400 thru 0x1d358f (0x5190 bytes) 2024-09-13T09:12:26.221 INFO:teuthology.orchestra.run.smithi157.stdout:924 trunc from 0x1479800 to 0x5af8800 2024-09-13T09:12:26.230 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:26.230 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:26.231 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:26.229+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:26.236 INFO:teuthology.orchestra.run.smithi157.stdout:925 read 0x151d000 thru 0x151fa8b (0x2a8c bytes) 2024-09-13T09:12:26.239 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:26.253 INFO:teuthology.orchestra.run.smithi157.stdout:926 write 0x892a400 thru 0x892dc37 (0x3838 bytes) 2024-09-13T09:12:26.254 INFO:teuthology.orchestra.run.smithi157.stdout:927 write 0x7964200 thru 0x796efd8 (0xadd9 bytes) 2024-09-13T09:12:26.254 INFO:teuthology.orchestra.run.smithi157.stdout:928 read 0x36dfc00 thru 0x36ea71a (0xab1b bytes) 2024-09-13T09:12:26.254 INFO:teuthology.orchestra.run.smithi157.stdout:930 read 0x7846600 thru 0x784bdac (0x57ad bytes) 2024-09-13T09:12:26.254 INFO:teuthology.orchestra.run.smithi157.stdout:932 read 0x3967000 thru 0x396733e (0x33f bytes) 2024-09-13T09:12:26.254 INFO:teuthology.orchestra.run.smithi157.stdout:935 write 0x133be00 thru 0x134b9fb (0xfbfc bytes) 2024-09-13T09:12:26.255 INFO:teuthology.orchestra.run.smithi157.stdout:937 write 0x41d8200 thru 0x41e0afe (0x88ff bytes) 2024-09-13T09:12:26.288 INFO:teuthology.orchestra.run.smithi157.stdout:938 write 0xe2c2400 thru 0xe2c8511 (0x6112 bytes) 2024-09-13T09:12:26.288 INFO:teuthology.orchestra.run.smithi157.stdout:940 write 0x6f50800 thru 0x6f5fb24 (0xf325 bytes) 2024-09-13T09:12:26.288 INFO:teuthology.orchestra.run.smithi157.stdout:942 clone 5 order 21 su 16384 sc 2 2024-09-13T09:12:26.332 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:26.331+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:26.387 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:26.387 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:26.396 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:26.426 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T09:12:26.433 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:26.432+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:26.441 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T09:12:26.512 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:26.512 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:26.521 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:26.562 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:26.561+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:26.635 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:26.634+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:26.671 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:26.671 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:26.680 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:26.735 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:26.734+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:26.833 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:26.833 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:26.836 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:26.835+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:26.842 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:26.905 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-13T09:12:26.937 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:26.935+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:26.949 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:26.949 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:26.957 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:26.996 INFO:teuthology.orchestra.run.smithi157.stdout:943 write 0x3a51a00 thru 0x3a51cdb (0x2dc bytes) 2024-09-13T09:12:26.997 INFO:teuthology.orchestra.run.smithi157.stdout:944 write 0xdd83000 thru 0xdd8eaa2 (0xbaa3 bytes) 2024-09-13T09:12:27.010 INFO:teuthology.orchestra.run.smithi157.stdout:945 trunc from 0xe2c8600 to 0x4075c00 2024-09-13T09:12:27.043 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:27.041+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:27.050 INFO:teuthology.orchestra.run.smithi157.stdout:947 trunc from 0x4075c00 to 0x574c00 2024-09-13T09:12:27.092 INFO:teuthology.orchestra.run.smithi157.stdout:948 read 0x10c200 thru 0x10cea2 (0xca3 bytes) 2024-09-13T09:12:27.097 INFO:teuthology.orchestra.run.smithi157.stdout:949 trunc from 0x574c00 to 0x40d6800 2024-09-13T09:12:27.110 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:27.110 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:27.111 INFO:teuthology.orchestra.run.smithi157.stdout:950 punch from 0x227d600 to 0x2284c00, (0x7600 bytes) 2024-09-13T09:12:27.114 INFO:teuthology.orchestra.run.smithi157.stdout:951 trunc from 0x40d6800 to 0x6991000 2024-09-13T09:12:27.121 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:27.130 INFO:teuthology.orchestra.run.smithi157.stdout:952 trunc from 0x6991000 to 0x8b0b800 2024-09-13T09:12:27.145 INFO:teuthology.orchestra.run.smithi157.stdout:953 trunc from 0x8b0b800 to 0xe969e00 2024-09-13T09:12:27.156 INFO:teuthology.orchestra.run.smithi157.stdout:954 write 0x18ce400 thru 0x18d5d6d (0x796e bytes) 2024-09-13T09:12:27.160 INFO:teuthology.orchestra.run.smithi157.stdout:956 trunc from 0xe969e00 to 0xb001c00 2024-09-13T09:12:27.163 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:27.161+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:27.172 INFO:teuthology.orchestra.run.smithi157.stdout:957 punch from 0x12dbc00 to 0x12e8c00, (0xd000 bytes) 2024-09-13T09:12:27.181 INFO:teuthology.orchestra.run.smithi157.stdout:963 write 0x21a400 thru 0x220cb2 (0x68b3 bytes) 2024-09-13T09:12:27.181 INFO:teuthology.orchestra.run.smithi157.stdout:968 punch from 0x7de2800 to 0x7de8000, (0x5800 bytes) 2024-09-13T09:12:27.184 INFO:teuthology.orchestra.run.smithi157.stdout:969 read 0x9d71400 thru 0x9d8103f (0xfc40 bytes) 2024-09-13T09:12:27.184 INFO:teuthology.orchestra.run.smithi157.stdout:972 punch from 0xa08800 to 0xa0c800, (0x4000 bytes) 2024-09-13T09:12:27.184 INFO:teuthology.orchestra.run.smithi157.stdout:974 read 0x5fef800 thru 0x5ff4247 (0x4a48 bytes) 2024-09-13T09:12:27.217 INFO:teuthology.orchestra.run.smithi157.stdout:975 write 0xbbf4400 thru 0xbbf52bd (0xebe bytes) 2024-09-13T09:12:27.237 INFO:teuthology.orchestra.run.smithi157.stdout:976 trunc from 0xbbf5400 to 0x19e600 2024-09-13T09:12:27.239 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:27.238+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:27.259 INFO:teuthology.orchestra.run.smithi157.stdout:977 clone 6 order 24 su 1048576 sc 7 2024-09-13T09:12:27.271 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:27.272 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:27.280 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:27.339 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:27.338+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:27.383 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:27.383 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:27.392 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:27.434 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T09:12:27.436 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:12:27.453 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T09:12:27.475 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:27.474+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:leaving image image_client.0-clone5 intact 2024-09-13T09:12:27.537 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:27.537 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:27.541 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:27.539+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:27.546 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:27.642 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:27.640+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:27.693 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:27.693 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:27.702 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:27.743 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:27.741+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:27.752 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-13T09:12:27.801 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:12:27.843 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:27.842+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:27.851 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:27.851 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:27.861 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:27.943 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:27.944 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:27.944 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:27.943+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:27.968 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:27.969 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:27.994 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:27.994 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:28.021 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:28.021 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:28.030 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:28.031 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:28.041 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:28.043 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:28.043 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:28.044 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:28.043+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:28.062 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:28.062 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:28.145 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:28.144+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:28.164 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:28.165 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:28.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:28.165 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:28.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.165 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4221.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:12:28.165 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:04.320224+0000", 2024-09-13T09:12:28.165 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.842084108000002, 2024-09-13T09:12:28.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010683929, 2024-09-13T09:12:28.165 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.165 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.165 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.166 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4221", 2024-09-13T09:12:28.166 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.44:0/558773684", 2024-09-13T09:12:28.166 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1 2024-09-13T09:12:28.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.166 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320224+0000", 2024-09-13T09:12:28.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320224+0000", 2024-09-13T09:12:28.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320220+0000", 2024-09-13T09:12:28.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999967 2024-09-13T09:12:28.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320230+0000", 2024-09-13T09:12:28.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0275000000000001e-05 2024-09-13T09:12:28.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320241+0000", 2024-09-13T09:12:28.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0154e-05 2024-09-13T09:12:28.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320256+0000", 2024-09-13T09:12:28.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5068999999999999e-05 2024-09-13T09:12:28.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320328+0000", 2024-09-13T09:12:28.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2507000000000002e-05 2024-09-13T09:12:28.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:28.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320336+0000", 2024-09-13T09:12:28.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:12:28.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.322475+0000", 2024-09-13T09:12:28.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021399430000000001 2024-09-13T09:12:28.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:28.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.322484+0000", 2024-09-13T09:12:28.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5779999999999999e-06 2024-09-13T09:12:28.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.325065+0000", 2024-09-13T09:12:28.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002581323 2024-09-13T09:12:28.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.325211+0000", 2024-09-13T09:12:28.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014534899999999999 2024-09-13T09:12:28.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:28.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.325374+0000", 2024-09-13T09:12:28.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000162898 2024-09-13T09:12:28.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.326528+0000", 2024-09-13T09:12:28.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001154415 2024-09-13T09:12:28.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.330811+0000", 2024-09-13T09:12:28.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042825090000000003 2024-09-13T09:12:28.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.330862+0000", 2024-09-13T09:12:28.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1430000000000001e-05 2024-09-13T09:12:28.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.330908+0000", 2024-09-13T09:12:28.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5636000000000001e-05 2024-09-13T09:12:28.173 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.173 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.173 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.173 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:12:28.173 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:16.385493+0000", 2024-09-13T09:12:28.173 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.776814302, 2024-09-13T09:12:28.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063914480000000001, 2024-09-13T09:12:28.173 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.174 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:12:28.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:12:28.174 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 10 2024-09-13T09:12:28.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.174 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385493+0000", 2024-09-13T09:12:28.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385493+0000", 2024-09-13T09:12:28.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385492+0000", 2024-09-13T09:12:28.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:28.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385498+0000", 2024-09-13T09:12:28.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2219999999999999e-06 2024-09-13T09:12:28.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385509+0000", 2024-09-13T09:12:28.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1207e-05 2024-09-13T09:12:28.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385519+0000", 2024-09-13T09:12:28.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8560000000000002e-06 2024-09-13T09:12:28.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385560+0000", 2024-09-13T09:12:28.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1223000000000002e-05 2024-09-13T09:12:28.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:28.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.385565+0000", 2024-09-13T09:12:28.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9409999999999998e-06 2024-09-13T09:12:28.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388159+0000", 2024-09-13T09:12:28.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0025934 2024-09-13T09:12:28.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388261+0000", 2024-09-13T09:12:28.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102799 2024-09-13T09:12:28.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:28.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.388406+0000", 2024-09-13T09:12:28.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000144857 2024-09-13T09:12:28.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.389373+0000", 2024-09-13T09:12:28.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00096632900000000004 2024-09-13T09:12:28.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.391797+0000", 2024-09-13T09:12:28.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0024246580000000001 2024-09-13T09:12:28.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.391847+0000", 2024-09-13T09:12:28.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9811999999999997e-05 2024-09-13T09:12:28.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.391885+0000", 2024-09-13T09:12:28.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7796999999999997e-05 2024-09-13T09:12:28.180 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.180 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.180 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.180 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1459 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:28.180 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.121753+0000", 2024-09-13T09:12:28.180 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.0405552460000003, 2024-09-13T09:12:28.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065126389999999998, 2024-09-13T09:12:28.180 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.180 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.181 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.181 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:28.181 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:28.181 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1459 2024-09-13T09:12:28.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.181 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121753+0000", 2024-09-13T09:12:28.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121753+0000", 2024-09-13T09:12:28.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121752+0000", 2024-09-13T09:12:28.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121754+0000", 2024-09-13T09:12:28.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9750000000000001e-06 2024-09-13T09:12:28.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121759+0000", 2024-09-13T09:12:28.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0239999999999997e-06 2024-09-13T09:12:28.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121762+0000", 2024-09-13T09:12:28.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8049999999999998e-06 2024-09-13T09:12:28.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122595+0000", 2024-09-13T09:12:28.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00083291799999999998 2024-09-13T09:12:28.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122697+0000", 2024-09-13T09:12:28.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101727 2024-09-13T09:12:28.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:28.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122961+0000", 2024-09-13T09:12:28.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026407399999999999 2024-09-13T09:12:28.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124336+0000", 2024-09-13T09:12:28.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0013751270000000001 2024-09-13T09:12:28.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128157+0000", 2024-09-13T09:12:28.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038212419999999999 2024-09-13T09:12:28.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128205+0000", 2024-09-13T09:12:28.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7722999999999997e-05 2024-09-13T09:12:28.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128265+0000", 2024-09-13T09:12:28.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.033e-05 2024-09-13T09:12:28.186 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.186 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.186 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.186 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1460 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:28.186 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.121842+0000", 2024-09-13T09:12:28.186 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.0404655030000001, 2024-09-13T09:12:28.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065154749999999997, 2024-09-13T09:12:28.186 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.187 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:28.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:28.187 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1460 2024-09-13T09:12:28.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.187 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121842+0000", 2024-09-13T09:12:28.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121842+0000", 2024-09-13T09:12:28.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121842+0000", 2024-09-13T09:12:28.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121845+0000", 2024-09-13T09:12:28.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:12:28.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121849+0000", 2024-09-13T09:12:28.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2540000000000003e-06 2024-09-13T09:12:28.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.121852+0000", 2024-09-13T09:12:28.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8710000000000001e-06 2024-09-13T09:12:28.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.123228+0000", 2024-09-13T09:12:28.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001376592 2024-09-13T09:12:28.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.123280+0000", 2024-09-13T09:12:28.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1975999999999998e-05 2024-09-13T09:12:28.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:28.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.123509+0000", 2024-09-13T09:12:28.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022916000000000001 2024-09-13T09:12:28.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.125491+0000", 2024-09-13T09:12:28.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0019817379999999998 2024-09-13T09:12:28.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128275+0000", 2024-09-13T09:12:28.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002784201 2024-09-13T09:12:28.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128328+0000", 2024-09-13T09:12:28.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3006e-05 2024-09-13T09:12:28.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128358+0000", 2024-09-13T09:12:28.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9397e-05 2024-09-13T09:12:28.192 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.192 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.192 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.192 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1462 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:28.192 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.122137+0000", 2024-09-13T09:12:28.193 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.0401711920000007, 2024-09-13T09:12:28.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063313809999999996, 2024-09-13T09:12:28.193 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.193 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.193 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.193 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:28.193 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:28.193 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1462 2024-09-13T09:12:28.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.193 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122137+0000", 2024-09-13T09:12:28.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122137+0000", 2024-09-13T09:12:28.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122136+0000", 2024-09-13T09:12:28.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:28.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122140+0000", 2024-09-13T09:12:28.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.425e-06 2024-09-13T09:12:28.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122148+0000", 2024-09-13T09:12:28.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1170000000000004e-06 2024-09-13T09:12:28.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122154+0000", 2024-09-13T09:12:28.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5990000000000001e-06 2024-09-13T09:12:28.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.123777+0000", 2024-09-13T09:12:28.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00162301 2024-09-13T09:12:28.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.123826+0000", 2024-09-13T09:12:28.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9051999999999997e-05 2024-09-13T09:12:28.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:28.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124038+0000", 2024-09-13T09:12:28.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021218499999999999 2024-09-13T09:12:28.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.125699+0000", 2024-09-13T09:12:28.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0016603589999999999 2024-09-13T09:12:28.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128373+0000", 2024-09-13T09:12:28.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026739770000000001 2024-09-13T09:12:28.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128426+0000", 2024-09-13T09:12:28.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3643000000000001e-05 2024-09-13T09:12:28.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128468+0000", 2024-09-13T09:12:28.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1826000000000002e-05 2024-09-13T09:12:28.198 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.198 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.198 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.198 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1463 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:28.198 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.122162+0000", 2024-09-13T09:12:28.198 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.0401459469999992, 2024-09-13T09:12:28.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064746990000000004, 2024-09-13T09:12:28.199 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.199 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.199 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.199 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:28.199 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:28.199 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1463 2024-09-13T09:12:28.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.199 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122162+0000", 2024-09-13T09:12:28.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122162+0000", 2024-09-13T09:12:28.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122161+0000", 2024-09-13T09:12:28.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122165+0000", 2024-09-13T09:12:28.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5209999999999998e-06 2024-09-13T09:12:28.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122172+0000", 2024-09-13T09:12:28.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2540000000000003e-06 2024-09-13T09:12:28.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122177+0000", 2024-09-13T09:12:28.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7979999999999996e-06 2024-09-13T09:12:28.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124358+0000", 2024-09-13T09:12:28.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021809350000000002 2024-09-13T09:12:28.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124407+0000", 2024-09-13T09:12:28.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9469999999999999e-05 2024-09-13T09:12:28.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:28.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124633+0000", 2024-09-13T09:12:28.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022538300000000001 2024-09-13T09:12:28.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.126674+0000", 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020412540000000002 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128475+0000", 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001801137 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128598+0000", 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012318600000000001 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128637+0000", 2024-09-13T09:12:28.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8219000000000001e-05 2024-09-13T09:12:28.204 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.204 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.204 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.204 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1465 3.5 3:a2e7e3cb:::rbd_object_map.11238d3f93be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:28.204 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.122262+0000", 2024-09-13T09:12:28.204 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.0400453330000001, 2024-09-13T09:12:28.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006503043, 2024-09-13T09:12:28.204 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.204 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.204 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:28.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:28.205 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1465 2024-09-13T09:12:28.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.205 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.205 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.205 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122262+0000", 2024-09-13T09:12:28.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.205 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.205 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122262+0000", 2024-09-13T09:12:28.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122261+0000", 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122268+0000", 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1720000000000004e-06 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122277+0000", 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7459999999999995e-06 2024-09-13T09:12:28.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.122284+0000", 2024-09-13T09:12:28.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.511e-06 2024-09-13T09:12:28.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124897+0000", 2024-09-13T09:12:28.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026131420000000002 2024-09-13T09:12:28.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.124947+0000", 2024-09-13T09:12:28.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9543000000000003e-05 2024-09-13T09:12:28.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:28.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.125149+0000", 2024-09-13T09:12:28.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000202709 2024-09-13T09:12:28.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.126698+0000", 2024-09-13T09:12:28.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015487649999999999 2024-09-13T09:12:28.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128645+0000", 2024-09-13T09:12:28.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001946632 2024-09-13T09:12:28.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128684+0000", 2024-09-13T09:12:28.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9860000000000001e-05 2024-09-13T09:12:28.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.128766+0000", 2024-09-13T09:12:28.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1025000000000004e-05 2024-09-13T09:12:28.209 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.209 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.210 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.210 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4483.0:1067 3.f 3:f00ae4d0:::rbd_data.1123ffac6bc2.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 897024~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:28.210 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:20.142567+0000", 2024-09-13T09:12:28.210 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.0197409220000004, 2024-09-13T09:12:28.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059237839999999996, 2024-09-13T09:12:28.210 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.210 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4483", 2024-09-13T09:12:28.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:28.210 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1067 2024-09-13T09:12:28.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.211 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142567+0000", 2024-09-13T09:12:28.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142567+0000", 2024-09-13T09:12:28.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142566+0000", 2024-09-13T09:12:28.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:28.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142578+0000", 2024-09-13T09:12:28.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2668e-05 2024-09-13T09:12:28.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142588+0000", 2024-09-13T09:12:28.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5980000000000003e-06 2024-09-13T09:12:28.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142595+0000", 2024-09-13T09:12:28.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0199999999999997e-06 2024-09-13T09:12:28.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142629+0000", 2024-09-13T09:12:28.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4351999999999999e-05 2024-09-13T09:12:28.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142712+0000", 2024-09-13T09:12:28.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2913999999999999e-05 2024-09-13T09:12:28.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:28.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142830+0000", 2024-09-13T09:12:28.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011835 2024-09-13T09:12:28.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.147783+0000", 2024-09-13T09:12:28.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0049524180000000001 2024-09-13T09:12:28.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.148393+0000", 2024-09-13T09:12:28.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00061000699999999998 2024-09-13T09:12:28.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.148456+0000", 2024-09-13T09:12:28.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2996999999999993e-05 2024-09-13T09:12:28.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.148491+0000", 2024-09-13T09:12:28.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4814000000000002e-05 2024-09-13T09:12:28.215 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.215 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.216 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.216 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4483.0:1068 3.c 3:313e6343:::rbd_data.1123ffac6bc2.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 786432~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:28.216 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:20.142607+0000", 2024-09-13T09:12:28.216 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.0197004169999992, 2024-09-13T09:12:28.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0058839460000000001, 2024-09-13T09:12:28.216 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.216 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.216 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.216 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4483", 2024-09-13T09:12:28.216 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:28.216 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1068 2024-09-13T09:12:28.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.217 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142607+0000", 2024-09-13T09:12:28.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142607+0000", 2024-09-13T09:12:28.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142607+0000", 2024-09-13T09:12:28.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142617+0000", 2024-09-13T09:12:28.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0339e-05 2024-09-13T09:12:28.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142626+0000", 2024-09-13T09:12:28.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5839999999999995e-06 2024-09-13T09:12:28.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142631+0000", 2024-09-13T09:12:28.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4319999999999998e-06 2024-09-13T09:12:28.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142669+0000", 2024-09-13T09:12:28.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7571999999999999e-05 2024-09-13T09:12:28.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142759+0000", 2024-09-13T09:12:28.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0662999999999996e-05 2024-09-13T09:12:28.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:28.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.142869+0000", 2024-09-13T09:12:28.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000109734 2024-09-13T09:12:28.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.147790+0000", 2024-09-13T09:12:28.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004921295 2024-09-13T09:12:28.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.148415+0000", 2024-09-13T09:12:28.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00062403000000000003 2024-09-13T09:12:28.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.148462+0000", 2024-09-13T09:12:28.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7933000000000003e-05 2024-09-13T09:12:28.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.148491+0000", 2024-09-13T09:12:28.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8873e-05 2024-09-13T09:12:28.221 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.221 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.222 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.222 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4483.0:1659 3.2 e28/21 3:436e1b7d:::rbd_object_map.1123ffac6bc2:head v 28'145, mlcod=28'142)", 2024-09-13T09:12:28.222 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:20.418410+0000", 2024-09-13T09:12:28.222 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7438977549999999, 2024-09-13T09:12:28.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0058268540000000002, 2024-09-13T09:12:28.222 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.222 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.222 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418410+0000", 2024-09-13T09:12:28.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418410+0000", 2024-09-13T09:12:28.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418410+0000", 2024-09-13T09:12:28.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418413+0000", 2024-09-13T09:12:28.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7900000000000001e-06 2024-09-13T09:12:28.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418420+0000", 2024-09-13T09:12:28.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:12:28.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418423+0000", 2024-09-13T09:12:28.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7189999999999999e-06 2024-09-13T09:12:28.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418459+0000", 2024-09-13T09:12:28.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5822999999999999e-05 2024-09-13T09:12:28.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.418479+0000", 2024-09-13T09:12:28.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9714000000000001e-05 2024-09-13T09:12:28.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.424202+0000", 2024-09-13T09:12:28.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057233350000000004 2024-09-13T09:12:28.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.424237+0000", 2024-09-13T09:12:28.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4691999999999999e-05 2024-09-13T09:12:28.226 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.226 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.226 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.226 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4546.0:2175 3.10 3:0eca404c:::rbd_data.1123ffac6bc2.0000000000000080:head [sparse-read 4096~126976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:28.226 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.048825+0000", 2024-09-13T09:12:28.226 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.113482715, 2024-09-13T09:12:28.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026988175999999999, 2024-09-13T09:12:28.227 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.227 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:28.227 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.227 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4546", 2024-09-13T09:12:28.227 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2955085030", 2024-09-13T09:12:28.227 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2175 2024-09-13T09:12:28.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.227 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048825+0000", 2024-09-13T09:12:28.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048825+0000", 2024-09-13T09:12:28.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048825+0000", 2024-09-13T09:12:28.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048828+0000", 2024-09-13T09:12:28.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2160000000000002e-06 2024-09-13T09:12:28.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048837+0000", 2024-09-13T09:12:28.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6180000000000001e-06 2024-09-13T09:12:28.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048841+0000", 2024-09-13T09:12:28.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5389999999999999e-06 2024-09-13T09:12:28.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048952+0000", 2024-09-13T09:12:28.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011115 2024-09-13T09:12:28.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.075813+0000", 2024-09-13T09:12:28.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026861182000000001 2024-09-13T09:12:28.230 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.230 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.230 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.230 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4546.0:2178 3.13 3:c89627a6:::rbd_data.1123ffac6bc2.0000000000000082:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:28.230 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.049364+0000", 2024-09-13T09:12:28.230 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1129436830000001, 2024-09-13T09:12:28.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026490942999999999, 2024-09-13T09:12:28.231 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.231 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:28.231 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.231 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4546", 2024-09-13T09:12:28.231 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2955085030", 2024-09-13T09:12:28.231 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2178 2024-09-13T09:12:28.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.231 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.049364+0000", 2024-09-13T09:12:28.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.049364+0000", 2024-09-13T09:12:28.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.049363+0000", 2024-09-13T09:12:28.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.049367+0000", 2024-09-13T09:12:28.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4960000000000001e-06 2024-09-13T09:12:28.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.075707+0000", 2024-09-13T09:12:28.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026339912 2024-09-13T09:12:28.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.075727+0000", 2024-09-13T09:12:28.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9898000000000002e-05 2024-09-13T09:12:28.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.075778+0000", 2024-09-13T09:12:28.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1617999999999999e-05 2024-09-13T09:12:28.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.075855+0000", 2024-09-13T09:12:28.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6644999999999995e-05 2024-09-13T09:12:28.234 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.234 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.235 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.235 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1551 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:28.235 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510403+0000", 2024-09-13T09:12:28.235 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.651904842, 2024-09-13T09:12:28.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0058898470000000001, 2024-09-13T09:12:28.235 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.235 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:28.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:28.235 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1551 2024-09-13T09:12:28.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.236 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510403+0000", 2024-09-13T09:12:28.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510403+0000", 2024-09-13T09:12:28.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510403+0000", 2024-09-13T09:12:28.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510406+0000", 2024-09-13T09:12:28.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.219e-06 2024-09-13T09:12:28.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510409+0000", 2024-09-13T09:12:28.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5939999999999998e-06 2024-09-13T09:12:28.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510412+0000", 2024-09-13T09:12:28.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5229999999999998e-06 2024-09-13T09:12:28.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.512770+0000", 2024-09-13T09:12:28.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002358537 2024-09-13T09:12:28.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.512853+0000", 2024-09-13T09:12:28.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2608999999999995e-05 2024-09-13T09:12:28.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:28.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.513186+0000", 2024-09-13T09:12:28.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000332617 2024-09-13T09:12:28.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516155+0000", 2024-09-13T09:12:28.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0029687939999999999 2024-09-13T09:12:28.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516203+0000", 2024-09-13T09:12:28.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8265e-05 2024-09-13T09:12:28.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516253+0000", 2024-09-13T09:12:28.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0538999999999998e-05 2024-09-13T09:12:28.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516293+0000", 2024-09-13T09:12:28.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9490000000000003e-05 2024-09-13T09:12:28.240 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.241 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.241 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.241 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1552 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:28.241 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510415+0000", 2024-09-13T09:12:28.241 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.6518931700000001, 2024-09-13T09:12:28.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059682420000000003, 2024-09-13T09:12:28.241 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.241 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.241 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.241 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:28.241 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:28.242 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1552 2024-09-13T09:12:28.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.242 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510415+0000", 2024-09-13T09:12:28.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510415+0000", 2024-09-13T09:12:28.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510414+0000", 2024-09-13T09:12:28.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:28.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510416+0000", 2024-09-13T09:12:28.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:12:28.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510420+0000", 2024-09-13T09:12:28.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7000000000000002e-06 2024-09-13T09:12:28.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510423+0000", 2024-09-13T09:12:28.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6599999999999999e-06 2024-09-13T09:12:28.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.513672+0000", 2024-09-13T09:12:28.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0032489179999999999 2024-09-13T09:12:28.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.513761+0000", 2024-09-13T09:12:28.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9604999999999993e-05 2024-09-13T09:12:28.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:28.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.514081+0000", 2024-09-13T09:12:28.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00031942599999999999 2024-09-13T09:12:28.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516158+0000", 2024-09-13T09:12:28.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002077162 2024-09-13T09:12:28.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516344+0000", 2024-09-13T09:12:28.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018647800000000001 2024-09-13T09:12:28.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516366+0000", 2024-09-13T09:12:28.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1488000000000002e-05 2024-09-13T09:12:28.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516383+0000", 2024-09-13T09:12:28.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7030000000000001e-05 2024-09-13T09:12:28.246 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.246 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.246 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.247 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1553 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:28.247 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510425+0000", 2024-09-13T09:12:28.247 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.651882837, 2024-09-13T09:12:28.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0089516189999999992, 2024-09-13T09:12:28.247 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.247 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.247 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.247 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:28.247 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:28.247 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1553 2024-09-13T09:12:28.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.247 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510425+0000", 2024-09-13T09:12:28.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510425+0000", 2024-09-13T09:12:28.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510425+0000", 2024-09-13T09:12:28.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510426+0000", 2024-09-13T09:12:28.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.72e-06 2024-09-13T09:12:28.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510434+0000", 2024-09-13T09:12:28.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7649999999999993e-06 2024-09-13T09:12:28.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510441+0000", 2024-09-13T09:12:28.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8449999999999997e-06 2024-09-13T09:12:28.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.514480+0000", 2024-09-13T09:12:28.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0040390199999999999 2024-09-13T09:12:28.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.514545+0000", 2024-09-13T09:12:28.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4834999999999998e-05 2024-09-13T09:12:28.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:28.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.514776+0000", 2024-09-13T09:12:28.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023143 2024-09-13T09:12:28.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519286+0000", 2024-09-13T09:12:28.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045099720000000001 2024-09-13T09:12:28.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519296+0000", 2024-09-13T09:12:28.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-06 2024-09-13T09:12:28.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519342+0000", 2024-09-13T09:12:28.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6436000000000001e-05 2024-09-13T09:12:28.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519377+0000", 2024-09-13T09:12:28.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4269e-05 2024-09-13T09:12:28.252 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.252 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.252 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.252 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1554 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:28.253 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510444+0000", 2024-09-13T09:12:28.253 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.6518640229999999, 2024-09-13T09:12:28.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0090383259999999993, 2024-09-13T09:12:28.253 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.253 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.253 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.253 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:28.253 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:28.253 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1554 2024-09-13T09:12:28.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.253 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510444+0000", 2024-09-13T09:12:28.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510444+0000", 2024-09-13T09:12:28.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510444+0000", 2024-09-13T09:12:28.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:28.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510445+0000", 2024-09-13T09:12:28.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.951e-06 2024-09-13T09:12:28.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510449+0000", 2024-09-13T09:12:28.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3569999999999998e-06 2024-09-13T09:12:28.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510456+0000", 2024-09-13T09:12:28.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6599999999999998e-06 2024-09-13T09:12:28.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.515200+0000", 2024-09-13T09:12:28.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0047443889999999999 2024-09-13T09:12:28.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.515286+0000", 2024-09-13T09:12:28.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5626999999999999e-05 2024-09-13T09:12:28.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:28.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.515685+0000", 2024-09-13T09:12:28.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039960899999999999 2024-09-13T09:12:28.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519386+0000", 2024-09-13T09:12:28.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0037010659999999998 2024-09-13T09:12:28.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519426+0000", 2024-09-13T09:12:28.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9929999999999999e-05 2024-09-13T09:12:28.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519458+0000", 2024-09-13T09:12:28.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2189e-05 2024-09-13T09:12:28.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519482+0000", 2024-09-13T09:12:28.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3779999999999999e-05 2024-09-13T09:12:28.258 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.259 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.259 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.259 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1555 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:28.259 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510460+0000", 2024-09-13T09:12:28.259 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.6518481830000002, 2024-09-13T09:12:28.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0091250399999999992, 2024-09-13T09:12:28.259 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.259 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.259 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.259 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:28.259 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:28.260 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1555 2024-09-13T09:12:28.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.260 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510460+0000", 2024-09-13T09:12:28.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510460+0000", 2024-09-13T09:12:28.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510459+0000", 2024-09-13T09:12:28.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510464+0000", 2024-09-13T09:12:28.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8709999999999996e-06 2024-09-13T09:12:28.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510469+0000", 2024-09-13T09:12:28.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:12:28.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510472+0000", 2024-09-13T09:12:28.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.813e-06 2024-09-13T09:12:28.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516420+0000", 2024-09-13T09:12:28.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059477749999999998 2024-09-13T09:12:28.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516523+0000", 2024-09-13T09:12:28.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103222 2024-09-13T09:12:28.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:28.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516877+0000", 2024-09-13T09:12:28.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00035438100000000002 2024-09-13T09:12:28.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519391+0000", 2024-09-13T09:12:28.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002514166 2024-09-13T09:12:28.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519520+0000", 2024-09-13T09:12:28.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000128582 2024-09-13T09:12:28.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519566+0000", 2024-09-13T09:12:28.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6224999999999999e-05 2024-09-13T09:12:28.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519585+0000", 2024-09-13T09:12:28.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8391000000000001e-05 2024-09-13T09:12:28.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.265 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.265 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1556 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:28.265 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510474+0000", 2024-09-13T09:12:28.265 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.651833361, 2024-09-13T09:12:28.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0091831980000000001, 2024-09-13T09:12:28.265 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.265 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.266 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.266 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:28.266 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:28.266 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1556 2024-09-13T09:12:28.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.266 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510474+0000", 2024-09-13T09:12:28.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510474+0000", 2024-09-13T09:12:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510474+0000", 2024-09-13T09:12:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510476+0000", 2024-09-13T09:12:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1799999999999999e-06 2024-09-13T09:12:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510479+0000", 2024-09-13T09:12:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1159999999999999e-06 2024-09-13T09:12:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510482+0000", 2024-09-13T09:12:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:12:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.517292+0000", 2024-09-13T09:12:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068100239999999996 2024-09-13T09:12:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.517344+0000", 2024-09-13T09:12:28.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2092999999999997e-05 2024-09-13T09:12:28.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:28.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.517558+0000", 2024-09-13T09:12:28.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021441900000000001 2024-09-13T09:12:28.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519397+0000", 2024-09-13T09:12:28.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001838891 2024-09-13T09:12:28.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519617+0000", 2024-09-13T09:12:28.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021970400000000001 2024-09-13T09:12:28.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519639+0000", 2024-09-13T09:12:28.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1773999999999999e-05 2024-09-13T09:12:28.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519658+0000", 2024-09-13T09:12:28.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8787000000000002e-05 2024-09-13T09:12:28.270 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.271 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.271 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.271 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1557 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:28.271 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510490+0000", 2024-09-13T09:12:28.271 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.6518177650000001, 2024-09-13T09:12:28.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0092238319999999995, 2024-09-13T09:12:28.271 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.271 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.271 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.271 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:28.271 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:28.271 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1557 2024-09-13T09:12:28.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.272 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510490+0000", 2024-09-13T09:12:28.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510490+0000", 2024-09-13T09:12:28.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510490+0000", 2024-09-13T09:12:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510492+0000", 2024-09-13T09:12:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-13T09:12:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510495+0000", 2024-09-13T09:12:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5020000000000001e-06 2024-09-13T09:12:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510498+0000", 2024-09-13T09:12:28.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6809999999999998e-06 2024-09-13T09:12:28.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.517928+0000", 2024-09-13T09:12:28.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0074299259999999999 2024-09-13T09:12:28.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.518005+0000", 2024-09-13T09:12:28.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6642000000000002e-05 2024-09-13T09:12:28.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:28.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.518340+0000", 2024-09-13T09:12:28.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00033502400000000001 2024-09-13T09:12:28.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519592+0000", 2024-09-13T09:12:28.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001252647 2024-09-13T09:12:28.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519677+0000", 2024-09-13T09:12:28.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4751000000000002e-05 2024-09-13T09:12:28.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519698+0000", 2024-09-13T09:12:28.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0792999999999999e-05 2024-09-13T09:12:28.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519714+0000", 2024-09-13T09:12:28.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6008e-05 2024-09-13T09:12:28.276 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.277 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.277 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.277 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1558 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:28.277 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510501+0000", 2024-09-13T09:12:28.277 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.6518067360000002, 2024-09-13T09:12:28.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0094031559999999993, 2024-09-13T09:12:28.277 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:28.277 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.277 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:28.277 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:28.277 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:28.278 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1558 2024-09-13T09:12:28.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.278 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:28.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:28.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510501+0000", 2024-09-13T09:12:28.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:28.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510501+0000", 2024-09-13T09:12:28.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:28.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:28.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510501+0000", 2024-09-13T09:12:28.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:28.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:28.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510503+0000", 2024-09-13T09:12:28.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.72e-06 2024-09-13T09:12:28.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:28.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510506+0000", 2024-09-13T09:12:28.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.286e-06 2024-09-13T09:12:28.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510508+0000", 2024-09-13T09:12:28.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-13T09:12:28.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:28.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.518739+0000", 2024-09-13T09:12:28.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0082309080000000003 2024-09-13T09:12:28.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:28.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.518787+0000", 2024-09-13T09:12:28.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8214999999999998e-05 2024-09-13T09:12:28.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:28.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519003+0000", 2024-09-13T09:12:28.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021570199999999999 2024-09-13T09:12:28.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:28.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519720+0000", 2024-09-13T09:12:28.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00071655099999999995 2024-09-13T09:12:28.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519868+0000", 2024-09-13T09:12:28.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014865000000000001 2024-09-13T09:12:28.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:28.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519888+0000", 2024-09-13T09:12:28.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0177e-05 2024-09-13T09:12:28.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:28.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:28.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:28.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519904+0000", 2024-09-13T09:12:28.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5767999999999999e-05 2024-09-13T09:12:28.283 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.283 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.283 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.283 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:28.283 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:28.283 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:28.285 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:28.282+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:28.288 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:28.385 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:28.384+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:28.398 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:28.399 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:28.399 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:28.399 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:28.407 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:28.453 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T09:12:28.456 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-09-13T09:12:28.467 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T09:12:28.511 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:28.510+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:28.518 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:28.518 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:28.526 INFO:teuthology.orchestra.run.smithi157.stdout:978 punch from 0x178e00 to 0x184600, (0xb800 bytes) 2024-09-13T09:12:28.527 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:28.613 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:28.612+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:28.678 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:28.678 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:28.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:28.678 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:28.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.678 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1321 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:28.678 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.242418+0000", 2024-09-13T09:12:28.679 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.4331451479999999, 2024-09-13T09:12:28.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0087640029999999994, 2024-09-13T09:12:28.679 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.679 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.679 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1321 2024-09-13T09:12:28.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.679 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.679 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.679 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242418+0000", 2024-09-13T09:12:28.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242418+0000", 2024-09-13T09:12:28.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242418+0000", 2024-09-13T09:12:28.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242419+0000", 2024-09-13T09:12:28.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7740000000000001e-06 2024-09-13T09:12:28.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242424+0000", 2024-09-13T09:12:28.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4839999999999998e-06 2024-09-13T09:12:28.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242427+0000", 2024-09-13T09:12:28.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8660000000000002e-06 2024-09-13T09:12:28.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.244978+0000", 2024-09-13T09:12:28.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002551139 2024-09-13T09:12:28.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.245025+0000", 2024-09-13T09:12:28.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6832999999999997e-05 2024-09-13T09:12:28.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.245221+0000", 2024-09-13T09:12:28.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000195954 2024-09-13T09:12:28.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.246365+0000", 2024-09-13T09:12:28.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011444669999999999 2024-09-13T09:12:28.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.251115+0000", 2024-09-13T09:12:28.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047497779999999996 2024-09-13T09:12:28.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.251150+0000", 2024-09-13T09:12:28.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4687999999999997e-05 2024-09-13T09:12:28.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.251182+0000", 2024-09-13T09:12:28.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2264000000000001e-05 2024-09-13T09:12:28.684 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.685 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.685 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.685 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1550 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:28.685 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.329689+0000", 2024-09-13T09:12:28.685 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.3458744739999999, 2024-09-13T09:12:28.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.009203391, 2024-09-13T09:12:28.685 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.685 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.685 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.685 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.685 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.686 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1550 2024-09-13T09:12:28.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.686 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.686 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.686 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329689+0000", 2024-09-13T09:12:28.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.686 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.686 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329689+0000", 2024-09-13T09:12:28.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329688+0000", 2024-09-13T09:12:28.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329690+0000", 2024-09-13T09:12:28.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7e-06 2024-09-13T09:12:28.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329695+0000", 2024-09-13T09:12:28.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4880000000000001e-06 2024-09-13T09:12:28.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329697+0000", 2024-09-13T09:12:28.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.74e-06 2024-09-13T09:12:28.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.333003+0000", 2024-09-13T09:12:28.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003305451 2024-09-13T09:12:28.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.333084+0000", 2024-09-13T09:12:28.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1062000000000004e-05 2024-09-13T09:12:28.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.333427+0000", 2024-09-13T09:12:28.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000342932 2024-09-13T09:12:28.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338607+0000", 2024-09-13T09:12:28.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051806580000000003 2024-09-13T09:12:28.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338821+0000", 2024-09-13T09:12:28.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00021341099999999999 2024-09-13T09:12:28.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338860+0000", 2024-09-13T09:12:28.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9320000000000003e-05 2024-09-13T09:12:28.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338892+0000", 2024-09-13T09:12:28.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1868e-05 2024-09-13T09:12:28.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.691 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.691 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.691 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1551 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:28.691 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.329715+0000", 2024-09-13T09:12:28.691 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.345847622, 2024-09-13T09:12:28.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0093095490000000003, 2024-09-13T09:12:28.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.691 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.692 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1551 2024-09-13T09:12:28.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.692 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329715+0000", 2024-09-13T09:12:28.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329715+0000", 2024-09-13T09:12:28.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329715+0000", 2024-09-13T09:12:28.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329718+0000", 2024-09-13T09:12:28.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8279999999999999e-06 2024-09-13T09:12:28.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329726+0000", 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0420000000000003e-06 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329729+0000", 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1099999999999999e-06 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.333843+0000", 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0041141499999999996 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.333899+0000", 2024-09-13T09:12:28.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6273000000000001e-05 2024-09-13T09:12:28.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.334107+0000", 2024-09-13T09:12:28.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00020787400000000001 2024-09-13T09:12:28.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338610+0000", 2024-09-13T09:12:28.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004502944 2024-09-13T09:12:28.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338943+0000", 2024-09-13T09:12:28.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033260000000000001 2024-09-13T09:12:28.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338980+0000", 2024-09-13T09:12:28.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7382999999999998e-05 2024-09-13T09:12:28.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.339025+0000", 2024-09-13T09:12:28.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4765999999999998e-05 2024-09-13T09:12:28.696 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.696 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.697 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.697 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1990 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.697 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.504086+0000", 2024-09-13T09:12:28.697 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.1714766829999999, 2024-09-13T09:12:28.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0098999020000000004, 2024-09-13T09:12:28.697 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.697 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.697 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.697 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.697 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.697 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1990 2024-09-13T09:12:28.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.698 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504086+0000", 2024-09-13T09:12:28.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504086+0000", 2024-09-13T09:12:28.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504086+0000", 2024-09-13T09:12:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504090+0000", 2024-09-13T09:12:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000004e-06 2024-09-13T09:12:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504097+0000", 2024-09-13T09:12:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4630000000000004e-06 2024-09-13T09:12:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504103+0000", 2024-09-13T09:12:28.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4589999999999997e-06 2024-09-13T09:12:28.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.505791+0000", 2024-09-13T09:12:28.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0016881260000000001 2024-09-13T09:12:28.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.505955+0000", 2024-09-13T09:12:28.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016378200000000001 2024-09-13T09:12:28.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.506366+0000", 2024-09-13T09:12:28.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000411667 2024-09-13T09:12:28.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.513522+0000", 2024-09-13T09:12:28.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071554360000000003 2024-09-13T09:12:28.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.513662+0000", 2024-09-13T09:12:28.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000140195 2024-09-13T09:12:28.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.513949+0000", 2024-09-13T09:12:28.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000286908 2024-09-13T09:12:28.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.513986+0000", 2024-09-13T09:12:28.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7444999999999998e-05 2024-09-13T09:12:28.703 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.703 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.703 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.703 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1991 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.703 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.504108+0000", 2024-09-13T09:12:28.703 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.1714548009999999, 2024-09-13T09:12:28.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011285132, 2024-09-13T09:12:28.703 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.703 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.703 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.703 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.703 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1991 2024-09-13T09:12:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504108+0000", 2024-09-13T09:12:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504108+0000", 2024-09-13T09:12:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504108+0000", 2024-09-13T09:12:28.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504111+0000", 2024-09-13T09:12:28.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4070000000000002e-06 2024-09-13T09:12:28.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504117+0000", 2024-09-13T09:12:28.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:12:28.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504123+0000", 2024-09-13T09:12:28.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:12:28.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507071+0000", 2024-09-13T09:12:28.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029475579999999999 2024-09-13T09:12:28.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507151+0000", 2024-09-13T09:12:28.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0000000000000007e-05 2024-09-13T09:12:28.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507556+0000", 2024-09-13T09:12:28.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00040499900000000002 2024-09-13T09:12:28.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515305+0000", 2024-09-13T09:12:28.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0077488820000000003 2024-09-13T09:12:28.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515312+0000", 2024-09-13T09:12:28.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1640000000000004e-06 2024-09-13T09:12:28.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515358+0000", 2024-09-13T09:12:28.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.604e-05 2024-09-13T09:12:28.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515393+0000", 2024-09-13T09:12:28.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5670999999999998e-05 2024-09-13T09:12:28.708 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.709 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.709 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.709 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1992 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.709 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.504132+0000", 2024-09-13T09:12:28.709 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.1714308899999999, 2024-09-13T09:12:28.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011378241000000001, 2024-09-13T09:12:28.709 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.709 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.709 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.709 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.709 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.710 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1992 2024-09-13T09:12:28.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.710 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504132+0000", 2024-09-13T09:12:28.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504132+0000", 2024-09-13T09:12:28.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504132+0000", 2024-09-13T09:12:28.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504135+0000", 2024-09-13T09:12:28.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4580000000000002e-06 2024-09-13T09:12:28.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504143+0000", 2024-09-13T09:12:28.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0590000000000003e-06 2024-09-13T09:12:28.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504149+0000", 2024-09-13T09:12:28.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7649999999999999e-06 2024-09-13T09:12:28.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.508003+0000", 2024-09-13T09:12:28.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003854176 2024-09-13T09:12:28.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.508075+0000", 2024-09-13T09:12:28.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2197e-05 2024-09-13T09:12:28.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.508374+0000", 2024-09-13T09:12:28.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00029824099999999998 2024-09-13T09:12:28.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515402+0000", 2024-09-13T09:12:28.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0070279380000000001 2024-09-13T09:12:28.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515437+0000", 2024-09-13T09:12:28.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5274000000000001e-05 2024-09-13T09:12:28.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515475+0000", 2024-09-13T09:12:28.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8296000000000001e-05 2024-09-13T09:12:28.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515510+0000", 2024-09-13T09:12:28.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5261e-05 2024-09-13T09:12:28.714 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.715 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.715 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.715 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1993 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.715 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.504155+0000", 2024-09-13T09:12:28.715 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.1714078610000001, 2024-09-13T09:12:28.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01145854, 2024-09-13T09:12:28.715 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.715 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.715 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.715 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.715 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.716 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1993 2024-09-13T09:12:28.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.716 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504155+0000", 2024-09-13T09:12:28.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504155+0000", 2024-09-13T09:12:28.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504155+0000", 2024-09-13T09:12:28.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504158+0000", 2024-09-13T09:12:28.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3100000000000001e-06 2024-09-13T09:12:28.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504165+0000", 2024-09-13T09:12:28.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7519999999999999e-06 2024-09-13T09:12:28.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.718 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.718 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504170+0000", 2024-09-13T09:12:28.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5350000000000004e-06 2024-09-13T09:12:28.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.718 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.718 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.508797+0000", 2024-09-13T09:12:28.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046267319999999997 2024-09-13T09:12:28.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.508893+0000", 2024-09-13T09:12:28.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5356e-05 2024-09-13T09:12:28.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.509224+0000", 2024-09-13T09:12:28.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033109799999999999 2024-09-13T09:12:28.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515406+0000", 2024-09-13T09:12:28.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061827640000000003 2024-09-13T09:12:28.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515544+0000", 2024-09-13T09:12:28.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013761599999999999 2024-09-13T09:12:28.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515581+0000", 2024-09-13T09:12:28.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6902e-05 2024-09-13T09:12:28.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.721 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515614+0000", 2024-09-13T09:12:28.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2848000000000001e-05 2024-09-13T09:12:28.721 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.721 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.721 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.721 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2156 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.721 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.568613+0000", 2024-09-13T09:12:28.721 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.1069500950000002, 2024-09-13T09:12:28.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0087999440000000005, 2024-09-13T09:12:28.721 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.722 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.722 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.722 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.722 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.722 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2156 2024-09-13T09:12:28.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.722 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568613+0000", 2024-09-13T09:12:28.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568613+0000", 2024-09-13T09:12:28.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568612+0000", 2024-09-13T09:12:28.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568616+0000", 2024-09-13T09:12:28.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9380000000000002e-06 2024-09-13T09:12:28.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568625+0000", 2024-09-13T09:12:28.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5080000000000005e-06 2024-09-13T09:12:28.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568630+0000", 2024-09-13T09:12:28.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2920000000000003e-06 2024-09-13T09:12:28.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.571451+0000", 2024-09-13T09:12:28.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028206749999999999 2024-09-13T09:12:28.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.571530+0000", 2024-09-13T09:12:28.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9571000000000001e-05 2024-09-13T09:12:28.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.571881+0000", 2024-09-13T09:12:28.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035083500000000002 2024-09-13T09:12:28.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.575089+0000", 2024-09-13T09:12:28.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003207831 2024-09-13T09:12:28.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.577325+0000", 2024-09-13T09:12:28.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022359139999999999 2024-09-13T09:12:28.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.577374+0000", 2024-09-13T09:12:28.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8667e-05 2024-09-13T09:12:28.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.577413+0000", 2024-09-13T09:12:28.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9211000000000001e-05 2024-09-13T09:12:28.727 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.727 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.727 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.727 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2279 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.727 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.614790+0000", 2024-09-13T09:12:28.727 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.0607727269999998, 2024-09-13T09:12:28.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0081187259999999997, 2024-09-13T09:12:28.727 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.727 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.727 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.728 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.728 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.728 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2279 2024-09-13T09:12:28.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.728 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614790+0000", 2024-09-13T09:12:28.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614790+0000", 2024-09-13T09:12:28.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614790+0000", 2024-09-13T09:12:28.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614794+0000", 2024-09-13T09:12:28.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9929999999999997e-06 2024-09-13T09:12:28.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614816+0000", 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2218e-05 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614823+0000", 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.793e-06 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616601+0000", 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0017785380000000001 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616675+0000", 2024-09-13T09:12:28.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4093999999999999e-05 2024-09-13T09:12:28.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617285+0000", 2024-09-13T09:12:28.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00060970299999999996 2024-09-13T09:12:28.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.621036+0000", 2024-09-13T09:12:28.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003751225 2024-09-13T09:12:28.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.732 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.732 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.622815+0000", 2024-09-13T09:12:28.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001779047 2024-09-13T09:12:28.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.732 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.732 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.622871+0000", 2024-09-13T09:12:28.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5742000000000003e-05 2024-09-13T09:12:28.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.732 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.732 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.622909+0000", 2024-09-13T09:12:28.733 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7917999999999998e-05 2024-09-13T09:12:28.733 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.733 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.733 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.733 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2280 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.733 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.614829+0000", 2024-09-13T09:12:28.733 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.0607343290000002, 2024-09-13T09:12:28.733 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010493406, 2024-09-13T09:12:28.733 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2280 2024-09-13T09:12:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614829+0000", 2024-09-13T09:12:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614829+0000", 2024-09-13T09:12:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614828+0000", 2024-09-13T09:12:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614848+0000", 2024-09-13T09:12:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9653e-05 2024-09-13T09:12:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614880+0000", 2024-09-13T09:12:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2008999999999998e-05 2024-09-13T09:12:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614887+0000", 2024-09-13T09:12:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7750000000000004e-06 2024-09-13T09:12:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617693+0000", 2024-09-13T09:12:28.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028061119999999999 2024-09-13T09:12:28.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.737 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617770+0000", 2024-09-13T09:12:28.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7286999999999994e-05 2024-09-13T09:12:28.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.737 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618140+0000", 2024-09-13T09:12:28.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000370355 2024-09-13T09:12:28.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.622954+0000", 2024-09-13T09:12:28.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048138829999999997 2024-09-13T09:12:28.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625235+0000", 2024-09-13T09:12:28.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022808160000000002 2024-09-13T09:12:28.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625289+0000", 2024-09-13T09:12:28.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3604999999999999e-05 2024-09-13T09:12:28.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.739 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.739 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625322+0000", 2024-09-13T09:12:28.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3491999999999997e-05 2024-09-13T09:12:28.739 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.739 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.739 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.739 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2281 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.739 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.614893+0000", 2024-09-13T09:12:28.739 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.0606703230000001, 2024-09-13T09:12:28.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010532901000000001, 2024-09-13T09:12:28.740 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.740 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.740 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.740 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.740 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.740 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2281 2024-09-13T09:12:28.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.740 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.740 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614893+0000", 2024-09-13T09:12:28.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614893+0000", 2024-09-13T09:12:28.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614892+0000", 2024-09-13T09:12:28.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614900+0000", 2024-09-13T09:12:28.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2119999999999997e-06 2024-09-13T09:12:28.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614924+0000", 2024-09-13T09:12:28.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4260000000000002e-05 2024-09-13T09:12:28.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614934+0000", 2024-09-13T09:12:28.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7659999999999993e-06 2024-09-13T09:12:28.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619174+0000", 2024-09-13T09:12:28.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042408890000000003 2024-09-13T09:12:28.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619255+0000", 2024-09-13T09:12:28.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0189000000000001e-05 2024-09-13T09:12:28.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.744 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619609+0000", 2024-09-13T09:12:28.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035430500000000002 2024-09-13T09:12:28.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.744 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.744 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.622991+0000", 2024-09-13T09:12:28.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033823220000000001 2024-09-13T09:12:28.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.744 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.744 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625333+0000", 2024-09-13T09:12:28.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023419309999999998 2024-09-13T09:12:28.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625385+0000", 2024-09-13T09:12:28.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2194999999999997e-05 2024-09-13T09:12:28.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625426+0000", 2024-09-13T09:12:28.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0278999999999998e-05 2024-09-13T09:12:28.745 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.745 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.745 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.746 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2282 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.746 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.614942+0000", 2024-09-13T09:12:28.746 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.0606212020000001, 2024-09-13T09:12:28.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010565567999999999, 2024-09-13T09:12:28.746 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.746 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.746 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.746 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.746 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.746 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2282 2024-09-13T09:12:28.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.746 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614942+0000", 2024-09-13T09:12:28.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614942+0000", 2024-09-13T09:12:28.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614941+0000", 2024-09-13T09:12:28.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.748 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.748 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614947+0000", 2024-09-13T09:12:28.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3789999999999997e-06 2024-09-13T09:12:28.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.748 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.748 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614956+0000", 2024-09-13T09:12:28.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2969999999999992e-06 2024-09-13T09:12:28.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.748 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.749 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614961+0000", 2024-09-13T09:12:28.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:12:28.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.749 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.749 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620209+0000", 2024-09-13T09:12:28.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052470429999999998 2024-09-13T09:12:28.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.749 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.749 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620286+0000", 2024-09-13T09:12:28.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7061e-05 2024-09-13T09:12:28.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620646+0000", 2024-09-13T09:12:28.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000360164 2024-09-13T09:12:28.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.623024+0000", 2024-09-13T09:12:28.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002378186 2024-09-13T09:12:28.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625434+0000", 2024-09-13T09:12:28.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0024102630000000002 2024-09-13T09:12:28.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625475+0000", 2024-09-13T09:12:28.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0867e-05 2024-09-13T09:12:28.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625507+0000", 2024-09-13T09:12:28.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2394000000000002e-05 2024-09-13T09:12:28.751 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.751 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.751 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.752 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2283 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.752 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.614977+0000", 2024-09-13T09:12:28.752 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.0605864839999999, 2024-09-13T09:12:28.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011508387, 2024-09-13T09:12:28.752 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.752 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.752 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.752 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.752 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.752 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2283 2024-09-13T09:12:28.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.752 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.753 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614977+0000", 2024-09-13T09:12:28.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614977+0000", 2024-09-13T09:12:28.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614976+0000", 2024-09-13T09:12:28.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614980+0000", 2024-09-13T09:12:28.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8659999999999999e-06 2024-09-13T09:12:28.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614997+0000", 2024-09-13T09:12:28.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6946e-05 2024-09-13T09:12:28.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615003+0000", 2024-09-13T09:12:28.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4829999999999996e-06 2024-09-13T09:12:28.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.621063+0000", 2024-09-13T09:12:28.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0060601860000000004 2024-09-13T09:12:28.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.621415+0000", 2024-09-13T09:12:28.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035131399999999998 2024-09-13T09:12:28.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.621769+0000", 2024-09-13T09:12:28.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035409500000000002 2024-09-13T09:12:28.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.760 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.760 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625219+0000", 2024-09-13T09:12:28.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034497970000000001 2024-09-13T09:12:28.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.760 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.760 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626400+0000", 2024-09-13T09:12:28.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011811910000000001 2024-09-13T09:12:28.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.760 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.760 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626448+0000", 2024-09-13T09:12:28.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8615000000000001e-05 2024-09-13T09:12:28.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.761 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.761 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.761 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626485+0000", 2024-09-13T09:12:28.761 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6491e-05 2024-09-13T09:12:28.761 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.761 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.761 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.761 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.761 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2286 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.761 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.615037+0000", 2024-09-13T09:12:28.761 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.0605264139999999, 2024-09-13T09:12:28.761 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013941546000000001, 2024-09-13T09:12:28.762 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.762 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.762 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.762 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.762 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.762 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2286 2024-09-13T09:12:28.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.762 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.762 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.762 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615037+0000", 2024-09-13T09:12:28.762 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.763 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615037+0000", 2024-09-13T09:12:28.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.763 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615036+0000", 2024-09-13T09:12:28.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.763 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615039+0000", 2024-09-13T09:12:28.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1829999999999998e-06 2024-09-13T09:12:28.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.764 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.764 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615047+0000", 2024-09-13T09:12:28.764 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:12:28.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.764 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.764 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615052+0000", 2024-09-13T09:12:28.764 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2780000000000001e-06 2024-09-13T09:12:28.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.764 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.765 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.623049+0000", 2024-09-13T09:12:28.765 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0079966499999999992 2024-09-13T09:12:28.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.765 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.765 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.623128+0000", 2024-09-13T09:12:28.765 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9531999999999997e-05 2024-09-13T09:12:28.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.765 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.765 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.623468+0000", 2024-09-13T09:12:28.765 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033982700000000001 2024-09-13T09:12:28.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.766 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.766 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625551+0000", 2024-09-13T09:12:28.766 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020836140000000001 2024-09-13T09:12:28.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.766 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.766 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628931+0000", 2024-09-13T09:12:28.766 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033793709999999999 2024-09-13T09:12:28.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.766 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.766 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628958+0000", 2024-09-13T09:12:28.766 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6928e-05 2024-09-13T09:12:28.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.767 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628978+0000", 2024-09-13T09:12:28.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0426999999999999e-05 2024-09-13T09:12:28.767 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.767 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.767 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.767 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2293 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.767 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.616955+0000", 2024-09-13T09:12:28.767 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.0586078209999998, 2024-09-13T09:12:28.768 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014655674, 2024-09-13T09:12:28.768 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.768 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.768 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.768 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.768 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.768 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2293 2024-09-13T09:12:28.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.769 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.769 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.769 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616955+0000", 2024-09-13T09:12:28.769 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.769 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.769 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616955+0000", 2024-09-13T09:12:28.769 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.769 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.770 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616954+0000", 2024-09-13T09:12:28.770 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:28.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.770 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.770 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616960+0000", 2024-09-13T09:12:28.770 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1809999999999998e-06 2024-09-13T09:12:28.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.770 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.770 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616973+0000", 2024-09-13T09:12:28.770 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3001e-05 2024-09-13T09:12:28.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616981+0000", 2024-09-13T09:12:28.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5089999999999996e-06 2024-09-13T09:12:28.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625596+0000", 2024-09-13T09:12:28.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086152260000000001 2024-09-13T09:12:28.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625680+0000", 2024-09-13T09:12:28.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4098999999999996e-05 2024-09-13T09:12:28.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.772 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.772 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626054+0000", 2024-09-13T09:12:28.772 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00037370300000000002 2024-09-13T09:12:28.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.772 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.772 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.629114+0000", 2024-09-13T09:12:28.772 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0030600279999999998 2024-09-13T09:12:28.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.772 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.772 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631561+0000", 2024-09-13T09:12:28.773 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0024473009999999998 2024-09-13T09:12:28.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.773 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.773 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.773 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631590+0000", 2024-09-13T09:12:28.773 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9033000000000001e-05 2024-09-13T09:12:28.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.773 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.773 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.773 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631611+0000", 2024-09-13T09:12:28.773 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0665999999999998e-05 2024-09-13T09:12:28.773 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.774 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.774 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.774 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2297 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.774 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.617105+0000", 2024-09-13T09:12:28.774 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.0584584850000001, 2024-09-13T09:12:28.774 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01465442, 2024-09-13T09:12:28.774 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.774 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.774 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.774 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.774 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.774 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2297 2024-09-13T09:12:28.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.775 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.775 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.775 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617105+0000", 2024-09-13T09:12:28.775 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.775 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.775 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617105+0000", 2024-09-13T09:12:28.775 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.776 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.776 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617104+0000", 2024-09-13T09:12:28.776 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:28.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.776 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.776 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617109+0000", 2024-09-13T09:12:28.776 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7579999999999998e-06 2024-09-13T09:12:28.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.776 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.776 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617121+0000", 2024-09-13T09:12:28.776 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1439e-05 2024-09-13T09:12:28.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.777 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.777 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617137+0000", 2024-09-13T09:12:28.777 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6481999999999999e-05 2024-09-13T09:12:28.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.777 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.777 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626526+0000", 2024-09-13T09:12:28.777 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0093890499999999995 2024-09-13T09:12:28.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.777 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.778 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626617+0000", 2024-09-13T09:12:28.778 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0116000000000003e-05 2024-09-13T09:12:28.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.778 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.778 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626962+0000", 2024-09-13T09:12:28.778 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034533300000000002 2024-09-13T09:12:28.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.778 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.778 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.629177+0000", 2024-09-13T09:12:28.778 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022147529999999999 2024-09-13T09:12:28.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631706+0000", 2024-09-13T09:12:28.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025289829999999998 2024-09-13T09:12:28.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631735+0000", 2024-09-13T09:12:28.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9309e-05 2024-09-13T09:12:28.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631759+0000", 2024-09-13T09:12:28.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4066e-05 2024-09-13T09:12:28.779 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.780 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.780 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.780 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2298 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.780 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.617145+0000", 2024-09-13T09:12:28.780 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.0584181240000001, 2024-09-13T09:12:28.780 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014745108, 2024-09-13T09:12:28.780 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.780 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.780 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.780 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.780 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.781 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2298 2024-09-13T09:12:28.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.781 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.781 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.781 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617145+0000", 2024-09-13T09:12:28.781 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.781 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.781 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617145+0000", 2024-09-13T09:12:28.781 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.782 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.782 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617144+0000", 2024-09-13T09:12:28.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.782 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.782 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617148+0000", 2024-09-13T09:12:28.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6830000000000001e-06 2024-09-13T09:12:28.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.782 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.782 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617156+0000", 2024-09-13T09:12:28.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1610000000000003e-06 2024-09-13T09:12:28.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617162+0000", 2024-09-13T09:12:28.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1600000000000003e-06 2024-09-13T09:12:28.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.627419+0000", 2024-09-13T09:12:28.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010256517 2024-09-13T09:12:28.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.627516+0000", 2024-09-13T09:12:28.784 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7600000000000001e-05 2024-09-13T09:12:28.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.784 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.784 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.627872+0000", 2024-09-13T09:12:28.784 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035532400000000002 2024-09-13T09:12:28.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.784 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.784 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631665+0000", 2024-09-13T09:12:28.784 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037928200000000001 2024-09-13T09:12:28.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.784 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.785 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631771+0000", 2024-09-13T09:12:28.785 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000106288 2024-09-13T09:12:28.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.785 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.785 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631807+0000", 2024-09-13T09:12:28.785 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5695999999999998e-05 2024-09-13T09:12:28.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.785 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.785 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631890+0000", 2024-09-13T09:12:28.785 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3473000000000005e-05 2024-09-13T09:12:28.785 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.785 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.786 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.786 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2308 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.786 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.618672+0000", 2024-09-13T09:12:28.786 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.0568908279999998, 2024-09-13T09:12:28.786 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013301396, 2024-09-13T09:12:28.786 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.786 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.786 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.786 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.786 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.786 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2308 2024-09-13T09:12:28.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.787 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.787 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.787 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618672+0000", 2024-09-13T09:12:28.787 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.787 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.787 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618672+0000", 2024-09-13T09:12:28.787 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.787 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.787 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618672+0000", 2024-09-13T09:12:28.788 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.788 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.788 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618681+0000", 2024-09-13T09:12:28.788 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3819999999999993e-06 2024-09-13T09:12:28.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.788 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.788 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618690+0000", 2024-09-13T09:12:28.788 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2420000000000006e-06 2024-09-13T09:12:28.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.789 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.789 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618696+0000", 2024-09-13T09:12:28.789 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2859999999999996e-06 2024-09-13T09:12:28.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.789 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.789 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628271+0000", 2024-09-13T09:12:28.789 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.009574249 2024-09-13T09:12:28.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.789 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.789 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628314+0000", 2024-09-13T09:12:28.789 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3467999999999998e-05 2024-09-13T09:12:28.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.790 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.790 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628672+0000", 2024-09-13T09:12:28.790 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000357726 2024-09-13T09:12:28.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.790 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.790 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631699+0000", 2024-09-13T09:12:28.790 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003026759 2024-09-13T09:12:28.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.790 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.790 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631902+0000", 2024-09-13T09:12:28.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000202862 2024-09-13T09:12:28.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.791 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631943+0000", 2024-09-13T09:12:28.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1084000000000002e-05 2024-09-13T09:12:28.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.791 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631974+0000", 2024-09-13T09:12:28.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1007000000000003e-05 2024-09-13T09:12:28.791 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.791 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.791 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.792 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2309 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.792 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.619333+0000", 2024-09-13T09:12:28.792 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.0562301650000001, 2024-09-13T09:12:28.792 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012775455999999999, 2024-09-13T09:12:28.792 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.792 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.792 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.792 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.792 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.792 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2309 2024-09-13T09:12:28.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.793 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.793 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.793 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619333+0000", 2024-09-13T09:12:28.793 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.793 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.793 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619333+0000", 2024-09-13T09:12:28.793 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.793 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.793 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619332+0000", 2024-09-13T09:12:28.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:28.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.794 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.794 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619338+0000", 2024-09-13T09:12:28.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.648e-06 2024-09-13T09:12:28.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.794 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.794 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619349+0000", 2024-09-13T09:12:28.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1545e-05 2024-09-13T09:12:28.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.794 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619367+0000", 2024-09-13T09:12:28.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7419e-05 2024-09-13T09:12:28.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.629223+0000", 2024-09-13T09:12:28.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0098561680000000002 2024-09-13T09:12:28.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.629311+0000", 2024-09-13T09:12:28.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8057000000000001e-05 2024-09-13T09:12:28.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.796 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.796 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.630009+0000", 2024-09-13T09:12:28.796 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00069777100000000002 2024-09-13T09:12:28.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.796 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.796 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631985+0000", 2024-09-13T09:12:28.796 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019758919999999999 2024-09-13T09:12:28.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.796 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.796 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.632022+0000", 2024-09-13T09:12:28.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7410000000000003e-05 2024-09-13T09:12:28.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.797 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.632075+0000", 2024-09-13T09:12:28.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3248000000000002e-05 2024-09-13T09:12:28.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.797 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.632108+0000", 2024-09-13T09:12:28.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3191000000000001e-05 2024-09-13T09:12:28.797 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.797 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.797 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.798 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2310 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:28.798 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.619384+0000", 2024-09-13T09:12:28.798 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.0561791440000001, 2024-09-13T09:12:28.798 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014600315000000001, 2024-09-13T09:12:28.798 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:28.798 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:28.798 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:28.798 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:28.798 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:28.798 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2310 2024-09-13T09:12:28.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.799 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:28.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.799 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:28.799 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619384+0000", 2024-09-13T09:12:28.799 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.799 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:28.799 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619384+0000", 2024-09-13T09:12:28.799 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:28.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.799 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:28.799 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619383+0000", 2024-09-13T09:12:28.800 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:28.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.800 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:28.800 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619387+0000", 2024-09-13T09:12:28.800 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1200000000000004e-06 2024-09-13T09:12:28.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.800 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:28.800 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619398+0000", 2024-09-13T09:12:28.800 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0841e-05 2024-09-13T09:12:28.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.800 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:28.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619404+0000", 2024-09-13T09:12:28.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8329999999999999e-06 2024-09-13T09:12:28.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.801 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:28.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.630863+0000", 2024-09-13T09:12:28.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011459278 2024-09-13T09:12:28.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.801 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:28.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.630971+0000", 2024-09-13T09:12:28.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010728100000000001 2024-09-13T09:12:28.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.802 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:28.802 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631296+0000", 2024-09-13T09:12:28.802 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00032547400000000002 2024-09-13T09:12:28.802 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.802 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:28.802 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.632133+0000", 2024-09-13T09:12:28.802 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00083720599999999995 2024-09-13T09:12:28.802 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.802 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:28.802 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.633830+0000", 2024-09-13T09:12:28.802 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001696342 2024-09-13T09:12:28.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:28.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.633950+0000", 2024-09-13T09:12:28.803 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120444 2024-09-13T09:12:28.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:28.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:28.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:28.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.633984+0000", 2024-09-13T09:12:28.803 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4082000000000003e-05 2024-09-13T09:12:28.803 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.803 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.803 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.803 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:28.803 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:28.804 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:28.806 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:28.806 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:28.803+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:28.807 INFO:teuthology.orchestra.run.smithi157.stdout:980 read 0x29600 thru 0x30fe0 (0x79e1 bytes) 2024-09-13T09:12:28.807 INFO:teuthology.orchestra.run.smithi157.stdout:986 read 0x24000 thru 0x2be17 (0x7e18 bytes) 2024-09-13T09:12:28.807 INFO:teuthology.orchestra.run.smithi157.stdout:988 read 0x181a00 thru 0x182dca (0x13cb bytes) 2024-09-13T09:12:28.807 INFO:teuthology.orchestra.run.smithi157.stdout:991 read 0x158000 thru 0x167309 (0xf30a bytes) 2024-09-13T09:12:28.807 INFO:teuthology.orchestra.run.smithi157.stdout:992 read 0x5a800 thru 0x60373 (0x5b74 bytes) 2024-09-13T09:12:28.807 INFO:teuthology.orchestra.run.smithi157.stdout:993 read 0x175000 thru 0x183463 (0xe464 bytes) 2024-09-13T09:12:28.807 INFO:teuthology.orchestra.run.smithi157.stdout:994 punch from 0x8e00 to 0xfe00, (0x7000 bytes) 2024-09-13T09:12:28.807 INFO:teuthology.orchestra.run.smithi157.stdout:995 punch from 0x5aa00 to 0x66000, (0xb600 bytes) 2024-09-13T09:12:28.807 INFO:teuthology.orchestra.run.smithi157.stdout:998 read 0x49e00 thru 0x50d07 (0x6f08 bytes) 2024-09-13T09:12:28.808 INFO:teuthology.orchestra.run.smithi157.stdout:1003 write 0x8a0fc00 thru 0x8a1db62 (0xdf63 bytes) 2024-09-13T09:12:28.808 INFO:teuthology.orchestra.run.smithi157.stdout:1005 read 0x4bbf400 thru 0x4bc37ec (0x43ed bytes) 2024-09-13T09:12:28.808 INFO:teuthology.orchestra.run.smithi157.stdout:1006 clone 7 order 21 su 262144 sc 13 2024-09-13T09:12:28.906 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:28.905+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:28.952 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:28.952 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:28.953 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:28.953 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:28.961 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:29.006 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:29.005+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:29.196 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:29.196 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:29.196 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:29.196 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:29.193+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:29.295 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:29.294+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:946 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.304801+0000", 2024-09-13T09:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0374466949999999, 2024-09-13T09:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010858095, 2024-09-13T09:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 946 2024-09-13T09:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304801+0000", 2024-09-13T09:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304801+0000", 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304800+0000", 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304803+0000", 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304817+0000", 2024-09-13T09:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3736e-05 2024-09-13T09:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304823+0000", 2024-09-13T09:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1680000000000001e-06 2024-09-13T09:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315506+0000", 2024-09-13T09:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010682764000000001 2024-09-13T09:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315580+0000", 2024-09-13T09:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4306000000000003e-05 2024-09-13T09:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315659+0000", 2024-09-13T09:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8807999999999995e-05 2024-09-13T09:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:947 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.304829+0000", 2024-09-13T09:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.037418733, 2024-09-13T09:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011020509, 2024-09-13T09:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 947 2024-09-13T09:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304829+0000", 2024-09-13T09:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304829+0000", 2024-09-13T09:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304828+0000", 2024-09-13T09:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304831+0000", 2024-09-13T09:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7819999999999998e-06 2024-09-13T09:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304838+0000", 2024-09-13T09:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9909999999999996e-06 2024-09-13T09:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304843+0000", 2024-09-13T09:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2140000000000004e-06 2024-09-13T09:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315686+0000", 2024-09-13T09:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010842762000000001 2024-09-13T09:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315763+0000", 2024-09-13T09:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6429999999999998e-05 2024-09-13T09:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315849+0000", 2024-09-13T09:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6891999999999994e-05 2024-09-13T09:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1053 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 13369344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.309361+0000", 2024-09-13T09:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0328867119999998, 2024-09-13T09:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011710174, 2024-09-13T09:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1053 2024-09-13T09:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309361+0000", 2024-09-13T09:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309361+0000", 2024-09-13T09:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309361+0000", 2024-09-13T09:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309363+0000", 2024-09-13T09:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1900000000000002e-06 2024-09-13T09:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309368+0000", 2024-09-13T09:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6740000000000002e-06 2024-09-13T09:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309373+0000", 2024-09-13T09:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5730000000000001e-06 2024-09-13T09:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320071+0000", 2024-09-13T09:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010697628000000001 2024-09-13T09:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320149+0000", 2024-09-13T09:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8536999999999997e-05 2024-09-13T09:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.321071+0000", 2024-09-13T09:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00092195099999999996 2024-09-13T09:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1054 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.309377+0000", 2024-09-13T09:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0328705520000003, 2024-09-13T09:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011868086, 2024-09-13T09:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1054 2024-09-13T09:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309377+0000", 2024-09-13T09:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309377+0000", 2024-09-13T09:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309377+0000", 2024-09-13T09:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309379+0000", 2024-09-13T09:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309385+0000", 2024-09-13T09:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309391+0000", 2024-09-13T09:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7440000000000004e-06 2024-09-13T09:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.321098+0000", 2024-09-13T09:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011707255999999999 2024-09-13T09:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.321166+0000", 2024-09-13T09:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8436999999999995e-05 2024-09-13T09:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.321245+0000", 2024-09-13T09:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8988999999999999e-05 2024-09-13T09:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1055 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.309395+0000", 2024-09-13T09:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0328524740000002, 2024-09-13T09:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012096084, 2024-09-13T09:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1055 2024-09-13T09:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309395+0000", 2024-09-13T09:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309395+0000", 2024-09-13T09:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309395+0000", 2024-09-13T09:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309397+0000", 2024-09-13T09:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.503e-06 2024-09-13T09:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309405+0000", 2024-09-13T09:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4379999999999998e-06 2024-09-13T09:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309410+0000", 2024-09-13T09:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0660000000000003e-06 2024-09-13T09:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.321298+0000", 2024-09-13T09:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011888156 2024-09-13T09:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.321382+0000", 2024-09-13T09:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4444000000000007e-05 2024-09-13T09:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.367 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.321491+0000", 2024-09-13T09:12:29.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000108904 2024-09-13T09:12:29.367 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.367 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.367 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.367 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1157 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.367 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.314790+0000", 2024-09-13T09:12:29.367 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0274578700000001, 2024-09-13T09:12:29.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010738912, 2024-09-13T09:12:29.367 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.367 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.368 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.368 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.368 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.368 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1157 2024-09-13T09:12:29.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.368 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.368 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.368 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314790+0000", 2024-09-13T09:12:29.368 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.368 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.368 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314790+0000", 2024-09-13T09:12:29.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314789+0000", 2024-09-13T09:12:29.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314792+0000", 2024-09-13T09:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7190000000000001e-06 2024-09-13T09:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314798+0000", 2024-09-13T09:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2709999999999998e-06 2024-09-13T09:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314804+0000", 2024-09-13T09:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7819999999999999e-06 2024-09-13T09:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325369+0000", 2024-09-13T09:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010565194 2024-09-13T09:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325437+0000", 2024-09-13T09:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7872000000000003e-05 2024-09-13T09:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325529+0000", 2024-09-13T09:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1476000000000003e-05 2024-09-13T09:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1158 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.314815+0000", 2024-09-13T09:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0274324440000004, 2024-09-13T09:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010903037000000001, 2024-09-13T09:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1158 2024-09-13T09:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314815+0000", 2024-09-13T09:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314815+0000", 2024-09-13T09:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314815+0000", 2024-09-13T09:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314817+0000", 2024-09-13T09:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.773e-06 2024-09-13T09:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314824+0000", 2024-09-13T09:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.516e-06 2024-09-13T09:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314831+0000", 2024-09-13T09:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2130000000000002e-06 2024-09-13T09:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325558+0000", 2024-09-13T09:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010727093 2024-09-13T09:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325624+0000", 2024-09-13T09:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6088999999999997e-05 2024-09-13T09:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325718+0000", 2024-09-13T09:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3861000000000001e-05 2024-09-13T09:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1159 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.314836+0000", 2024-09-13T09:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0274113309999997, 2024-09-13T09:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011062855999999999, 2024-09-13T09:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1159 2024-09-13T09:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314836+0000", 2024-09-13T09:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314836+0000", 2024-09-13T09:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314836+0000", 2024-09-13T09:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314838+0000", 2024-09-13T09:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314845+0000", 2024-09-13T09:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3589999999999996e-06 2024-09-13T09:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314849+0000", 2024-09-13T09:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7790000000000003e-06 2024-09-13T09:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325747+0000", 2024-09-13T09:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010897558999999999 2024-09-13T09:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325832+0000", 2024-09-13T09:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5143999999999997e-05 2024-09-13T09:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325899+0000", 2024-09-13T09:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7053999999999999e-05 2024-09-13T09:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1160 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.314854+0000", 2024-09-13T09:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0273938239999998, 2024-09-13T09:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011253521000000001, 2024-09-13T09:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1160 2024-09-13T09:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314854+0000", 2024-09-13T09:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314854+0000", 2024-09-13T09:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314853+0000", 2024-09-13T09:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314856+0000", 2024-09-13T09:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6220000000000001e-06 2024-09-13T09:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314862+0000", 2024-09-13T09:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1179999999999997e-06 2024-09-13T09:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314867+0000", 2024-09-13T09:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.848e-06 2024-09-13T09:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325928+0000", 2024-09-13T09:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011060558 2024-09-13T09:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326014+0000", 2024-09-13T09:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6793999999999996e-05 2024-09-13T09:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326107+0000", 2024-09-13T09:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2998999999999995e-05 2024-09-13T09:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1161 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 14155776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.314872+0000", 2024-09-13T09:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.027375352, 2024-09-13T09:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011426746999999999, 2024-09-13T09:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1161 2024-09-13T09:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314872+0000", 2024-09-13T09:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314872+0000", 2024-09-13T09:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314872+0000", 2024-09-13T09:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314874+0000", 2024-09-13T09:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3159999999999999e-06 2024-09-13T09:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314881+0000", 2024-09-13T09:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2700000000000001e-06 2024-09-13T09:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314885+0000", 2024-09-13T09:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326137+0000", 2024-09-13T09:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011251585 2024-09-13T09:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326210+0000", 2024-09-13T09:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3453000000000003e-05 2024-09-13T09:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326299+0000", 2024-09-13T09:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8651000000000002e-05 2024-09-13T09:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1162 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.315036+0000", 2024-09-13T09:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0272112250000003, 2024-09-13T09:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011465312, 2024-09-13T09:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1162 2024-09-13T09:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315036+0000", 2024-09-13T09:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315036+0000", 2024-09-13T09:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315036+0000", 2024-09-13T09:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315040+0000", 2024-09-13T09:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0330000000000002e-06 2024-09-13T09:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315050+0000", 2024-09-13T09:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9310000000000003e-06 2024-09-13T09:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315057+0000", 2024-09-13T09:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.934e-06 2024-09-13T09:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326333+0000", 2024-09-13T09:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01127591 2024-09-13T09:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326415+0000", 2024-09-13T09:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2649000000000001e-05 2024-09-13T09:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326502+0000", 2024-09-13T09:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6524e-05 2024-09-13T09:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1261 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 4456448~47616] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320345+0000", 2024-09-13T09:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0219029529999997, 2024-09-13T09:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011029959000000001, 2024-09-13T09:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1261 2024-09-13T09:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320345+0000", 2024-09-13T09:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320345+0000", 2024-09-13T09:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320344+0000", 2024-09-13T09:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320347+0000", 2024-09-13T09:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3259999999999998e-06 2024-09-13T09:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320354+0000", 2024-09-13T09:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6210000000000004e-06 2024-09-13T09:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320359+0000", 2024-09-13T09:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9320000000000004e-06 2024-09-13T09:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.329121+0000", 2024-09-13T09:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0087622510000000004 2024-09-13T09:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.329171+0000", 2024-09-13T09:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9815999999999999e-05 2024-09-13T09:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331375+0000", 2024-09-13T09:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002203408 2024-09-13T09:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1262 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 6160384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320364+0000", 2024-09-13T09:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0218838039999998, 2024-09-13T09:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011167203000000001, 2024-09-13T09:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1262 2024-09-13T09:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320364+0000", 2024-09-13T09:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320364+0000", 2024-09-13T09:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320363+0000", 2024-09-13T09:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320366+0000", 2024-09-13T09:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320372+0000", 2024-09-13T09:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0719999999999996e-06 2024-09-13T09:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320377+0000", 2024-09-13T09:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331409+0000", 2024-09-13T09:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011031891 2024-09-13T09:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331459+0000", 2024-09-13T09:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9607999999999997e-05 2024-09-13T09:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331531+0000", 2024-09-13T09:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2192000000000003e-05 2024-09-13T09:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1263 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320382+0000", 2024-09-13T09:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0218653309999999, 2024-09-13T09:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011363743000000001, 2024-09-13T09:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1263 2024-09-13T09:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320382+0000", 2024-09-13T09:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320382+0000", 2024-09-13T09:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320382+0000", 2024-09-13T09:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320384+0000", 2024-09-13T09:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.362e-06 2024-09-13T09:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320392+0000", 2024-09-13T09:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.481e-06 2024-09-13T09:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320397+0000", 2024-09-13T09:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0669999999999999e-06 2024-09-13T09:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331566+0000", 2024-09-13T09:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011168757 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331654+0000", 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8577000000000003e-05 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331746+0000", 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1831999999999998e-05 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1264 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 8519680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320401+0000", 2024-09-13T09:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0218462779999999, 2024-09-13T09:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011595169000000001, 2024-09-13T09:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1264 2024-09-13T09:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320401+0000", 2024-09-13T09:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320401+0000", 2024-09-13T09:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320401+0000", 2024-09-13T09:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320404+0000", 2024-09-13T09:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6299999999999998e-06 2024-09-13T09:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320410+0000", 2024-09-13T09:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9519999999999999e-06 2024-09-13T09:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320414+0000", 2024-09-13T09:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331782+0000", 2024-09-13T09:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011367962000000001 2024-09-13T09:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331878+0000", 2024-09-13T09:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5261999999999997e-05 2024-09-13T09:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331997+0000", 2024-09-13T09:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011889 2024-09-13T09:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1265 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320419+0000", 2024-09-13T09:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0218286829999998, 2024-09-13T09:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011788669, 2024-09-13T09:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1265 2024-09-13T09:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320419+0000", 2024-09-13T09:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320419+0000", 2024-09-13T09:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320419+0000", 2024-09-13T09:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320421+0000", 2024-09-13T09:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.024e-06 2024-09-13T09:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320428+0000", 2024-09-13T09:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2150000000000004e-06 2024-09-13T09:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320432+0000", 2024-09-13T09:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5719999999999996e-06 2024-09-13T09:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332032+0000", 2024-09-13T09:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011599412 2024-09-13T09:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332110+0000", 2024-09-13T09:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8318000000000005e-05 2024-09-13T09:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332208+0000", 2024-09-13T09:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7449999999999997e-05 2024-09-13T09:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1266 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320437+0000", 2024-09-13T09:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0218101959999997, 2024-09-13T09:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012024399, 2024-09-13T09:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1266 2024-09-13T09:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320437+0000", 2024-09-13T09:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320437+0000", 2024-09-13T09:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320437+0000", 2024-09-13T09:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320439+0000", 2024-09-13T09:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5409999999999999e-06 2024-09-13T09:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320447+0000", 2024-09-13T09:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7379999999999994e-06 2024-09-13T09:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320452+0000", 2024-09-13T09:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6909999999999996e-06 2024-09-13T09:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332249+0000", 2024-09-13T09:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011797626 2024-09-13T09:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332339+0000", 2024-09-13T09:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9959999999999999e-05 2024-09-13T09:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332462+0000", 2024-09-13T09:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012244999999999999 2024-09-13T09:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1267 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320456+0000", 2024-09-13T09:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0217915099999999, 2024-09-13T09:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012214348, 2024-09-13T09:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1267 2024-09-13T09:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320456+0000", 2024-09-13T09:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320456+0000", 2024-09-13T09:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320456+0000", 2024-09-13T09:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320458+0000", 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320464+0000", 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1079999999999998e-06 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320469+0000", 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7600000000000002e-06 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332515+0000", 2024-09-13T09:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012045907 2024-09-13T09:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332601+0000", 2024-09-13T09:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5568000000000005e-05 2024-09-13T09:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332670+0000", 2024-09-13T09:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9958999999999999e-05 2024-09-13T09:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1268 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320474+0000", 2024-09-13T09:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.021774035, 2024-09-13T09:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012370015, 2024-09-13T09:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1268 2024-09-13T09:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320474+0000", 2024-09-13T09:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320474+0000", 2024-09-13T09:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320473+0000", 2024-09-13T09:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320476+0000", 2024-09-13T09:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320483+0000", 2024-09-13T09:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0770000000000002e-06 2024-09-13T09:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320488+0000", 2024-09-13T09:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2240000000000003e-06 2024-09-13T09:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332713+0000", 2024-09-13T09:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012225592 2024-09-13T09:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332766+0000", 2024-09-13T09:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2203000000000001e-05 2024-09-13T09:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332844+0000", 2024-09-13T09:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8016000000000007e-05 2024-09-13T09:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1269 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 15990784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320492+0000", 2024-09-13T09:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0217553959999996, 2024-09-13T09:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012500928, 2024-09-13T09:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1269 2024-09-13T09:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320492+0000", 2024-09-13T09:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320492+0000", 2024-09-13T09:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320492+0000", 2024-09-13T09:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320494+0000", 2024-09-13T09:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.435e-06 2024-09-13T09:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320502+0000", 2024-09-13T09:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.571e-06 2024-09-13T09:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320507+0000", 2024-09-13T09:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7589999999999997e-06 2024-09-13T09:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332874+0000", 2024-09-13T09:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012366902000000001 2024-09-13T09:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332921+0000", 2024-09-13T09:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7615999999999999e-05 2024-09-13T09:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332993+0000", 2024-09-13T09:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2000000000000002e-05 2024-09-13T09:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.437 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.437 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:29.438 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:29.439 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:29.436+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:29.454 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T09:12:29.468 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T09:12:29.468 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:12:29.531 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone6 from 0x8a1dc00 (overlap 0x19e600) to 0x29600 2024-09-13T09:12:29.553 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:29.551+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:29.562 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:29.647 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:29.645+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:29.715 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:29.715 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:29.715 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:29.716 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:29.716 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:29.724 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:29.741 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:29.739+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:29.810 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:12:29.810 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:12:29.822 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2024-09-13T09:12:29.522281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":204005594,"num_objects":128,"num_object_clones":21,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":10,"num_read":19127,"num_read_kb":68000,"num_write":2127,"num_write_kb":17156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1848,"ondisk_log_size":1848,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":163720,"kb_used_data":7960,"kb_used_omap":7,"kb_used_meta":155512,"kb_avail":471695480,"statfs":{"total":483183820800,"available":483016171520,"internally_reserved":0,"allocated":8151040,"data_stored":6718515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7971,"internal_metadata":159244509},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":173330265,"num_objects":79,"num_object_clones":21,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":10,"num_read":12052,"num_read_kb":52495,"num_write":1151,"num_write_kb":9527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000558"},"pg_stats":[{"pgid":"3.1f","version":"33'49","reported_seq":473,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.078574+0000","last_change":"2024-09-13T09:12:16.380367+0000","last_active":"2024-09-13T09:12:26.078574+0000","last_peered":"2024-09-13T09:12:26.078574+0000","last_clean":"2024-09-13T09:12:26.078574+0000","last_became_active":"2024-09-13T09:12:14.373518+0000","last_became_peered":"2024-09-13T09:12:14.373518+0000","last_unstale":"2024-09-13T09:12:26.078574+0000","last_undegraded":"2024-09-13T09:12:26.078574+0000","last_fullsized":"2024-09-13T09:12:26.078574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:54.097050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058630900000000003,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":934,"num_read_kb":780,"num_write":31,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"33'27","reported_seq":290,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.300038+0000","last_change":"2024-09-13T09:12:16.378252+0000","last_active":"2024-09-13T09:12:26.300038+0000","last_peered":"2024-09-13T09:12:26.300038+0000","last_clean":"2024-09-13T09:12:26.300038+0000","last_became_active":"2024-09-13T09:12:14.371850+0000","last_became_peered":"2024-09-13T09:12:14.371850+0000","last_unstale":"2024-09-13T09:12:26.300038+0000","last_undegraded":"2024-09-13T09:12:26.300038+0000","last_fullsized":"2024-09-13T09:12:26.300038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:54:23.465779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088145900000000002,"stat_sum":{"num_bytes":16744448,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":242,"num_read_kb":2852,"num_write":26,"num_write_kb":331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"33'23","reported_seq":148,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.140318+0000","last_change":"2024-09-13T09:12:16.388369+0000","last_active":"2024-09-13T09:12:26.140318+0000","last_peered":"2024-09-13T09:12:26.140318+0000","last_clean":"2024-09-13T09:12:26.140318+0000","last_became_active":"2024-09-13T09:12:13.362666+0000","last_became_peered":"2024-09-13T09:12:13.362666+0000","last_unstale":"2024-09-13T09:12:26.140318+0000","last_undegraded":"2024-09-13T09:12:26.140318+0000","last_fullsized":"2024-09-13T09:12:26.140318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:04.594826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039115800000000001,"stat_sum":{"num_bytes":4717568,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":101,"num_read_kb":212,"num_write":21,"num_write_kb":362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"34'117","reported_seq":1551,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.454426+0000","last_change":"2024-09-13T09:12:16.388972+0000","last_active":"2024-09-13T09:12:26.454426+0000","last_peered":"2024-09-13T09:12:26.454426+0000","last_clean":"2024-09-13T09:12:26.454426+0000","last_became_active":"2024-09-13T09:12:13.364622+0000","last_became_peered":"2024-09-13T09:12:13.364622+0000","last_unstale":"2024-09-13T09:12:26.454426+0000","last_undegraded":"2024-09-13T09:12:26.454426+0000","last_fullsized":"2024-09-13T09:12:26.454426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:34:20.301837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088725000000000002,"stat_sum":{"num_bytes":5130240,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2814,"num_read_kb":4444,"num_write":66,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"33'39","reported_seq":1937,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.269448+0000","last_change":"2024-09-13T09:12:16.380908+0000","last_active":"2024-09-13T09:12:26.269448+0000","last_peered":"2024-09-13T09:12:26.269448+0000","last_clean":"2024-09-13T09:12:26.269448+0000","last_became_active":"2024-09-13T09:12:13.368138+0000","last_became_peered":"2024-09-13T09:12:13.368138+0000","last_unstale":"2024-09-13T09:12:26.269448+0000","last_undegraded":"2024-09-13T09:12:26.269448+0000","last_fullsized":"2024-09-13T09:12:26.269448+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:10.836108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082447100000000001,"stat_sum":{"num_bytes":19108864,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":1881,"num_read_kb":13683,"num_write":46,"num_write_kb":3201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"33'127","reported_seq":1855,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.483260+0000","last_change":"2024-09-13T09:12:16.380731+0000","last_active":"2024-09-13T09:12:26.483260+0000","last_peered":"2024-09-13T09:12:26.483260+0000","last_clean":"2024-09-13T09:12:26.483260+0000","last_became_active":"2024-09-13T09:12:13.362533+0000","last_became_peered":"2024-09-13T09:12:13.362533+0000","last_unstale":"2024-09-13T09:12:26.483260+0000","last_undegraded":"2024-09-13T09:12:26.483260+0000","last_fullsized":"2024-09-13T09:12:26.483260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:50.399315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072061100000000002,"stat_sum":{"num_bytes":2489875,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2581,"num_read_kb":5265,"num_write":89,"num_write_kb":618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"34'9","reported_seq":35,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.430622+0000","last_change":"2024-09-13T09:12:16.376447+0000","last_active":"2024-09-13T09:12:26.430622+0000","last_peered":"2024-09-13T09:12:26.430622+0000","last_clean":"2024-09-13T09:12:26.430622+0000","last_became_active":"2024-09-13T09:12:13.368221+0000","last_became_peered":"2024-09-13T09:12:13.368221+0000","last_unstale":"2024-09-13T09:12:26.430622+0000","last_undegraded":"2024-09-13T09:12:26.430622+0000","last_fullsized":"2024-09-13T09:12:26.430622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:10.966835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000740483,"stat_sum":{"num_bytes":458815,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"34'44","reported_seq":280,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.479632+0000","last_change":"2024-09-13T09:12:16.377490+0000","last_active":"2024-09-13T09:12:26.479632+0000","last_peered":"2024-09-13T09:12:26.479632+0000","last_clean":"2024-09-13T09:12:26.479632+0000","last_became_active":"2024-09-13T09:12:13.367773+0000","last_became_peered":"2024-09-13T09:12:13.367773+0000","last_unstale":"2024-09-13T09:12:26.479632+0000","last_undegraded":"2024-09-13T09:12:26.479632+0000","last_fullsized":"2024-09-13T09:12:26.479632+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:34.023964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000479879,"stat_sum":{"num_bytes":1245184,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":553,"num_read_kb":515,"num_write":38,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"32'18","reported_seq":88,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.428164+0000","last_change":"2024-09-13T09:12:16.381209+0000","last_active":"2024-09-13T09:12:25.428164+0000","last_peered":"2024-09-13T09:12:25.428164+0000","last_clean":"2024-09-13T09:12:25.428164+0000","last_became_active":"2024-09-13T09:12:13.365820+0000","last_became_peered":"2024-09-13T09:12:13.365820+0000","last_unstale":"2024-09-13T09:12:25.428164+0000","last_undegraded":"2024-09-13T09:12:25.428164+0000","last_fullsized":"2024-09-13T09:12:25.428164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:48.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097766600000000008,"stat_sum":{"num_bytes":1179648,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":1236,"num_write":17,"num_write_kb":224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"33'30","reported_seq":230,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.299534+0000","last_change":"2024-09-13T09:12:16.388388+0000","last_active":"2024-09-13T09:12:26.299534+0000","last_peered":"2024-09-13T09:12:26.299534+0000","last_clean":"2024-09-13T09:12:26.299534+0000","last_became_active":"2024-09-13T09:12:14.377461+0000","last_became_peered":"2024-09-13T09:12:14.377461+0000","last_unstale":"2024-09-13T09:12:26.299534+0000","last_undegraded":"2024-09-13T09:12:26.299534+0000","last_fullsized":"2024-09-13T09:12:26.299534+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:45.926455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040009000000000003,"stat_sum":{"num_bytes":1740800,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":3055,"num_write":30,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"33'29","reported_seq":753,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.299458+0000","last_change":"2024-09-13T09:12:16.380705+0000","last_active":"2024-09-13T09:12:26.299458+0000","last_peered":"2024-09-13T09:12:26.299458+0000","last_clean":"2024-09-13T09:12:26.299458+0000","last_became_active":"2024-09-13T09:12:14.378265+0000","last_became_peered":"2024-09-13T09:12:14.378265+0000","last_unstale":"2024-09-13T09:12:26.299458+0000","last_undegraded":"2024-09-13T09:12:26.299458+0000","last_fullsized":"2024-09-13T09:12:26.299458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:13:12.655905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172600000000004,"stat_sum":{"num_bytes":3112960,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":706,"num_read_kb":5047,"num_write":28,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"34'54","reported_seq":262,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.471912+0000","last_change":"2024-09-13T09:12:16.380762+0000","last_active":"2024-09-13T09:12:26.471912+0000","last_peered":"2024-09-13T09:12:26.471912+0000","last_clean":"2024-09-13T09:12:26.471912+0000","last_became_active":"2024-09-13T09:12:14.377400+0000","last_became_peered":"2024-09-13T09:12:14.377400+0000","last_unstale":"2024-09-13T09:12:26.471912+0000","last_undegraded":"2024-09-13T09:12:26.471912+0000","last_fullsized":"2024-09-13T09:12:26.471912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:56:26.153094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074689299999999999,"stat_sum":{"num_bytes":20285008,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":192,"num_read_kb":397,"num_write":53,"num_write_kb":517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.423221+0000","last_change":"2024-09-13T09:12:14.368984+0000","last_active":"2024-09-13T09:12:25.423221+0000","last_peered":"2024-09-13T09:12:25.423221+0000","last_clean":"2024-09-13T09:12:25.423221+0000","last_became_active":"2024-09-13T09:12:14.368688+0000","last_became_peered":"2024-09-13T09:12:14.368688+0000","last_unstale":"2024-09-13T09:12:25.423221+0000","last_undegraded":"2024-09-13T09:12:25.423221+0000","last_fullsized":"2024-09-13T09:12:25.423221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:35:03.005158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"34'276","reported_seq":409,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.433552+0000","last_change":"2024-09-13T09:12:16.380593+0000","last_active":"2024-09-13T09:12:26.433552+0000","last_peered":"2024-09-13T09:12:26.433552+0000","last_clean":"2024-09-13T09:12:26.433552+0000","last_became_active":"2024-09-13T09:12:13.365496+0000","last_became_peered":"2024-09-13T09:12:13.365496+0000","last_unstale":"2024-09-13T09:12:26.433552+0000","last_undegraded":"2024-09-13T09:12:26.433552+0000","last_fullsized":"2024-09-13T09:12:26.433552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:29:49.115977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061775199999999995,"stat_sum":{"num_bytes":2359360,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1418,"num_read_kb":3657,"num_write":464,"num_write_kb":650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.420372+0000","last_change":"2024-09-13T09:12:14.380765+0000","last_active":"2024-09-13T09:12:25.420372+0000","last_peered":"2024-09-13T09:12:25.420372+0000","last_clean":"2024-09-13T09:12:25.420372+0000","last_became_active":"2024-09-13T09:12:14.380571+0000","last_became_peered":"2024-09-13T09:12:14.380571+0000","last_unstale":"2024-09-13T09:12:25.420372+0000","last_undegraded":"2024-09-13T09:12:25.420372+0000","last_fullsized":"2024-09-13T09:12:25.420372+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:14.941610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"34'157","reported_seq":2063,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.489647+0000","last_change":"2024-09-13T09:12:16.378483+0000","last_active":"2024-09-13T09:12:26.489647+0000","last_peered":"2024-09-13T09:12:26.489647+0000","last_clean":"2024-09-13T09:12:26.489647+0000","last_became_active":"2024-09-13T09:12:13.367146+0000","last_became_peered":"2024-09-13T09:12:13.367146+0000","last_unstale":"2024-09-13T09:12:26.489647+0000","last_undegraded":"2024-09-13T09:12:26.489647+0000","last_fullsized":"2024-09-13T09:12:26.489647+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:49.370653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032368300000000001,"stat_sum":{"num_bytes":2100736,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3163,"num_read_kb":5625,"num_write":92,"num_write_kb":560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.428180+0000","last_change":"2024-09-13T09:12:14.378512+0000","last_active":"2024-09-13T09:12:25.428180+0000","last_peered":"2024-09-13T09:12:25.428180+0000","last_clean":"2024-09-13T09:12:25.428180+0000","last_became_active":"2024-09-13T09:12:14.378221+0000","last_became_peered":"2024-09-13T09:12:14.378221+0000","last_unstale":"2024-09-13T09:12:25.428180+0000","last_undegraded":"2024-09-13T09:12:25.428180+0000","last_fullsized":"2024-09-13T09:12:25.428180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:13.447319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"33'27","reported_seq":303,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.269444+0000","last_change":"2024-09-13T09:12:16.376528+0000","last_active":"2024-09-13T09:12:26.269444+0000","last_peered":"2024-09-13T09:12:26.269444+0000","last_clean":"2024-09-13T09:12:26.269444+0000","last_became_active":"2024-09-13T09:12:13.367848+0000","last_became_peered":"2024-09-13T09:12:13.367848+0000","last_unstale":"2024-09-13T09:12:26.269444+0000","last_undegraded":"2024-09-13T09:12:26.269444+0000","last_fullsized":"2024-09-13T09:12:26.269444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:21.943266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067051299999999997,"stat_sum":{"num_bytes":17876992,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":261,"num_read_kb":2552,"num_write":25,"num_write_kb":169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.420320+0000","last_change":"2024-09-13T09:12:06.328091+0000","last_active":"2024-09-13T09:12:25.420320+0000","last_peered":"2024-09-13T09:12:25.420320+0000","last_clean":"2024-09-13T09:12:25.420320+0000","last_became_active":"2024-09-13T09:12:04.446508+0000","last_became_peered":"2024-09-13T09:12:04.446508+0000","last_unstale":"2024-09-13T09:12:25.420320+0000","last_undegraded":"2024-09-13T09:12:25.420320+0000","last_fullsized":"2024-09-13T09:12:25.420320+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:33.410873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042514000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"33'213","reported_seq":325,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.236425+0000","last_change":"2024-09-13T09:12:16.389006+0000","last_active":"2024-09-13T09:12:26.236425+0000","last_peered":"2024-09-13T09:12:26.236425+0000","last_clean":"2024-09-13T09:12:26.236425+0000","last_became_active":"2024-09-13T09:12:13.369489+0000","last_became_peered":"2024-09-13T09:12:13.369489+0000","last_unstale":"2024-09-13T09:12:26.236425+0000","last_undegraded":"2024-09-13T09:12:26.236425+0000","last_fullsized":"2024-09-13T09:12:26.236425+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:33.879162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088595999999999998,"stat_sum":{"num_bytes":2228351,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":965,"num_read_kb":935,"num_write":332,"num_write_kb":635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"33'24","reported_seq":82,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.198039+0000","last_change":"2024-09-13T09:12:16.380981+0000","last_active":"2024-09-13T09:12:26.198039+0000","last_peered":"2024-09-13T09:12:26.198039+0000","last_clean":"2024-09-13T09:12:26.198039+0000","last_became_active":"2024-09-13T09:12:13.367770+0000","last_became_peered":"2024-09-13T09:12:13.367770+0000","last_unstale":"2024-09-13T09:12:26.198039+0000","last_undegraded":"2024-09-13T09:12:26.198039+0000","last_fullsized":"2024-09-13T09:12:26.198039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:21:50.690425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083150800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":302,"num_write":24,"num_write_kb":313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":33,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.421474+0000","last_change":"2024-09-13T09:12:06.330649+0000","last_active":"2024-09-13T09:12:25.421474+0000","last_peered":"2024-09-13T09:12:25.421474+0000","last_clean":"2024-09-13T09:12:25.421474+0000","last_became_active":"2024-09-13T09:12:04.325470+0000","last_became_peered":"2024-09-13T09:12:04.325470+0000","last_unstale":"2024-09-13T09:12:25.421474+0000","last_undegraded":"2024-09-13T09:12:25.421474+0000","last_fullsized":"2024-09-13T09:12:25.421474+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:37.971615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093634899999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"33'14","reported_seq":135,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.807753+0000","last_change":"2024-09-13T09:12:16.378750+0000","last_active":"2024-09-13T09:12:25.807753+0000","last_peered":"2024-09-13T09:12:25.807753+0000","last_clean":"2024-09-13T09:12:25.807753+0000","last_became_active":"2024-09-13T09:12:13.367656+0000","last_became_peered":"2024-09-13T09:12:13.367656+0000","last_unstale":"2024-09-13T09:12:25.807753+0000","last_undegraded":"2024-09-13T09:12:25.807753+0000","last_fullsized":"2024-09-13T09:12:25.807753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:11.942294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003044836,"stat_sum":{"num_bytes":1717264,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":107,"num_read_kb":1403,"num_write":13,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.416922+0000","last_change":"2024-09-13T09:12:06.845125+0000","last_active":"2024-09-13T09:12:25.416922+0000","last_peered":"2024-09-13T09:12:25.416922+0000","last_clean":"2024-09-13T09:12:25.416922+0000","last_became_active":"2024-09-13T09:12:04.321941+0000","last_became_peered":"2024-09-13T09:12:04.321941+0000","last_unstale":"2024-09-13T09:12:25.416922+0000","last_undegraded":"2024-09-13T09:12:25.416922+0000","last_fullsized":"2024-09-13T09:12:25.416922+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:57.706060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031569600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"33'21","reported_seq":223,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.300092+0000","last_change":"2024-09-13T09:12:16.380222+0000","last_active":"2024-09-13T09:12:26.300092+0000","last_peered":"2024-09-13T09:12:26.300092+0000","last_clean":"2024-09-13T09:12:26.300092+0000","last_became_active":"2024-09-13T09:12:13.367633+0000","last_became_peered":"2024-09-13T09:12:13.367633+0000","last_unstale":"2024-09-13T09:12:26.300092+0000","last_undegraded":"2024-09-13T09:12:26.300092+0000","last_fullsized":"2024-09-13T09:12:26.300092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:34.734632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438009,"stat_sum":{"num_bytes":5595136,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":180,"num_read_kb":2888,"num_write":21,"num_write_kb":313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.420110+0000","last_change":"2024-09-13T09:12:14.375145+0000","last_active":"2024-09-13T09:12:25.420110+0000","last_peered":"2024-09-13T09:12:25.420110+0000","last_clean":"2024-09-13T09:12:25.420110+0000","last_became_active":"2024-09-13T09:12:14.374798+0000","last_became_peered":"2024-09-13T09:12:14.374798+0000","last_unstale":"2024-09-13T09:12:25.420110+0000","last_undegraded":"2024-09-13T09:12:25.420110+0000","last_fullsized":"2024-09-13T09:12:25.420110+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:47.212138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"33'153","reported_seq":256,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.197422+0000","last_change":"2024-09-13T09:12:16.377544+0000","last_active":"2024-09-13T09:12:26.197422+0000","last_peered":"2024-09-13T09:12:26.197422+0000","last_clean":"2024-09-13T09:12:26.197422+0000","last_became_active":"2024-09-13T09:12:13.361909+0000","last_became_peered":"2024-09-13T09:12:13.361909+0000","last_unstale":"2024-09-13T09:12:26.197422+0000","last_undegraded":"2024-09-13T09:12:26.197422+0000","last_fullsized":"2024-09-13T09:12:26.197422+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:34.570438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001546782,"stat_sum":{"num_bytes":488535,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":857,"num_read_kb":842,"num_write":267,"num_write_kb":374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.421547+0000","last_change":"2024-09-13T09:12:06.330140+0000","last_active":"2024-09-13T09:12:25.421547+0000","last_peered":"2024-09-13T09:12:25.421547+0000","last_clean":"2024-09-13T09:12:25.421547+0000","last_became_active":"2024-09-13T09:12:04.324857+0000","last_became_peered":"2024-09-13T09:12:04.324857+0000","last_unstale":"2024-09-13T09:12:25.421547+0000","last_undegraded":"2024-09-13T09:12:25.421547+0000","last_fullsized":"2024-09-13T09:12:25.421547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:57.400286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038911000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":192,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.427929+0000","last_change":"2024-09-13T09:12:17.384247+0000","last_active":"2024-09-13T09:12:25.427929+0000","last_peered":"2024-09-13T09:12:25.427929+0000","last_clean":"2024-09-13T09:12:25.427929+0000","last_became_active":"2024-09-13T09:12:17.383949+0000","last_became_peered":"2024-09-13T09:12:17.383949+0000","last_unstale":"2024-09-13T09:12:25.427929+0000","last_undegraded":"2024-09-13T09:12:25.427929+0000","last_fullsized":"2024-09-13T09:12:25.427929+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:44:43.174965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"33'27","reported_seq":233,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.197703+0000","last_change":"2024-09-13T09:12:16.377642+0000","last_active":"2024-09-13T09:12:26.197703+0000","last_peered":"2024-09-13T09:12:26.197703+0000","last_clean":"2024-09-13T09:12:26.197703+0000","last_became_active":"2024-09-13T09:12:13.369742+0000","last_became_peered":"2024-09-13T09:12:13.369742+0000","last_unstale":"2024-09-13T09:12:26.197703+0000","last_undegraded":"2024-09-13T09:12:26.197703+0000","last_fullsized":"2024-09-13T09:12:26.197703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:27:48.361252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060355399999999996,"stat_sum":{"num_bytes":15204368,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":190,"num_read_kb":157,"num_write":26,"num_write_kb":401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"33'52","reported_seq":523,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.299920+0000","last_change":"2024-09-13T09:12:16.380989+0000","last_active":"2024-09-13T09:12:26.299920+0000","last_peered":"2024-09-13T09:12:26.299920+0000","last_clean":"2024-09-13T09:12:26.299920+0000","last_became_active":"2024-09-13T09:12:13.370096+0000","last_became_peered":"2024-09-13T09:12:13.370096+0000","last_unstale":"2024-09-13T09:12:26.299920+0000","last_undegraded":"2024-09-13T09:12:26.299920+0000","last_fullsized":"2024-09-13T09:12:26.299920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:14.224546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000903458,"stat_sum":{"num_bytes":21512192,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":456,"num_read_kb":4224,"num_write":50,"num_write_kb":476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"34'40","reported_seq":100,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.435680+0000","last_change":"2024-09-13T09:12:16.380353+0000","last_active":"2024-09-13T09:12:26.435680+0000","last_peered":"2024-09-13T09:12:26.435680+0000","last_clean":"2024-09-13T09:12:26.435680+0000","last_became_active":"2024-09-13T09:12:14.371868+0000","last_became_peered":"2024-09-13T09:12:14.371868+0000","last_unstale":"2024-09-13T09:12:26.435680+0000","last_undegraded":"2024-09-13T09:12:26.435680+0000","last_fullsized":"2024-09-13T09:12:26.435680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:52.308848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041459800000000002,"stat_sum":{"num_bytes":2161152,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":41,"num_read_kb":198,"num_write":38,"num_write_kb":517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"33'32","reported_seq":382,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.272600+0000","last_change":"2024-09-13T09:12:16.377864+0000","last_active":"2024-09-13T09:12:26.272600+0000","last_peered":"2024-09-13T09:12:26.272600+0000","last_clean":"2024-09-13T09:12:26.272600+0000","last_became_active":"2024-09-13T09:12:13.362409+0000","last_became_peered":"2024-09-13T09:12:13.362409+0000","last_unstale":"2024-09-13T09:12:26.272600+0000","last_undegraded":"2024-09-13T09:12:26.272600+0000","last_fullsized":"2024-09-13T09:12:26.272600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:37.023594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001713904,"stat_sum":{"num_bytes":31204864,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":334,"num_read_kb":348,"num_write":37,"num_write_kb":2181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"33'33","reported_seq":322,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.301328+0000","last_change":"2024-09-13T09:12:16.389470+0000","last_active":"2024-09-13T09:12:26.301328+0000","last_peered":"2024-09-13T09:12:26.301328+0000","last_clean":"2024-09-13T09:12:26.301328+0000","last_became_active":"2024-09-13T09:12:13.365758+0000","last_became_peered":"2024-09-13T09:12:13.365758+0000","last_unstale":"2024-09-13T09:12:26.301328+0000","last_undegraded":"2024-09-13T09:12:26.301328+0000","last_fullsized":"2024-09-13T09:12:26.301328+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:46.649879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013123500000000001,"stat_sum":{"num_bytes":18433580,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":290,"num_read_kb":1710,"num_write":32,"num_write_kb":436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"27'4","reported_seq":29,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.417068+0000","last_change":"2024-09-13T09:12:16.378016+0000","last_active":"2024-09-13T09:12:25.417068+0000","last_peered":"2024-09-13T09:12:25.417068+0000","last_clean":"2024-09-13T09:12:25.417068+0000","last_became_active":"2024-09-13T09:12:14.369550+0000","last_became_peered":"2024-09-13T09:12:14.369550+0000","last_unstale":"2024-09-13T09:12:25.417068+0000","last_undegraded":"2024-09-13T09:12:25.417068+0000","last_fullsized":"2024-09-13T09:12:25.417068+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:50.552432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000679308,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"33'18","reported_seq":139,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.073358+0000","last_change":"2024-09-13T09:12:16.389243+0000","last_active":"2024-09-13T09:12:26.073358+0000","last_peered":"2024-09-13T09:12:26.073358+0000","last_clean":"2024-09-13T09:12:26.073358+0000","last_became_active":"2024-09-13T09:12:13.362547+0000","last_became_peered":"2024-09-13T09:12:13.362547+0000","last_unstale":"2024-09-13T09:12:26.073358+0000","last_undegraded":"2024-09-13T09:12:26.073358+0000","last_fullsized":"2024-09-13T09:12:26.073358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:24:24.557400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011202860000000001,"stat_sum":{"num_bytes":1441792,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":1816,"num_write":17,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"33'18","reported_seq":98,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.197440+0000","last_change":"2024-09-13T09:12:16.388621+0000","last_active":"2024-09-13T09:12:26.197440+0000","last_peered":"2024-09-13T09:12:26.197440+0000","last_clean":"2024-09-13T09:12:26.197440+0000","last_became_active":"2024-09-13T09:12:14.377144+0000","last_became_peered":"2024-09-13T09:12:14.377144+0000","last_unstale":"2024-09-13T09:12:26.197440+0000","last_undegraded":"2024-09-13T09:12:26.197440+0000","last_fullsized":"2024-09-13T09:12:26.197440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:36.921070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059255799999999997,"stat_sum":{"num_bytes":917612,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":150,"num_write":18,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"33'11","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.272685+0000","last_change":"2024-09-13T09:12:16.389277+0000","last_active":"2024-09-13T09:12:26.272685+0000","last_peered":"2024-09-13T09:12:26.272685+0000","last_clean":"2024-09-13T09:12:26.272685+0000","last_became_active":"2024-09-13T09:12:13.367757+0000","last_became_peered":"2024-09-13T09:12:13.367757+0000","last_unstale":"2024-09-13T09:12:26.272685+0000","last_undegraded":"2024-09-13T09:12:26.272685+0000","last_fullsized":"2024-09-13T09:12:26.272685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:33.596156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011263219999999999,"stat_sum":{"num_bytes":1241088,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":72,"num_write":11,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"33'19","reported_seq":264,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.299434+0000","last_change":"2024-09-13T09:12:16.380313+0000","last_active":"2024-09-13T09:12:26.299434+0000","last_peered":"2024-09-13T09:12:26.299434+0000","last_clean":"2024-09-13T09:12:26.299434+0000","last_became_active":"2024-09-13T09:12:14.373807+0000","last_became_peered":"2024-09-13T09:12:14.373807+0000","last_unstale":"2024-09-13T09:12:26.299434+0000","last_undegraded":"2024-09-13T09:12:26.299434+0000","last_fullsized":"2024-09-13T09:12:26.299434+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:07.135989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034391499999999998,"stat_sum":{"num_bytes":1911847,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":229,"num_read_kb":3252,"num_write":19,"num_write_kb":206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"33'8","reported_seq":61,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.075030+0000","last_change":"2024-09-13T09:12:16.380043+0000","last_active":"2024-09-13T09:12:26.075030+0000","last_peered":"2024-09-13T09:12:26.075030+0000","last_clean":"2024-09-13T09:12:26.075030+0000","last_became_active":"2024-09-13T09:12:14.373710+0000","last_became_peered":"2024-09-13T09:12:14.373710+0000","last_unstale":"2024-09-13T09:12:26.075030+0000","last_undegraded":"2024-09-13T09:12:26.075030+0000","last_fullsized":"2024-09-13T09:12:26.075030+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:19:36.129916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049261300000000004,"stat_sum":{"num_bytes":786432,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":28,"num_read_kb":88,"num_write":8,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"33'15","reported_seq":90,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.077837+0000","last_change":"2024-09-13T09:12:16.388608+0000","last_active":"2024-09-13T09:12:26.077837+0000","last_peered":"2024-09-13T09:12:26.077837+0000","last_clean":"2024-09-13T09:12:26.077837+0000","last_became_active":"2024-09-13T09:12:14.377339+0000","last_became_peered":"2024-09-13T09:12:14.377339+0000","last_unstale":"2024-09-13T09:12:26.077837+0000","last_undegraded":"2024-09-13T09:12:26.077837+0000","last_fullsized":"2024-09-13T09:12:26.077837+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:36:17.808759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059370899999999999,"stat_sum":{"num_bytes":20480,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":52,"num_read_kb":195,"num_write":15,"num_write_kb":244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":203415207,"num_objects":124,"num_object_clones":21,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":10,"num_read":19011,"num_read_kb":67900,"num_write":1966,"num_write_kb":14738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":5095424,"data_stored":4769102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1728,"ondisk_log_size":1728,"up":64,"acting":64,"num_store_stats":5},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640265,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1396736,"data_stored":1091642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]}]},{"osd":4,"up_from":11,"seq":47244640266,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1785856,"data_stored":1517811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]}]},{"osd":3,"up_from":0,"seq":47244640267,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":11,"seq":47244640265,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":843776,"data_stored":566632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.17599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]}]},{"osd":1,"up_from":11,"seq":47244640266,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1859584,"data_stored":1561014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33324,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2265088,"data_stored":1981416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1318912,"data_stored":1258767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":913408,"data_stored":838409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":417455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1532105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":722366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:12:29.825 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:12:29.855 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-13T09:12:29.861 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:29.840+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:29.880 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:285 3.7 3:e76cef2e:::rbd_data.1123ffac6bc2.00000000000000c2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.778209+0000", 2024-09-13T09:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.100388479999999, 2024-09-13T09:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051374489999999997, 2024-09-13T09:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 285 2024-09-13T09:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778209+0000", 2024-09-13T09:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778209+0000", 2024-09-13T09:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778209+0000", 2024-09-13T09:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778213+0000", 2024-09-13T09:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.301e-06 2024-09-13T09:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778226+0000", 2024-09-13T09:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2923e-05 2024-09-13T09:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778233+0000", 2024-09-13T09:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5739999999999998e-06 2024-09-13T09:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778279+0000", 2024-09-13T09:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5300000000000003e-05 2024-09-13T09:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778352+0000", 2024-09-13T09:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2960999999999995e-05 2024-09-13T09:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778461+0000", 2024-09-13T09:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010903700000000001 2024-09-13T09:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.780559+0000", 2024-09-13T09:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020987150000000001 2024-09-13T09:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783266+0000", 2024-09-13T09:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002707096 2024-09-13T09:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783307+0000", 2024-09-13T09:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0043999999999998e-05 2024-09-13T09:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783347+0000", 2024-09-13T09:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0333000000000002e-05 2024-09-13T09:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:288 3.1e 3:7da6a7d6:::rbd_data.1123ffac6bc2.00000000000000cb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.778243+0000", 2024-09-13T09:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.100355133000001, 2024-09-13T09:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050784439999999997, 2024-09-13T09:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 288 2024-09-13T09:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778243+0000", 2024-09-13T09:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778243+0000", 2024-09-13T09:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778242+0000", 2024-09-13T09:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778245+0000", 2024-09-13T09:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9909999999999998e-06 2024-09-13T09:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778264+0000", 2024-09-13T09:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8698999999999998e-05 2024-09-13T09:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778268+0000", 2024-09-13T09:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.002e-06 2024-09-13T09:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778302+0000", 2024-09-13T09:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3926e-05 2024-09-13T09:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778348+0000", 2024-09-13T09:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6458999999999997e-05 2024-09-13T09:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778427+0000", 2024-09-13T09:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8645000000000003e-05 2024-09-13T09:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.779006+0000", 2024-09-13T09:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00057902799999999999 2024-09-13T09:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783241+0000", 2024-09-13T09:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042353790000000001 2024-09-13T09:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783286+0000", 2024-09-13T09:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4537999999999999e-05 2024-09-13T09:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783321+0000", 2024-09-13T09:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5308999999999997e-05 2024-09-13T09:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:290 3.8 3:17d1c290:::rbd_data.1123ffac6bc2.00000000000000e6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.778368+0000", 2024-09-13T09:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.100230158, 2024-09-13T09:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049790379999999999, 2024-09-13T09:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 290 2024-09-13T09:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778368+0000", 2024-09-13T09:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778368+0000", 2024-09-13T09:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778367+0000", 2024-09-13T09:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778371+0000", 2024-09-13T09:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.98e-06 2024-09-13T09:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778382+0000", 2024-09-13T09:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778388+0000", 2024-09-13T09:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3459999999999999e-06 2024-09-13T09:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778425+0000", 2024-09-13T09:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6721000000000003e-05 2024-09-13T09:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778486+0000", 2024-09-13T09:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1344000000000001e-05 2024-09-13T09:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778585+0000", 2024-09-13T09:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8061999999999997e-05 2024-09-13T09:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.780584+0000", 2024-09-13T09:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019999649999999998 2024-09-13T09:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783298+0000", 2024-09-13T09:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027135779999999999 2024-09-13T09:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783326+0000", 2024-09-13T09:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7727999999999999e-05 2024-09-13T09:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783347+0000", 2024-09-13T09:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0971e-05 2024-09-13T09:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:393 3.7 3:e1888e44:::rbd_data.1123ffac6bc2.00000000000000a4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.836837+0000", 2024-09-13T09:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.04176062, 2024-09-13T09:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048048379999999996, 2024-09-13T09:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 393 2024-09-13T09:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836837+0000", 2024-09-13T09:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836837+0000", 2024-09-13T09:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836835+0000", 2024-09-13T09:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-09-13T09:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836844+0000", 2024-09-13T09:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2930000000000001e-06 2024-09-13T09:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836860+0000", 2024-09-13T09:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5772000000000001e-05 2024-09-13T09:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836868+0000", 2024-09-13T09:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0090000000000007e-06 2024-09-13T09:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836932+0000", 2024-09-13T09:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4336000000000002e-05 2024-09-13T09:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837025+0000", 2024-09-13T09:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3282999999999994e-05 2024-09-13T09:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837149+0000", 2024-09-13T09:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000123776 2024-09-13T09:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.838672+0000", 2024-09-13T09:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015235579999999999 2024-09-13T09:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.841574+0000", 2024-09-13T09:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029013899999999998 2024-09-13T09:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.841609+0000", 2024-09-13T09:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5278999999999999e-05 2024-09-13T09:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.841642+0000", 2024-09-13T09:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2947999999999997e-05 2024-09-13T09:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:451 3.18 3:1c88c91a:::rbd_data.1123ffac6bc2.000000000000002d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.865164+0000", 2024-09-13T09:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.013433597000001, 2024-09-13T09:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051471370000000004, 2024-09-13T09:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 451 2024-09-13T09:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865164+0000", 2024-09-13T09:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865164+0000", 2024-09-13T09:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865163+0000", 2024-09-13T09:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865169+0000", 2024-09-13T09:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.798e-06 2024-09-13T09:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865185+0000", 2024-09-13T09:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5404e-05 2024-09-13T09:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865193+0000", 2024-09-13T09:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865240+0000", 2024-09-13T09:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7293000000000003e-05 2024-09-13T09:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865331+0000", 2024-09-13T09:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0672000000000002e-05 2024-09-13T09:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865410+0000", 2024-09-13T09:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9319000000000004e-05 2024-09-13T09:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.867480+0000", 2024-09-13T09:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020703430000000001 2024-09-13T09:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870227+0000", 2024-09-13T09:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00274705 2024-09-13T09:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870272+0000", 2024-09-13T09:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4554000000000001e-05 2024-09-13T09:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870311+0000", 2024-09-13T09:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9536000000000002e-05 2024-09-13T09:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:453 3.7 3:e75bf08e:::rbd_data.1123ffac6bc2.000000000000002e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.865211+0000", 2024-09-13T09:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.013387206000001, 2024-09-13T09:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051785149999999999, 2024-09-13T09:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 453 2024-09-13T09:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865211+0000", 2024-09-13T09:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865211+0000", 2024-09-13T09:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865210+0000", 2024-09-13T09:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865213+0000", 2024-09-13T09:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865222+0000", 2024-09-13T09:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8380000000000008e-06 2024-09-13T09:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865228+0000", 2024-09-13T09:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9900000000000002e-06 2024-09-13T09:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865276+0000", 2024-09-13T09:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7407999999999997e-05 2024-09-13T09:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865364+0000", 2024-09-13T09:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8412000000000007e-05 2024-09-13T09:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865471+0000", 2024-09-13T09:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000106609 2024-09-13T09:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.869879+0000", 2024-09-13T09:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044085640000000002 2024-09-13T09:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870308+0000", 2024-09-13T09:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00042895900000000002 2024-09-13T09:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870353+0000", 2024-09-13T09:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4749999999999997e-05 2024-09-13T09:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870389+0000", 2024-09-13T09:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6269999999999997e-05 2024-09-13T09:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:1052 3.7 3:e7c11647:::rbd_data.1123ffac6bc2.0000000000000090:head [set-alloc-hint object_size 1048576 write_size 1048576,write 503808~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.140204+0000", 2024-09-13T09:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.7383942710000007, 2024-09-13T09:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055287460000000002, 2024-09-13T09:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1052 2024-09-13T09:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140204+0000", 2024-09-13T09:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140204+0000", 2024-09-13T09:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140202+0000", 2024-09-13T09:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-09-13T09:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140242+0000", 2024-09-13T09:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0512000000000001e-05 2024-09-13T09:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140263+0000", 2024-09-13T09:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1268e-05 2024-09-13T09:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140277+0000", 2024-09-13T09:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3346000000000001e-05 2024-09-13T09:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140333+0000", 2024-09-13T09:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5695000000000001e-05 2024-09-13T09:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140461+0000", 2024-09-13T09:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012853800000000001 2024-09-13T09:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140645+0000", 2024-09-13T09:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000183463 2024-09-13T09:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145465+0000", 2024-09-13T09:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048209289999999998 2024-09-13T09:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145639+0000", 2024-09-13T09:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017306799999999999 2024-09-13T09:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145696+0000", 2024-09-13T09:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7599000000000002e-05 2024-09-13T09:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145732+0000", 2024-09-13T09:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6254000000000002e-05 2024-09-13T09:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:1055 3.18 3:192ac1d1:::rbd_data.1123ffac6bc2.0000000000000091:head [set-alloc-hint object_size 1048576 write_size 1048576,write 393216~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.140290+0000", 2024-09-13T09:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.738307399, 2024-09-13T09:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.005447454, 2024-09-13T09:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1055 2024-09-13T09:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140290+0000", 2024-09-13T09:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140290+0000", 2024-09-13T09:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140290+0000", 2024-09-13T09:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140311+0000", 2024-09-13T09:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1484e-05 2024-09-13T09:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140320+0000", 2024-09-13T09:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1500000000000005e-06 2024-09-13T09:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140324+0000", 2024-09-13T09:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140375+0000", 2024-09-13T09:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1082000000000003e-05 2024-09-13T09:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140504+0000", 2024-09-13T09:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012903599999999999 2024-09-13T09:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140608+0000", 2024-09-13T09:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000103669 2024-09-13T09:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142875+0000", 2024-09-13T09:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002267388 2024-09-13T09:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145639+0000", 2024-09-13T09:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027635369999999999 2024-09-13T09:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145700+0000", 2024-09-13T09:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1453999999999998e-05 2024-09-13T09:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145738+0000", 2024-09-13T09:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7478999999999998e-05 2024-09-13T09:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:1069 3.5 e27/21 3:a4a542ae:::rbd_data.1123ffac6bc2.0000000000000036:head v 27'123, mlcod=27'121)", 2024-09-13T09:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.142984+0000", 2024-09-13T09:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.7356141110000003, 2024-09-13T09:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052211899999999997, 2024-09-13T09:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142984+0000", 2024-09-13T09:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142984+0000", 2024-09-13T09:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142983+0000", 2024-09-13T09:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142996+0000", 2024-09-13T09:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3361e-05 2024-09-13T09:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143012+0000", 2024-09-13T09:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5574999999999999e-05 2024-09-13T09:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143019+0000", 2024-09-13T09:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7290000000000003e-06 2024-09-13T09:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143066+0000", 2024-09-13T09:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7250999999999999e-05 2024-09-13T09:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143088+0000", 2024-09-13T09:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2450999999999999e-05 2024-09-13T09:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148171+0000", 2024-09-13T09:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050823190000000001 2024-09-13T09:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148205+0000", 2024-09-13T09:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4350000000000001e-05 2024-09-13T09:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:396 3.8 3:1239702a:::rbd_data.11238839e366.0000000000000006:6 [sparse-read 16515072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.282786+0000", 2024-09-13T09:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.5958122580000005, 2024-09-13T09:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046983729999999996, 2024-09-13T09:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 396 2024-09-13T09:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.282786+0000", 2024-09-13T09:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.282786+0000", 2024-09-13T09:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.282785+0000", 2024-09-13T09:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.282787+0000", 2024-09-13T09:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.624e-06 2024-09-13T09:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.282792+0000", 2024-09-13T09:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2780000000000001e-06 2024-09-13T09:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.282796+0000", 2024-09-13T09:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5949999999999999e-06 2024-09-13T09:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.287333+0000", 2024-09-13T09:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045368270000000002 2024-09-13T09:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.287406+0000", 2024-09-13T09:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2854000000000004e-05 2024-09-13T09:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.287484+0000", 2024-09-13T09:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8499999999999997e-05 2024-09-13T09:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:503 3.8 3:1239702a:::rbd_data.11238839e366.0000000000000006:6 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.284527+0000", 2024-09-13T09:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.5940704649999997, 2024-09-13T09:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004859057, 2024-09-13T09:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 503 2024-09-13T09:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.284527+0000", 2024-09-13T09:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.284527+0000", 2024-09-13T09:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.284527+0000", 2024-09-13T09:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.284534+0000", 2024-09-13T09:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3679999999999996e-06 2024-09-13T09:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.284541+0000", 2024-09-13T09:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3470000000000004e-06 2024-09-13T09:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.284546+0000", 2024-09-13T09:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1479999999999997e-06 2024-09-13T09:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.289186+0000", 2024-09-13T09:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046402539999999999 2024-09-13T09:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.289278+0000", 2024-09-13T09:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1766000000000002e-05 2024-09-13T09:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.289386+0000", 2024-09-13T09:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0001087 2024-09-13T09:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1997 3.8 3:17d36caf:::rbd_data.1123b8b4a28a.0000000000000012:head [set-alloc-hint object_size 2097152 write_size 2097152,write 524288~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.507177+0000", 2024-09-13T09:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.3714207179999995, 2024-09-13T09:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0085576109999999997, 2024-09-13T09:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1997 2024-09-13T09:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507177+0000", 2024-09-13T09:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507177+0000", 2024-09-13T09:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507175+0000", 2024-09-13T09:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-09-13T09:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507217+0000", 2024-09-13T09:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2543000000000002e-05 2024-09-13T09:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507238+0000", 2024-09-13T09:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0445000000000002e-05 2024-09-13T09:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507247+0000", 2024-09-13T09:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5959999999999993e-06 2024-09-13T09:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507299+0000", 2024-09-13T09:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1966999999999999e-05 2024-09-13T09:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507448+0000", 2024-09-13T09:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014851600000000001 2024-09-13T09:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507598+0000", 2024-09-13T09:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015040100000000001 2024-09-13T09:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.513345+0000", 2024-09-13T09:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057464170000000002 2024-09-13T09:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515644+0000", 2024-09-13T09:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022998300000000001 2024-09-13T09:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515697+0000", 2024-09-13T09:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2210000000000003e-05 2024-09-13T09:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515735+0000", 2024-09-13T09:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8126000000000001e-05 2024-09-13T09:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2296 3.7 3:e05a98c0:::rbd_data.1123b8b4a28a.000000000000002f:head [write 1933312~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.617024+0000", 2024-09-13T09:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.261573523, 2024-09-13T09:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0054662820000000003, 2024-09-13T09:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2296 2024-09-13T09:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617024+0000", 2024-09-13T09:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617024+0000", 2024-09-13T09:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617023+0000", 2024-09-13T09:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617043+0000", 2024-09-13T09:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9673e-05 2024-09-13T09:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617058+0000", 2024-09-13T09:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4966999999999999e-05 2024-09-13T09:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617066+0000", 2024-09-13T09:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7069999999999992e-06 2024-09-13T09:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617138+0000", 2024-09-13T09:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2360999999999994e-05 2024-09-13T09:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617213+0000", 2024-09-13T09:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5009e-05 2024-09-13T09:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617356+0000", 2024-09-13T09:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014259 2024-09-13T09:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618868+0000", 2024-09-13T09:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015122480000000001 2024-09-13T09:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.622378+0000", 2024-09-13T09:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0035100069999999999 2024-09-13T09:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.622426+0000", 2024-09-13T09:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8106999999999999e-05 2024-09-13T09:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.622491+0000", 2024-09-13T09:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4759999999999997e-05 2024-09-13T09:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4567.0:2311 3.9 e33/21 3:93ffaa8c:::rbd_data.1123b8b4a28a.0000000000000041:head v 33'32, mlcod=33'30)", 2024-09-13T09:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.619681+0000", 2024-09-13T09:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2589169780000002, 2024-09-13T09:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049754830000000002, 2024-09-13T09:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619681+0000", 2024-09-13T09:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619681+0000", 2024-09-13T09:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619680+0000", 2024-09-13T09:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619692+0000", 2024-09-13T09:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.24e-05 2024-09-13T09:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619708+0000", 2024-09-13T09:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5475e-05 2024-09-13T09:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619715+0000", 2024-09-13T09:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1219999999999997e-06 2024-09-13T09:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619781+0000", 2024-09-13T09:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5957999999999994e-05 2024-09-13T09:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619801+0000", 2024-09-13T09:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9658999999999999e-05 2024-09-13T09:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.624615+0000", 2024-09-13T09:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048141399999999997 2024-09-13T09:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.624656+0000", 2024-09-13T09:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1619000000000003e-05 2024-09-13T09:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4567.0:2316 3.1f e33/22 3:f905e326:::rbd_data.1123b8b4a28a.0000000000000042:head v 33'45, mlcod=32'43)", 2024-09-13T09:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.620013+0000", 2024-09-13T09:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2585844479999997, 2024-09-13T09:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047162929999999999, 2024-09-13T09:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620013+0000", 2024-09-13T09:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620013+0000", 2024-09-13T09:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620013+0000", 2024-09-13T09:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620029+0000", 2024-09-13T09:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6342e-05 2024-09-13T09:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620044+0000", 2024-09-13T09:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4958e-05 2024-09-13T09:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620050+0000", 2024-09-13T09:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2439999999999998e-06 2024-09-13T09:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620115+0000", 2024-09-13T09:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4462999999999996e-05 2024-09-13T09:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620134+0000", 2024-09-13T09:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9692999999999999e-05 2024-09-13T09:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.624698+0000", 2024-09-13T09:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045634120000000002 2024-09-13T09:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.624730+0000", 2024-09-13T09:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2027999999999999e-05 2024-09-13T09:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2436 3.18 3:195218e5:::rbd_data.1123b8b4a28a.0000000000000039:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.674604+0000", 2024-09-13T09:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2039934099999998, 2024-09-13T09:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055259439999999996, 2024-09-13T09:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2436 2024-09-13T09:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674604+0000", 2024-09-13T09:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674604+0000", 2024-09-13T09:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674603+0000", 2024-09-13T09:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674609+0000", 2024-09-13T09:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0660000000000003e-06 2024-09-13T09:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674621+0000", 2024-09-13T09:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.202e-05 2024-09-13T09:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674628+0000", 2024-09-13T09:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5020000000000003e-06 2024-09-13T09:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674677+0000", 2024-09-13T09:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9211e-05 2024-09-13T09:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674774+0000", 2024-09-13T09:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6341000000000005e-05 2024-09-13T09:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674899+0000", 2024-09-13T09:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000125537 2024-09-13T09:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.676820+0000", 2024-09-13T09:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019208109999999999 2024-09-13T09:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680047+0000", 2024-09-13T09:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0032271130000000002 2024-09-13T09:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680095+0000", 2024-09-13T09:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7691999999999997e-05 2024-09-13T09:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680130+0000", 2024-09-13T09:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5651999999999997e-05 2024-09-13T09:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2440 3.18 3:1b8df09d:::rbd_data.1123b8b4a28a.000000000000003c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.675520+0000", 2024-09-13T09:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.203077908, 2024-09-13T09:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047183549999999996, 2024-09-13T09:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2440 2024-09-13T09:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675520+0000", 2024-09-13T09:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675520+0000", 2024-09-13T09:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675519+0000", 2024-09-13T09:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675523+0000", 2024-09-13T09:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675532+0000", 2024-09-13T09:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675537+0000", 2024-09-13T09:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.429e-06 2024-09-13T09:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675903+0000", 2024-09-13T09:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000366051 2024-09-13T09:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675977+0000", 2024-09-13T09:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3775000000000004e-05 2024-09-13T09:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.676093+0000", 2024-09-13T09:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000115912 2024-09-13T09:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.676826+0000", 2024-09-13T09:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00073320999999999998 2024-09-13T09:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680167+0000", 2024-09-13T09:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033408650000000002 2024-09-13T09:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680207+0000", 2024-09-13T09:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9901999999999998e-05 2024-09-13T09:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680238+0000", 2024-09-13T09:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1587000000000001e-05 2024-09-13T09:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4567.0:2454 3.9 e33/21 3:96917efd:::rbd_data.1123b8b4a28a.0000000000000066:head v 33'37, mlcod=33'33)", 2024-09-13T09:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.677108+0000", 2024-09-13T09:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2014895299999999, 2024-09-13T09:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046617639999999997, 2024-09-13T09:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677108+0000", 2024-09-13T09:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677108+0000", 2024-09-13T09:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677108+0000", 2024-09-13T09:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677113+0000", 2024-09-13T09:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7309999999999998e-06 2024-09-13T09:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677122+0000", 2024-09-13T09:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9080000000000001e-06 2024-09-13T09:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677137+0000", 2024-09-13T09:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4358e-05 2024-09-13T09:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677175+0000", 2024-09-13T09:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8696999999999999e-05 2024-09-13T09:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677192+0000", 2024-09-13T09:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6328999999999999e-05 2024-09-13T09:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.681720+0000", 2024-09-13T09:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00452799 2024-09-13T09:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.681770+0000", 2024-09-13T09:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0386999999999997e-05 2024-09-13T09:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2486 3.7 3:e1179f14:::rbd_header.1123b8b4a28a:head [notify cookie 140249875563536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.680326+0000", 2024-09-13T09:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.1982716739999999, 2024-09-13T09:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046642200000000002, 2024-09-13T09:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2486 2024-09-13T09:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680326+0000", 2024-09-13T09:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680326+0000", 2024-09-13T09:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680325+0000", 2024-09-13T09:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680331+0000", 2024-09-13T09:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7880000000000003e-06 2024-09-13T09:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680343+0000", 2024-09-13T09:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2026e-05 2024-09-13T09:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680350+0000", 2024-09-13T09:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9149999999999999e-06 2024-09-13T09:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.684893+0000", 2024-09-13T09:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045435379999999997 2024-09-13T09:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.684927+0000", 2024-09-13T09:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3200000000000001e-05 2024-09-13T09:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.684990+0000", 2024-09-13T09:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3870999999999999e-05 2024-09-13T09:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2490 3.7 3:e1179f14:::rbd_header.1123b8b4a28a:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.680585+0000", 2024-09-13T09:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.1980131219999999, 2024-09-13T09:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047373210000000001, 2024-09-13T09:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2490 2024-09-13T09:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680585+0000", 2024-09-13T09:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680585+0000", 2024-09-13T09:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680584+0000", 2024-09-13T09:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680615+0000", 2024-09-13T09:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0546000000000002e-05 2024-09-13T09:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680624+0000", 2024-09-13T09:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6320000000000003e-06 2024-09-13T09:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680630+0000", 2024-09-13T09:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7130000000000002e-06 2024-09-13T09:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.685242+0000", 2024-09-13T09:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046121699999999996 2024-09-13T09:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.685277+0000", 2024-09-13T09:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4307999999999997e-05 2024-09-13T09:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.685322+0000", 2024-09-13T09:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5438000000000001e-05 2024-09-13T09:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:29.988 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:29.989 INFO:teuthology.orchestra.run.smithi157.stdout:1008 write 0xe6b4600 thru 0xe6ba6dc (0x60dd bytes) 2024-09-13T09:12:29.989 INFO:teuthology.orchestra.run.smithi157.stdout:1011 read 0xa069e00 thru 0xa078c78 (0xee79 bytes) 2024-09-13T09:12:29.989 INFO:teuthology.orchestra.run.smithi157.stdout:1012 read 0x4a4ee00 thru 0x4a5c30c (0xd50d bytes) 2024-09-13T09:12:29.989 INFO:teuthology.orchestra.run.smithi157.stdout:1015 read 0xdf26a00 thru 0xdf2cba1 (0x61a2 bytes) 2024-09-13T09:12:29.989 INFO:teuthology.orchestra.run.smithi157.stdout:1017 punch from 0x6093200 to 0x6097400, (0x4200 bytes) 2024-09-13T09:12:29.989 INFO:teuthology.orchestra.run.smithi157.stdout:1019 write 0xeaa5a00 thru 0xeaad6ae (0x7caf bytes) 2024-09-13T09:12:29.989 INFO:teuthology.orchestra.run.smithi157.stdout:1021 write 0x46fa600 thru 0x47097e5 (0xf1e6 bytes) 2024-09-13T09:12:29.990 INFO:teuthology.orchestra.run.smithi157.stdout:1023 write 0xa9de600 thru 0xa9ea51a (0xbf1b bytes) 2024-09-13T09:12:29.990 INFO:teuthology.orchestra.run.smithi157.stdout:1024 read 0xbb6da00 thru 0xbb7196e (0x3f6f bytes) 2024-09-13T09:12:29.990 INFO:teuthology.orchestra.run.smithi157.stdout:1028 read 0x4a73000 thru 0x4a82dbb (0xfdbc bytes) 2024-09-13T09:12:29.990 INFO:teuthology.orchestra.run.smithi157.stdout:1031 write 0x351ba00 thru 0x351e08d (0x268e bytes) 2024-09-13T09:12:29.990 INFO:teuthology.orchestra.run.smithi157.stdout:1033 trunc from 0xeaad800 to 0xbd5d800 2024-09-13T09:12:29.990 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:29.985+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:29.991 INFO:teuthology.orchestra.run.smithi157.stdout:1035 read 0x1348000 thru 0x1354ffd (0xcffe bytes) 2024-09-13T09:12:29.992 INFO:teuthology.orchestra.run.smithi157.stdout:1036 punch from 0xb84b400 to 0xb85a400, (0xf000 bytes) 2024-09-13T09:12:29.997 INFO:teuthology.orchestra.run.smithi157.stdout:1037 trunc from 0xbd5d800 to 0x7a25200 2024-09-13T09:12:30.039 INFO:teuthology.orchestra.run.smithi157.stdout:1038 write 0xa909200 thru 0xa91033d (0x713e bytes) 2024-09-13T09:12:30.052 INFO:teuthology.orchestra.run.smithi157.stdout:1039 trunc from 0xa910400 to 0x4d50000 2024-09-13T09:12:30.073 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:12:30.077 INFO:teuthology.orchestra.run.smithi157.stdout:1041 read 0xbd7000 thru 0xbd81e8 (0x11e9 bytes) 2024-09-13T09:12:30.087 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:30.086+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:30.100 INFO:teuthology.orchestra.run.smithi157.stdout:1043 write 0x98ca400 thru 0x98d15e5 (0x71e6 bytes) 2024-09-13T09:12:30.100 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:30.100 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:30.100 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:30.100 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:30.101 INFO:teuthology.orchestra.run.smithi157.stdout:1044 read 0x2d8d800 thru 0x2d8d87c (0x7d bytes) 2024-09-13T09:12:30.101 INFO:teuthology.orchestra.run.smithi157.stdout:1045 write 0x2dcba00 thru 0x2dd3934 (0x7f35 bytes) 2024-09-13T09:12:30.102 INFO:teuthology.orchestra.run.smithi157.stdout:1047 read 0x626a400 thru 0x6279434 (0xf035 bytes) 2024-09-13T09:12:30.102 INFO:teuthology.orchestra.run.smithi157.stdout:1048 read 0x447c800 thru 0x4481b2f (0x5330 bytes) 2024-09-13T09:12:30.102 INFO:teuthology.orchestra.run.smithi157.stdout:1049 read 0x257a200 thru 0x25833f7 (0x91f8 bytes) 2024-09-13T09:12:30.105 INFO:teuthology.orchestra.run.smithi157.stdout:1050 trunc from 0x98d1600 to 0xc0eb600 2024-09-13T09:12:30.110 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:30.117 INFO:teuthology.orchestra.run.smithi157.stdout:1051 punch from 0x2957e00 to 0x295c800, (0x4a00 bytes) 2024-09-13T09:12:30.125 INFO:teuthology.orchestra.run.smithi157.stdout:1052 trunc from 0xc0eb600 to 0x5cb9c00 2024-09-13T09:12:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:12:30.147 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:12:30.148 INFO:teuthology.orchestra.run.smithi157.stdout:1054 trunc from 0x5cb9c00 to 0xd197c00 2024-09-13T09:12:30.159 INFO:teuthology.orchestra.run.smithi157.stdout:1056 trunc from 0xd197c00 to 0xcc76a00 2024-09-13T09:12:30.162 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2024-09-13T09:12:29.522281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":204005594,"num_objects":128,"num_object_clones":21,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":10,"num_read":19127,"num_read_kb":68000,"num_write":2127,"num_write_kb":17156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1848,"ondisk_log_size":1848,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":163720,"kb_used_data":7960,"kb_used_omap":7,"kb_used_meta":155512,"kb_avail":471695480,"statfs":{"total":483183820800,"available":483016171520,"internally_reserved":0,"allocated":8151040,"data_stored":6718515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7971,"internal_metadata":159244509},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":173330265,"num_objects":79,"num_object_clones":21,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":10,"num_read":12052,"num_read_kb":52495,"num_write":1151,"num_write_kb":9527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000558"},"pg_stats":[{"pgid":"3.1f","version":"33'49","reported_seq":473,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.078574+0000","last_change":"2024-09-13T09:12:16.380367+0000","last_active":"2024-09-13T09:12:26.078574+0000","last_peered":"2024-09-13T09:12:26.078574+0000","last_clean":"2024-09-13T09:12:26.078574+0000","last_became_active":"2024-09-13T09:12:14.373518+0000","last_became_peered":"2024-09-13T09:12:14.373518+0000","last_unstale":"2024-09-13T09:12:26.078574+0000","last_undegraded":"2024-09-13T09:12:26.078574+0000","last_fullsized":"2024-09-13T09:12:26.078574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:54.097050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058630900000000003,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":934,"num_read_kb":780,"num_write":31,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"33'27","reported_seq":290,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.300038+0000","last_change":"2024-09-13T09:12:16.378252+0000","last_active":"2024-09-13T09:12:26.300038+0000","last_peered":"2024-09-13T09:12:26.300038+0000","last_clean":"2024-09-13T09:12:26.300038+0000","last_became_active":"2024-09-13T09:12:14.371850+0000","last_became_peered":"2024-09-13T09:12:14.371850+0000","last_unstale":"2024-09-13T09:12:26.300038+0000","last_undegraded":"2024-09-13T09:12:26.300038+0000","last_fullsized":"2024-09-13T09:12:26.300038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:54:23.465779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088145900000000002,"stat_sum":{"num_bytes":16744448,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":242,"num_read_kb":2852,"num_write":26,"num_write_kb":331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"33'23","reported_seq":148,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.140318+0000","last_change":"2024-09-13T09:12:16.388369+0000","last_active":"2024-09-13T09:12:26.140318+0000","last_peered":"2024-09-13T09:12:26.140318+0000","last_clean":"2024-09-13T09:12:26.140318+0000","last_became_active":"2024-09-13T09:12:13.362666+0000","last_became_peered":"2024-09-13T09:12:13.362666+0000","last_unstale":"2024-09-13T09:12:26.140318+0000","last_undegraded":"2024-09-13T09:12:26.140318+0000","last_fullsized":"2024-09-13T09:12:26.140318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:04.594826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039115800000000001,"stat_sum":{"num_bytes":4717568,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":101,"num_read_kb":212,"num_write":21,"num_write_kb":362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"34'117","reported_seq":1551,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.454426+0000","last_change":"2024-09-13T09:12:16.388972+0000","last_active":"2024-09-13T09:12:26.454426+0000","last_peered":"2024-09-13T09:12:26.454426+0000","last_clean":"2024-09-13T09:12:26.454426+0000","last_became_active":"2024-09-13T09:12:13.364622+0000","last_became_peered":"2024-09-13T09:12:13.364622+0000","last_unstale":"2024-09-13T09:12:26.454426+0000","last_undegraded":"2024-09-13T09:12:26.454426+0000","last_fullsized":"2024-09-13T09:12:26.454426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:34:20.301837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088725000000000002,"stat_sum":{"num_bytes":5130240,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2814,"num_read_kb":4444,"num_write":66,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"33'39","reported_seq":1937,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.269448+0000","last_change":"2024-09-13T09:12:16.380908+0000","last_active":"2024-09-13T09:12:26.269448+0000","last_peered":"2024-09-13T09:12:26.269448+0000","last_clean":"2024-09-13T09:12:26.269448+0000","last_became_active":"2024-09-13T09:12:13.368138+0000","last_became_peered":"2024-09-13T09:12:13.368138+0000","last_unstale":"2024-09-13T09:12:26.269448+0000","last_undegraded":"2024-09-13T09:12:26.269448+0000","last_fullsized":"2024-09-13T09:12:26.269448+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:10.836108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082447100000000001,"stat_sum":{"num_bytes":19108864,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":1881,"num_read_kb":13683,"num_write":46,"num_write_kb":3201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"33'127","reported_seq":1855,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.483260+0000","last_change":"2024-09-13T09:12:16.380731+0000","last_active":"2024-09-13T09:12:26.483260+0000","last_peered":"2024-09-13T09:12:26.483260+0000","last_clean":"2024-09-13T09:12:26.483260+0000","last_became_active":"2024-09-13T09:12:13.362533+0000","last_became_peered":"2024-09-13T09:12:13.362533+0000","last_unstale":"2024-09-13T09:12:26.483260+0000","last_undegraded":"2024-09-13T09:12:26.483260+0000","last_fullsized":"2024-09-13T09:12:26.483260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:50.399315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072061100000000002,"stat_sum":{"num_bytes":2489875,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2581,"num_read_kb":5265,"num_write":89,"num_write_kb":618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"34'9","reported_seq":35,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.430622+0000","last_change":"2024-09-13T09:12:16.376447+0000","last_active":"2024-09-13T09:12:26.430622+0000","last_peered":"2024-09-13T09:12:26.430622+0000","last_clean":"2024-09-13T09:12:26.430622+0000","last_became_active":"2024-09-13T09:12:13.368221+0000","last_became_peered":"2024-09-13T09:12:13.368221+0000","last_unstale":"2024-09-13T09:12:26.430622+0000","last_undegraded":"2024-09-13T09:12:26.430622+0000","last_fullsized":"2024-09-13T09:12:26.430622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:10.966835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000740483,"stat_sum":{"num_bytes":458815,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"34'44","reported_seq":280,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.479632+0000","last_change":"2024-09-13T09:12:16.377490+0000","last_active":"2024-09-13T09:12:26.479632+0000","last_peered":"2024-09-13T09:12:26.479632+0000","last_clean":"2024-09-13T09:12:26.479632+0000","last_became_active":"2024-09-13T09:12:13.367773+0000","last_became_peered":"2024-09-13T09:12:13.367773+0000","last_unstale":"2024-09-13T09:12:26.479632+0000","last_undegraded":"2024-09-13T09:12:26.479632+0000","last_fullsized":"2024-09-13T09:12:26.479632+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:34.023964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000479879,"stat_sum":{"num_bytes":1245184,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":553,"num_read_kb":515,"num_write":38,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"32'18","reported_seq":88,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.428164+0000","last_change":"2024-09-13T09:12:16.381209+0000","last_active":"2024-09-13T09:12:25.428164+0000","last_peered":"2024-09-13T09:12:25.428164+0000","last_clean":"2024-09-13T09:12:25.428164+0000","last_became_active":"2024-09-13T09:12:13.365820+0000","last_became_peered":"2024-09-13T09:12:13.365820+0000","last_unstale":"2024-09-13T09:12:25.428164+0000","last_undegraded":"2024-09-13T09:12:25.428164+0000","last_fullsized":"2024-09-13T09:12:25.428164+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:48.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097766600000000008,"stat_sum":{"num_bytes":1179648,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":1236,"num_write":17,"num_write_kb":224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"33'30","reported_seq":230,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.299534+0000","last_change":"2024-09-13T09:12:16.388388+0000","last_active":"2024-09-13T09:12:26.299534+0000","last_peered":"2024-09-13T09:12:26.299534+0000","last_clean":"2024-09-13T09:12:26.299534+0000","last_became_active":"2024-09-13T09:12:14.377461+0000","last_became_peered":"2024-09-13T09:12:14.377461+0000","last_unstale":"2024-09-13T09:12:26.299534+0000","last_undegraded":"2024-09-13T09:12:26.299534+0000","last_fullsized":"2024-09-13T09:12:26.299534+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:45.926455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040009000000000003,"stat_sum":{"num_bytes":1740800,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":3055,"num_write":30,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"33'29","reported_seq":753,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.299458+0000","last_change":"2024-09-13T09:12:16.380705+0000","last_active":"2024-09-13T09:12:26.299458+0000","last_peered":"2024-09-13T09:12:26.299458+0000","last_clean":"2024-09-13T09:12:26.299458+0000","last_became_active":"2024-09-13T09:12:14.378265+0000","last_became_peered":"2024-09-13T09:12:14.378265+0000","last_unstale":"2024-09-13T09:12:26.299458+0000","last_undegraded":"2024-09-13T09:12:26.299458+0000","last_fullsized":"2024-09-13T09:12:26.299458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:13:12.655905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172600000000004,"stat_sum":{"num_bytes":3112960,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":706,"num_read_kb":5047,"num_write":28,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"34'54","reported_seq":262,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.471912+0000","last_change":"2024-09-13T09:12:16.380762+0000","last_active":"2024-09-13T09:12:26.471912+0000","last_peered":"2024-09-13T09:12:26.471912+0000","last_clean":"2024-09-13T09:12:26.471912+0000","last_became_active":"2024-09-13T09:12:14.377400+0000","last_became_peered":"2024-09-13T09:12:14.377400+0000","last_unstale":"2024-09-13T09:12:26.471912+0000","last_undegraded":"2024-09-13T09:12:26.471912+0000","last_fullsized":"2024-09-13T09:12:26.471912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:56:26.153094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074689299999999999,"stat_sum":{"num_bytes":20285008,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":192,"num_read_kb":397,"num_write":53,"num_write_kb":517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.423221+0000","last_change":"2024-09-13T09:12:14.368984+0000","last_active":"2024-09-13T09:12:25.423221+0000","last_peered":"2024-09-13T09:12:25.423221+0000","last_clean":"2024-09-13T09:12:25.423221+0000","last_became_active":"2024-09-13T09:12:14.368688+0000","last_became_peered":"2024-09-13T09:12:14.368688+0000","last_unstale":"2024-09-13T09:12:25.423221+0000","last_undegraded":"2024-09-13T09:12:25.423221+0000","last_fullsized":"2024-09-13T09:12:25.423221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:35:03.005158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"34'276","reported_seq":409,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.433552+0000","last_change":"2024-09-13T09:12:16.380593+0000","last_active":"2024-09-13T09:12:26.433552+0000","last_peered":"2024-09-13T09:12:26.433552+0000","last_clean":"2024-09-13T09:12:26.433552+0000","last_became_active":"2024-09-13T09:12:13.365496+0000","last_became_peered":"2024-09-13T09:12:13.365496+0000","last_unstale":"2024-09-13T09:12:26.433552+0000","last_undegraded":"2024-09-13T09:12:26.433552+0000","last_fullsized":"2024-09-13T09:12:26.433552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:29:49.115977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061775199999999995,"stat_sum":{"num_bytes":2359360,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1418,"num_read_kb":3657,"num_write":464,"num_write_kb":650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.420372+0000","last_change":"2024-09-13T09:12:14.380765+0000","last_active":"2024-09-13T09:12:25.420372+0000","last_peered":"2024-09-13T09:12:25.420372+0000","last_clean":"2024-09-13T09:12:25.420372+0000","last_became_active":"2024-09-13T09:12:14.380571+0000","last_became_peered":"2024-09-13T09:12:14.380571+0000","last_unstale":"2024-09-13T09:12:25.420372+0000","last_undegraded":"2024-09-13T09:12:25.420372+0000","last_fullsized":"2024-09-13T09:12:25.420372+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:14.941610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"34'157","reported_seq":2063,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.489647+0000","last_change":"2024-09-13T09:12:16.378483+0000","last_active":"2024-09-13T09:12:26.489647+0000","last_peered":"2024-09-13T09:12:26.489647+0000","last_clean":"2024-09-13T09:12:26.489647+0000","last_became_active":"2024-09-13T09:12:13.367146+0000","last_became_peered":"2024-09-13T09:12:13.367146+0000","last_unstale":"2024-09-13T09:12:26.489647+0000","last_undegraded":"2024-09-13T09:12:26.489647+0000","last_fullsized":"2024-09-13T09:12:26.489647+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:49.370653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032368300000000001,"stat_sum":{"num_bytes":2100736,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3163,"num_read_kb":5625,"num_write":92,"num_write_kb":560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.428180+0000","last_change":"2024-09-13T09:12:14.378512+0000","last_active":"2024-09-13T09:12:25.428180+0000","last_peered":"2024-09-13T09:12:25.428180+0000","last_clean":"2024-09-13T09:12:25.428180+0000","last_became_active":"2024-09-13T09:12:14.378221+0000","last_became_peered":"2024-09-13T09:12:14.378221+0000","last_unstale":"2024-09-13T09:12:25.428180+0000","last_undegraded":"2024-09-13T09:12:25.428180+0000","last_fullsized":"2024-09-13T09:12:25.428180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:13.447319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"33'27","reported_seq":303,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.269444+0000","last_change":"2024-09-13T09:12:16.376528+0000","last_active":"2024-09-13T09:12:26.269444+0000","last_peered":"2024-09-13T09:12:26.269444+0000","last_clean":"2024-09-13T09:12:26.269444+0000","last_became_active":"2024-09-13T09:12:13.367848+0000","last_became_peered":"2024-09-13T09:12:13.367848+0000","last_unstale":"2024-09-13T09:12:26.269444+0000","last_undegraded":"2024-09-13T09:12:26.269444+0000","last_fullsized":"2024-09-13T09:12:26.269444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:21.943266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067051299999999997,"stat_sum":{"num_bytes":17876992,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":261,"num_read_kb":2552,"num_write":25,"num_write_kb":169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.420320+0000","last_change":"2024-09-13T09:12:06.328091+0000","last_active":"2024-09-13T09:12:25.420320+0000","last_peered":"2024-09-13T09:12:25.420320+0000","last_clean":"2024-09-13T09:12:25.420320+0000","last_became_active":"2024-09-13T09:12:04.446508+0000","last_became_peered":"2024-09-13T09:12:04.446508+0000","last_unstale":"2024-09-13T09:12:25.420320+0000","last_undegraded":"2024-09-13T09:12:25.420320+0000","last_fullsized":"2024-09-13T09:12:25.420320+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:33.410873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042514000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"33'213","reported_seq":325,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.236425+0000","last_change":"2024-09-13T09:12:16.389006+0000","last_active":"2024-09-13T09:12:26.236425+0000","last_peered":"2024-09-13T09:12:26.236425+0000","last_clean":"2024-09-13T09:12:26.236425+0000","last_became_active":"2024-09-13T09:12:13.369489+0000","last_became_peered":"2024-09-13T09:12:13.369489+0000","last_unstale":"2024-09-13T09:12:26.236425+0000","last_undegraded":"2024-09-13T09:12:26.236425+0000","last_fullsized":"2024-09-13T09:12:26.236425+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:33.879162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088595999999999998,"stat_sum":{"num_bytes":2228351,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":965,"num_read_kb":935,"num_write":332,"num_write_kb":635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"33'24","reported_seq":82,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.198039+0000","last_change":"2024-09-13T09:12:16.380981+0000","last_active":"2024-09-13T09:12:26.198039+0000","last_peered":"2024-09-13T09:12:26.198039+0000","last_clean":"2024-09-13T09:12:26.198039+0000","last_became_active":"2024-09-13T09:12:13.367770+0000","last_became_peered":"2024-09-13T09:12:13.367770+0000","last_unstale":"2024-09-13T09:12:26.198039+0000","last_undegraded":"2024-09-13T09:12:26.198039+0000","last_fullsized":"2024-09-13T09:12:26.198039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:21:50.690425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083150800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":302,"num_write":24,"num_write_kb":313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":33,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.421474+0000","last_change":"2024-09-13T09:12:06.330649+0000","last_active":"2024-09-13T09:12:25.421474+0000","last_peered":"2024-09-13T09:12:25.421474+0000","last_clean":"2024-09-13T09:12:25.421474+0000","last_became_active":"2024-09-13T09:12:04.325470+0000","last_became_peered":"2024-09-13T09:12:04.325470+0000","last_unstale":"2024-09-13T09:12:25.421474+0000","last_undegraded":"2024-09-13T09:12:25.421474+0000","last_fullsized":"2024-09-13T09:12:25.421474+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:37.971615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093634899999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"33'14","reported_seq":135,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.807753+0000","last_change":"2024-09-13T09:12:16.378750+0000","last_active":"2024-09-13T09:12:25.807753+0000","last_peered":"2024-09-13T09:12:25.807753+0000","last_clean":"2024-09-13T09:12:25.807753+0000","last_became_active":"2024-09-13T09:12:13.367656+0000","last_became_peered":"2024-09-13T09:12:13.367656+0000","last_unstale":"2024-09-13T09:12:25.807753+0000","last_undegraded":"2024-09-13T09:12:25.807753+0000","last_fullsized":"2024-09-13T09:12:25.807753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:11.942294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003044836,"stat_sum":{"num_bytes":1717264,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":107,"num_read_kb":1403,"num_write":13,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.416922+0000","last_change":"2024-09-13T09:12:06.845125+0000","last_active":"2024-09-13T09:12:25.416922+0000","last_peered":"2024-09-13T09:12:25.416922+0000","last_clean":"2024-09-13T09:12:25.416922+0000","last_became_active":"2024-09-13T09:12:04.321941+0000","last_became_peered":"2024-09-13T09:12:04.321941+0000","last_unstale":"2024-09-13T09:12:25.416922+0000","last_undegraded":"2024-09-13T09:12:25.416922+0000","last_fullsized":"2024-09-13T09:12:25.416922+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:57.706060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031569600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"33'21","reported_seq":223,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.300092+0000","last_change":"2024-09-13T09:12:16.380222+0000","last_active":"2024-09-13T09:12:26.300092+0000","last_peered":"2024-09-13T09:12:26.300092+0000","last_clean":"2024-09-13T09:12:26.300092+0000","last_became_active":"2024-09-13T09:12:13.367633+0000","last_became_peered":"2024-09-13T09:12:13.367633+0000","last_unstale":"2024-09-13T09:12:26.300092+0000","last_undegraded":"2024-09-13T09:12:26.300092+0000","last_fullsized":"2024-09-13T09:12:26.300092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:34.734632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438009,"stat_sum":{"num_bytes":5595136,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":180,"num_read_kb":2888,"num_write":21,"num_write_kb":313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.420110+0000","last_change":"2024-09-13T09:12:14.375145+0000","last_active":"2024-09-13T09:12:25.420110+0000","last_peered":"2024-09-13T09:12:25.420110+0000","last_clean":"2024-09-13T09:12:25.420110+0000","last_became_active":"2024-09-13T09:12:14.374798+0000","last_became_peered":"2024-09-13T09:12:14.374798+0000","last_unstale":"2024-09-13T09:12:25.420110+0000","last_undegraded":"2024-09-13T09:12:25.420110+0000","last_fullsized":"2024-09-13T09:12:25.420110+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:47.212138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"33'153","reported_seq":256,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.197422+0000","last_change":"2024-09-13T09:12:16.377544+0000","last_active":"2024-09-13T09:12:26.197422+0000","last_peered":"2024-09-13T09:12:26.197422+0000","last_clean":"2024-09-13T09:12:26.197422+0000","last_became_active":"2024-09-13T09:12:13.361909+0000","last_became_peered":"2024-09-13T09:12:13.361909+0000","last_unstale":"2024-09-13T09:12:26.197422+0000","last_undegraded":"2024-09-13T09:12:26.197422+0000","last_fullsized":"2024-09-13T09:12:26.197422+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:34.570438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001546782,"stat_sum":{"num_bytes":488535,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":857,"num_read_kb":842,"num_write":267,"num_write_kb":374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.421547+0000","last_change":"2024-09-13T09:12:06.330140+0000","last_active":"2024-09-13T09:12:25.421547+0000","last_peered":"2024-09-13T09:12:25.421547+0000","last_clean":"2024-09-13T09:12:25.421547+0000","last_became_active":"2024-09-13T09:12:04.324857+0000","last_became_peered":"2024-09-13T09:12:04.324857+0000","last_unstale":"2024-09-13T09:12:25.421547+0000","last_undegraded":"2024-09-13T09:12:25.421547+0000","last_fullsized":"2024-09-13T09:12:25.421547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:57.400286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038911000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":192,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.427929+0000","last_change":"2024-09-13T09:12:17.384247+0000","last_active":"2024-09-13T09:12:25.427929+0000","last_peered":"2024-09-13T09:12:25.427929+0000","last_clean":"2024-09-13T09:12:25.427929+0000","last_became_active":"2024-09-13T09:12:17.383949+0000","last_became_peered":"2024-09-13T09:12:17.383949+0000","last_unstale":"2024-09-13T09:12:25.427929+0000","last_undegraded":"2024-09-13T09:12:25.427929+0000","last_fullsized":"2024-09-13T09:12:25.427929+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:44:43.174965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"33'27","reported_seq":233,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.197703+0000","last_change":"2024-09-13T09:12:16.377642+0000","last_active":"2024-09-13T09:12:26.197703+0000","last_peered":"2024-09-13T09:12:26.197703+0000","last_clean":"2024-09-13T09:12:26.197703+0000","last_became_active":"2024-09-13T09:12:13.369742+0000","last_became_peered":"2024-09-13T09:12:13.369742+0000","last_unstale":"2024-09-13T09:12:26.197703+0000","last_undegraded":"2024-09-13T09:12:26.197703+0000","last_fullsized":"2024-09-13T09:12:26.197703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:27:48.361252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060355399999999996,"stat_sum":{"num_bytes":15204368,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":190,"num_read_kb":157,"num_write":26,"num_write_kb":401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"33'52","reported_seq":523,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.299920+0000","last_change":"2024-09-13T09:12:16.380989+0000","last_active":"2024-09-13T09:12:26.299920+0000","last_peered":"2024-09-13T09:12:26.299920+0000","last_clean":"2024-09-13T09:12:26.299920+0000","last_became_active":"2024-09-13T09:12:13.370096+0000","last_became_peered":"2024-09-13T09:12:13.370096+0000","last_unstale":"2024-09-13T09:12:26.299920+0000","last_undegraded":"2024-09-13T09:12:26.299920+0000","last_fullsized":"2024-09-13T09:12:26.299920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:14.224546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000903458,"stat_sum":{"num_bytes":21512192,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":456,"num_read_kb":4224,"num_write":50,"num_write_kb":476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"34'40","reported_seq":100,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.435680+0000","last_change":"2024-09-13T09:12:16.380353+0000","last_active":"2024-09-13T09:12:26.435680+0000","last_peered":"2024-09-13T09:12:26.435680+0000","last_clean":"2024-09-13T09:12:26.435680+0000","last_became_active":"2024-09-13T09:12:14.371868+0000","last_became_peered":"2024-09-13T09:12:14.371868+0000","last_unstale":"2024-09-13T09:12:26.435680+0000","last_undegraded":"2024-09-13T09:12:26.435680+0000","last_fullsized":"2024-09-13T09:12:26.435680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:52.308848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041459800000000002,"stat_sum":{"num_bytes":2161152,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":41,"num_read_kb":198,"num_write":38,"num_write_kb":517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"33'32","reported_seq":382,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.272600+0000","last_change":"2024-09-13T09:12:16.377864+0000","last_active":"2024-09-13T09:12:26.272600+0000","last_peered":"2024-09-13T09:12:26.272600+0000","last_clean":"2024-09-13T09:12:26.272600+0000","last_became_active":"2024-09-13T09:12:13.362409+0000","last_became_peered":"2024-09-13T09:12:13.362409+0000","last_unstale":"2024-09-13T09:12:26.272600+0000","last_undegraded":"2024-09-13T09:12:26.272600+0000","last_fullsized":"2024-09-13T09:12:26.272600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:37.023594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001713904,"stat_sum":{"num_bytes":31204864,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":334,"num_read_kb":348,"num_write":37,"num_write_kb":2181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"33'33","reported_seq":322,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.301328+0000","last_change":"2024-09-13T09:12:16.389470+0000","last_active":"2024-09-13T09:12:26.301328+0000","last_peered":"2024-09-13T09:12:26.301328+0000","last_clean":"2024-09-13T09:12:26.301328+0000","last_became_active":"2024-09-13T09:12:13.365758+0000","last_became_peered":"2024-09-13T09:12:13.365758+0000","last_unstale":"2024-09-13T09:12:26.301328+0000","last_undegraded":"2024-09-13T09:12:26.301328+0000","last_fullsized":"2024-09-13T09:12:26.301328+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:46.649879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013123500000000001,"stat_sum":{"num_bytes":18433580,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":290,"num_read_kb":1710,"num_write":32,"num_write_kb":436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"27'4","reported_seq":29,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:25.417068+0000","last_change":"2024-09-13T09:12:16.378016+0000","last_active":"2024-09-13T09:12:25.417068+0000","last_peered":"2024-09-13T09:12:25.417068+0000","last_clean":"2024-09-13T09:12:25.417068+0000","last_became_active":"2024-09-13T09:12:14.369550+0000","last_became_peered":"2024-09-13T09:12:14.369550+0000","last_unstale":"2024-09-13T09:12:25.417068+0000","last_undegraded":"2024-09-13T09:12:25.417068+0000","last_fullsized":"2024-09-13T09:12:25.417068+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:50.552432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000679308,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"33'18","reported_seq":139,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.073358+0000","last_change":"2024-09-13T09:12:16.389243+0000","last_active":"2024-09-13T09:12:26.073358+0000","last_peered":"2024-09-13T09:12:26.073358+0000","last_clean":"2024-09-13T09:12:26.073358+0000","last_became_active":"2024-09-13T09:12:13.362547+0000","last_became_peered":"2024-09-13T09:12:13.362547+0000","last_unstale":"2024-09-13T09:12:26.073358+0000","last_undegraded":"2024-09-13T09:12:26.073358+0000","last_fullsized":"2024-09-13T09:12:26.073358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:24:24.557400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011202860000000001,"stat_sum":{"num_bytes":1441792,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":1816,"num_write":17,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"33'18","reported_seq":98,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.197440+0000","last_change":"2024-09-13T09:12:16.388621+0000","last_active":"2024-09-13T09:12:26.197440+0000","last_peered":"2024-09-13T09:12:26.197440+0000","last_clean":"2024-09-13T09:12:26.197440+0000","last_became_active":"2024-09-13T09:12:14.377144+0000","last_became_peered":"2024-09-13T09:12:14.377144+0000","last_unstale":"2024-09-13T09:12:26.197440+0000","last_undegraded":"2024-09-13T09:12:26.197440+0000","last_fullsized":"2024-09-13T09:12:26.197440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:36.921070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059255799999999997,"stat_sum":{"num_bytes":917612,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":150,"num_write":18,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"33'11","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.272685+0000","last_change":"2024-09-13T09:12:16.389277+0000","last_active":"2024-09-13T09:12:26.272685+0000","last_peered":"2024-09-13T09:12:26.272685+0000","last_clean":"2024-09-13T09:12:26.272685+0000","last_became_active":"2024-09-13T09:12:13.367757+0000","last_became_peered":"2024-09-13T09:12:13.367757+0000","last_unstale":"2024-09-13T09:12:26.272685+0000","last_undegraded":"2024-09-13T09:12:26.272685+0000","last_fullsized":"2024-09-13T09:12:26.272685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:33.596156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011263219999999999,"stat_sum":{"num_bytes":1241088,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":72,"num_write":11,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"33'19","reported_seq":264,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.299434+0000","last_change":"2024-09-13T09:12:16.380313+0000","last_active":"2024-09-13T09:12:26.299434+0000","last_peered":"2024-09-13T09:12:26.299434+0000","last_clean":"2024-09-13T09:12:26.299434+0000","last_became_active":"2024-09-13T09:12:14.373807+0000","last_became_peered":"2024-09-13T09:12:14.373807+0000","last_unstale":"2024-09-13T09:12:26.299434+0000","last_undegraded":"2024-09-13T09:12:26.299434+0000","last_fullsized":"2024-09-13T09:12:26.299434+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:07.135989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034391499999999998,"stat_sum":{"num_bytes":1911847,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":229,"num_read_kb":3252,"num_write":19,"num_write_kb":206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"33'8","reported_seq":61,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.075030+0000","last_change":"2024-09-13T09:12:16.380043+0000","last_active":"2024-09-13T09:12:26.075030+0000","last_peered":"2024-09-13T09:12:26.075030+0000","last_clean":"2024-09-13T09:12:26.075030+0000","last_became_active":"2024-09-13T09:12:14.373710+0000","last_became_peered":"2024-09-13T09:12:14.373710+0000","last_unstale":"2024-09-13T09:12:26.075030+0000","last_undegraded":"2024-09-13T09:12:26.075030+0000","last_fullsized":"2024-09-13T09:12:26.075030+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:19:36.129916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049261300000000004,"stat_sum":{"num_bytes":786432,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":28,"num_read_kb":88,"num_write":8,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"33'15","reported_seq":90,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T09:12:26.077837+0000","last_change":"2024-09-13T09:12:16.388608+0000","last_active":"2024-09-13T09:12:26.077837+0000","last_peered":"2024-09-13T09:12:26.077837+0000","last_clean":"2024-09-13T09:12:26.077837+0000","last_became_active":"2024-09-13T09:12:14.377339+0000","last_became_peered":"2024-09-13T09:12:14.377339+0000","last_unstale":"2024-09-13T09:12:26.077837+0000","last_undegraded":"2024-09-13T09:12:26.077837+0000","last_fullsized":"2024-09-13T09:12:26.077837+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:36:17.808759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059370899999999999,"stat_sum":{"num_bytes":20480,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":52,"num_read_kb":195,"num_write":15,"num_write_kb":244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":203415207,"num_objects":124,"num_object_clones":21,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":10,"num_read":19011,"num_read_kb":67900,"num_write":1966,"num_write_kb":14738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":5095424,"data_stored":4769102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1728,"ondisk_log_size":1728,"up":64,"acting":64,"num_store_stats":5},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640265,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1396736,"data_stored":1091642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]}]},{"osd":4,"up_from":11,"seq":47244640266,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1785856,"data_stored":1517811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]}]},{"osd":3,"up_from":0,"seq":47244640267,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":11,"seq":47244640265,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":843776,"data_stored":566632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.17599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]}]},{"osd":1,"up_from":11,"seq":47244640266,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1859584,"data_stored":1561014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33324,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2265088,"data_stored":1981416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1318912,"data_stored":1258767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":913408,"data_stored":838409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":417455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1532105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":722366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:12:30.164 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T09:12:30.164 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:12:30.164 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-13T09:12:30.164 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-13T09:12:30.169 INFO:teuthology.orchestra.run.smithi157.stdout:1058 write 0xa5d1000 thru 0xa5dd6b4 (0xc6b5 bytes) 2024-09-13T09:12:30.169 INFO:teuthology.orchestra.run.smithi157.stdout:1059 punch from 0x640e600 to 0x641da00, (0xf400 bytes) 2024-09-13T09:12:30.170 INFO:teuthology.orchestra.run.smithi157.stdout:1062 write 0x5168000 thru 0x51687d3 (0x7d4 bytes) 2024-09-13T09:12:30.170 INFO:teuthology.orchestra.run.smithi157.stdout:1063 punch from 0x25f8200 to 0x2605200, (0xd000 bytes) 2024-09-13T09:12:30.172 INFO:teuthology.orchestra.run.smithi157.stdout:1064 trunc from 0xcc76a00 to 0xbae2a00 2024-09-13T09:12:30.193 INFO:teuthology.orchestra.run.smithi157.stdout:1065 write 0xac8200 thru 0xaccb1a (0x491b bytes) 2024-09-13T09:12:30.193 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:30.192+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:30.197 INFO:teuthology.orchestra.run.smithi157.stdout:1066 trunc from 0xbae2a00 to 0x8957800 2024-09-13T09:12:30.210 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:30.210 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:30.221 INFO:teuthology.orchestra.run.smithi157.stdout:1067 read 0x27c0c00 thru 0x27c24b7 (0x18b8 bytes) 2024-09-13T09:12:30.230 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:30.230 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:30.241 INFO:teuthology.orchestra.run.smithi157.stdout:1069 write 0xc245200 thru 0xc249cc1 (0x4ac2 bytes) 2024-09-13T09:12:30.241 INFO:teuthology.orchestra.run.smithi157.stdout:1070 read 0x908d400 thru 0x908e7b9 (0x13ba bytes) 2024-09-13T09:12:30.241 INFO:teuthology.orchestra.run.smithi157.stdout:1072 punch from 0x1f52800 to 0x1f54200, (0x1a00 bytes) 2024-09-13T09:12:30.242 INFO:teuthology.orchestra.run.smithi157.stdout:1074 punch from 0x2d5e400 to 0x2d61e00, (0x3a00 bytes) 2024-09-13T09:12:30.243 INFO:teuthology.orchestra.run.smithi157.stdout:1076 trunc from 0xc249e00 to 0xd69fa00 2024-09-13T09:12:30.248 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:30.248 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:30.254 INFO:teuthology.orchestra.run.smithi157.stdout:1077 punch from 0x12dfa00 to 0x12e0c00, (0x1200 bytes) 2024-09-13T09:12:30.255 INFO:teuthology.orchestra.run.smithi157.stdout:1078 punch from 0x379bc00 to 0x37a3200, (0x7600 bytes) 2024-09-13T09:12:30.263 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:30.264 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:30.267 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:30.267 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:30.270 INFO:teuthology.orchestra.run.smithi157.stdout:1079 write 0xd82fa00 thru 0xd831b31 (0x2132 bytes) 2024-09-13T09:12:30.273 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:30.276 INFO:teuthology.orchestra.run.smithi157.stdout:1080 punch from 0xadeac00 to 0xadec000, (0x1400 bytes) 2024-09-13T09:12:30.285 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:30.286 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:30.286 INFO:teuthology.orchestra.run.smithi157.stdout:1081 trunc from 0xd831c00 to 0x59c8a00 2024-09-13T09:12:30.289 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:30.287+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:30.300 INFO:teuthology.orchestra.run.smithi157.stdout:1083 punch from 0x27b200 to 0x282000, (0x6e00 bytes) 2024-09-13T09:12:30.305 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:30.305 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:30.308 INFO:teuthology.orchestra.run.smithi157.stdout:1085 trunc from 0x59c8a00 to 0x7c0200 2024-09-13T09:12:30.319 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-13T09:12:30.341 INFO:teuthology.orchestra.run.smithi157.stdout:1086 trunc from 0x7c0200 to 0x2420200 2024-09-13T09:12:30.349 INFO:teuthology.orchestra.run.smithi157.stdout:1088 read 0x181d400 thru 0x182432c (0x6f2d bytes) 2024-09-13T09:12:30.355 INFO:teuthology.orchestra.run.smithi157.stdout:1090 read 0x87e600 thru 0x888230 (0x9c31 bytes) 2024-09-13T09:12:30.357 INFO:teuthology.orchestra.run.smithi157.stdout:1092 trunc from 0x2420200 to 0x12ec800 2024-09-13T09:12:30.368 INFO:teuthology.orchestra.run.smithi157.stdout:1093 read 0x11ac000 thru 0x11af3e0 (0x33e1 bytes) 2024-09-13T09:12:30.373 INFO:teuthology.orchestra.run.smithi157.stdout:1094 read 0x842800 thru 0x848402 (0x5c03 bytes) 2024-09-13T09:12:30.380 INFO:teuthology.orchestra.run.smithi157.stdout:1098 trunc from 0x12ec800 to 0x6544400 2024-09-13T09:12:30.388 INFO:teuthology.orchestra.run.smithi157.stdout:1101 clone 8 order 23 su 4194304 sc 8 2024-09-13T09:12:30.404 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:30.402+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:30.423 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:30.423 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:30.432 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:30.471 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:12:30.506 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-13T09:12:30.506 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:30.503+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:30.540 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:30.541 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:30.550 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:30.557 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone7 from 0x6544400 (overlap 0x7c0200) to 0x3e0000 2024-09-13T09:12:30.594 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:30.593+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:30.661 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:30.661 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:30.671 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:30.690 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-13T09:12:30.712 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:30.710+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:30.793 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:30.791+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:30.820 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:30.820 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:30.829 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:30.830 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-13T09:12:30.912 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-13T09:12:30.915 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:30.913+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:30.937 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:30.937 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:30.946 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:30.971 INFO:teuthology.orchestra.run.smithi157.stdout:1103 read 0x1f49a00 thru 0x1f53caf (0xa2b0 bytes) 2024-09-13T09:12:30.990 INFO:teuthology.orchestra.run.smithi157.stdout:1105 write 0xb764e00 thru 0xb768a18 (0x3c19 bytes) 2024-09-13T09:12:30.995 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:30.993+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:30.997 INFO:teuthology.orchestra.run.smithi157.stdout:1106 read 0xab40c00 thru 0xab4f0d7 (0xe4d8 bytes) 2024-09-13T09:12:30.997 INFO:teuthology.orchestra.run.smithi157.stdout:1107 punch from 0x6d68200 to 0x6d68e00, (0xc00 bytes) 2024-09-13T09:12:31.007 INFO:teuthology.orchestra.run.smithi157.stdout:1108 trunc from 0xb768c00 to 0x31f7a00 2024-09-13T09:12:31.030 INFO:teuthology.orchestra.run.smithi157.stdout:1109 read 0x222f200 thru 0x2233e3c (0x4c3d bytes) 2024-09-13T09:12:31.031 INFO:teuthology.orchestra.run.smithi157.stdout:1111 read 0x1cea000 thru 0x1cee9d0 (0x49d1 bytes) 2024-09-13T09:12:31.035 INFO:teuthology.orchestra.run.smithi157.stdout:1114 punch from 0x17e7200 to 0x17f2600, (0xb400 bytes) 2024-09-13T09:12:31.040 INFO:teuthology.orchestra.run.smithi157.stdout:1121 trunc from 0x31f7a00 to 0x6b67200 2024-09-13T09:12:31.048 INFO:teuthology.orchestra.run.smithi157.stdout:1122 write 0x4b6c00 thru 0x4b6ddb (0x1dc bytes) 2024-09-13T09:12:31.050 INFO:teuthology.orchestra.run.smithi157.stdout:1124 read 0x1bbac00 thru 0x1bc9891 (0xec92 bytes) 2024-09-13T09:12:31.051 INFO:teuthology.orchestra.run.smithi157.stdout:1126 read 0x6813400 thru 0x6822973 (0xf574 bytes) 2024-09-13T09:12:31.051 INFO:teuthology.orchestra.run.smithi157.stdout:1127 write 0x683e600 thru 0x6843806 (0x5207 bytes) 2024-09-13T09:12:31.051 INFO:teuthology.orchestra.run.smithi157.stdout:1128 punch from 0x632d600 to 0x632ec00, (0x1600 bytes) 2024-09-13T09:12:31.051 INFO:teuthology.orchestra.run.smithi157.stdout:1129 read 0x61b1a00 thru 0x61c0f3e (0xf53f bytes) 2024-09-13T09:12:31.060 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:31.060 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:31.069 INFO:teuthology.orchestra.run.smithi157.stdout:1130 write 0xe9a8800 thru 0xe9b33fc (0xabfd bytes) 2024-09-13T09:12:31.070 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:31.075 INFO:teuthology.orchestra.run.smithi157.stdout:1132 read 0xaf32600 thru 0xaf423eb (0xfdec bytes) 2024-09-13T09:12:31.075 INFO:teuthology.orchestra.run.smithi157.stdout:1134 read 0x9bd1800 thru 0x9bdc488 (0xac89 bytes) 2024-09-13T09:12:31.078 INFO:teuthology.orchestra.run.smithi157.stdout:1139 trunc from 0xe9b3400 to 0xc439c00 2024-09-13T09:12:31.094 INFO:teuthology.orchestra.run.smithi157.stdout:1140 punch from 0x8f61000 to 0x8f6a400, (0x9400 bytes) 2024-09-13T09:12:31.097 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:31.094+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:31.100 INFO:teuthology.orchestra.run.smithi157.stdout:1141 punch from 0x9741c00 to 0x9746400, (0x4800 bytes) 2024-09-13T09:12:31.100 INFO:teuthology.orchestra.run.smithi157.stdout:1142 write 0x7459e00 thru 0x745c99b (0x2b9c bytes) 2024-09-13T09:12:31.111 INFO:teuthology.orchestra.run.smithi157.stdout:1144 trunc from 0xc439c00 to 0x38c5c00 2024-09-13T09:12:31.129 INFO:teuthology.orchestra.run.smithi157.stdout:1146 write 0x263d400 thru 0x26426bc (0x52bd bytes) 2024-09-13T09:12:31.130 INFO:teuthology.orchestra.run.smithi157.stdout:1149 write 0x30f4e00 thru 0x30fa8b4 (0x5ab5 bytes) 2024-09-13T09:12:31.131 INFO:teuthology.orchestra.run.smithi157.stdout:1150 punch from 0x11c3400 to 0x11c9400, (0x6000 bytes) 2024-09-13T09:12:31.131 INFO:teuthology.orchestra.run.smithi157.stdout:1151 punch from 0x1b1ea00 to 0x1b28200, (0x9800 bytes) 2024-09-13T09:12:31.132 INFO:teuthology.orchestra.run.smithi157.stdout:1158 read 0xb66400 thru 0xb6eb2c (0x872d bytes) 2024-09-13T09:12:31.132 INFO:teuthology.orchestra.run.smithi157.stdout:1159 read 0x2a09600 thru 0x2a16600 (0xd001 bytes) 2024-09-13T09:12:31.132 INFO:teuthology.orchestra.run.smithi157.stdout:1160 read 0xf07c00 thru 0xf0b2d1 (0x36d2 bytes) 2024-09-13T09:12:31.133 INFO:teuthology.orchestra.run.smithi157.stdout:1161 read 0x118a000 thru 0x1198810 (0xe811 bytes) 2024-09-13T09:12:31.144 INFO:teuthology.orchestra.run.smithi157.stdout:1162 trunc from 0x38c5c00 to 0xc6a8200 2024-09-13T09:12:31.152 INFO:teuthology.orchestra.run.smithi157.stdout:1163 trunc from 0xc6a8200 to 0xcacd200 2024-09-13T09:12:31.175 INFO:teuthology.orchestra.run.smithi157.stdout:1164 trunc from 0xcacd200 to 0x1bcc400 2024-09-13T09:12:31.188 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:31.188 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:31.194 INFO:teuthology.orchestra.run.smithi157.stdout:1166 read 0x3c9000 thru 0x3d5bf1 (0xcbf2 bytes) 2024-09-13T09:12:31.198 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:31.199 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:31.196+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:31.210 INFO:teuthology.orchestra.run.smithi157.stdout:1168 trunc from 0x1bcc400 to 0x9914e00 2024-09-13T09:12:31.227 INFO:teuthology.orchestra.run.smithi157.stdout:1169 trunc from 0x9914e00 to 0x318de00 2024-09-13T09:12:31.237 INFO:teuthology.orchestra.run.smithi157.stdout:1170 read 0x3016800 thru 0x30212bc (0xaabd bytes) 2024-09-13T09:12:31.255 INFO:teuthology.orchestra.run.smithi157.stdout:1172 write 0xb13d200 thru 0xb14a2f8 (0xd0f9 bytes) 2024-09-13T09:12:31.260 INFO:teuthology.orchestra.run.smithi157.stdout:1173 write 0x9bf1400 thru 0x9bf8efd (0x7afe bytes) 2024-09-13T09:12:31.260 INFO:teuthology.orchestra.run.smithi157.stdout:1175 punch from 0x87f7a00 to 0x87fce00, (0x5400 bytes) 2024-09-13T09:12:31.261 INFO:teuthology.orchestra.run.smithi157.stdout:1176 read 0x6207c00 thru 0x620ee78 (0x7279 bytes) 2024-09-13T09:12:31.261 INFO:teuthology.orchestra.run.smithi157.stdout:1177 punch from 0x3006800 to 0x300d800, (0x7000 bytes) 2024-09-13T09:12:31.261 INFO:teuthology.orchestra.run.smithi157.stdout:1180 read 0x17e4000 thru 0x17ee47a (0xa47b bytes) 2024-09-13T09:12:31.265 INFO:teuthology.orchestra.run.smithi157.stdout:1184 punch from 0x4ea6a00 to 0x4eb0800, (0x9e00 bytes) 2024-09-13T09:12:31.265 INFO:teuthology.orchestra.run.smithi157.stdout:1189 clone 9 order 19 su 32768 sc 10 2024-09-13T09:12:31.298 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:31.296+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:31.309 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:31.310 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:31.318 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:31.403 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:31.401+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:31.465 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:31.465 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:31.475 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:31.476 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-09-13T09:12:31.500 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:31.498+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:31.562 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone8 from 0xb14a400 (overlap 0x1bcc400) to 0x163d000 2024-09-13T09:12:31.589 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:31.589 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:31.599 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:31.601 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:31.599+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:31.701 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:31.699+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:31.707 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:31.707 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:31.716 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:31.802 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:31.800+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:31.826 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:31.826 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:31.837 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:31.904 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:31.902+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:31.953 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-13T09:12:31.989 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:31.989 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:31.998 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:32.004 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:32.003+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:32.026 INFO:teuthology.orchestra.run.smithi157.stdout:1191 write 0x9858200 thru 0x98654e7 (0xd2e8 bytes) 2024-09-13T09:12:32.028 INFO:teuthology.orchestra.run.smithi157.stdout:1193 write 0x23b000 thru 0x23e007 (0x3008 bytes) 2024-09-13T09:12:32.028 INFO:teuthology.orchestra.run.smithi157.stdout:1196 punch from 0x6436a00 to 0x643ca00, (0x6000 bytes) 2024-09-13T09:12:32.045 INFO:teuthology.orchestra.run.smithi157.stdout:1199 write 0x21ad400 thru 0x21bafeb (0xdbec bytes) 2024-09-13T09:12:32.046 INFO:teuthology.orchestra.run.smithi157.stdout:1202 write 0x2dd9c00 thru 0x2dddd27 (0x4128 bytes) 2024-09-13T09:12:32.046 INFO:teuthology.orchestra.run.smithi157.stdout:1203 punch from 0x337fc00 to 0x338ce00, (0xd200 bytes) 2024-09-13T09:12:32.082 INFO:teuthology.orchestra.run.smithi157.stdout:1204 write 0xebbd600 thru 0xebc1a57 (0x4458 bytes) 2024-09-13T09:12:32.082 INFO:teuthology.orchestra.run.smithi157.stdout:1205 write 0xa951800 thru 0xa95c5f4 (0xadf5 bytes) 2024-09-13T09:12:32.083 INFO:teuthology.orchestra.run.smithi157.stdout:1206 read 0x9a02400 thru 0x9a053a2 (0x2fa3 bytes) 2024-09-13T09:12:32.090 INFO:teuthology.orchestra.run.smithi157.stdout:1207 trunc from 0xebc1c00 to 0x866f200 2024-09-13T09:12:32.105 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:32.103+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:32.116 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:32.116 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:32.126 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:32.137 INFO:teuthology.orchestra.run.smithi157.stdout:1209 flatten 2024-09-13T09:12:32.211 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:32.209+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:32.291 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:32.292 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:32.300 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:32.306 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:32.305+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:32.312 INFO:teuthology.orchestra.run.smithi157.stdout:1211 read 0x51a8000 thru 0x51b4b22 (0xcb23 bytes) 2024-09-13T09:12:32.312 INFO:teuthology.orchestra.run.smithi157.stdout:1216 read 0x3c47800 thru 0x3c4ad81 (0x3582 bytes) 2024-09-13T09:12:32.317 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:12:32.335 INFO:teuthology.orchestra.run.smithi157.stdout:1217 write 0xe136600 thru 0xe14205b (0xba5c bytes) 2024-09-13T09:12:32.335 INFO:teuthology.orchestra.run.smithi157.stdout:1218 punch from 0x7476800 to 0x747ee00, (0x8600 bytes) 2024-09-13T09:12:32.335 INFO:teuthology.orchestra.run.smithi157.stdout:1219 punch from 0x3101a00 to 0x310ce00, (0xb400 bytes) 2024-09-13T09:12:32.335 INFO:teuthology.orchestra.run.smithi157.stdout:1220 write 0x93bd400 thru 0x93c8fd8 (0xbbd9 bytes) 2024-09-13T09:12:32.335 INFO:teuthology.orchestra.run.smithi157.stdout:1222 read 0xb1e7a00 thru 0xb1ea4cd (0x2ace bytes) 2024-09-13T09:12:32.335 INFO:teuthology.orchestra.run.smithi157.stdout:1223 trunc from 0xe142200 to 0xeb63a00 2024-09-13T09:12:32.358 INFO:teuthology.orchestra.run.smithi157.stdout:1224 write 0xd46a600 thru 0xd473f87 (0x9988 bytes) 2024-09-13T09:12:32.363 INFO:teuthology.orchestra.run.smithi157.stdout:1225 trunc from 0xeb63a00 to 0x9e00a00 2024-09-13T09:12:32.387 INFO:teuthology.orchestra.run.smithi157.stdout:1226 read 0x5d64000 thru 0x5d697a2 (0x57a3 bytes) 2024-09-13T09:12:32.387 INFO:teuthology.orchestra.run.smithi157.stdout:1227 write 0x331a00 thru 0x337681 (0x5c82 bytes) 2024-09-13T09:12:32.387 INFO:teuthology.orchestra.run.smithi157.stdout:1228 write 0x7985600 thru 0x7985e64 (0x865 bytes) 2024-09-13T09:12:32.387 INFO:teuthology.orchestra.run.smithi157.stdout:1229 write 0x5f45400 thru 0x5f46ba3 (0x17a4 bytes) 2024-09-13T09:12:32.388 INFO:teuthology.orchestra.run.smithi157.stdout:1230 write 0x42c8600 thru 0x42d1f39 (0x993a bytes) 2024-09-13T09:12:32.388 INFO:teuthology.orchestra.run.smithi157.stdout:1231 read 0x13d3a00 thru 0x13de082 (0xa683 bytes) 2024-09-13T09:12:32.388 INFO:teuthology.orchestra.run.smithi157.stdout:1232 read 0x78e5c00 thru 0x78e86c2 (0x2ac3 bytes) 2024-09-13T09:12:32.407 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:32.405+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:32.414 INFO:teuthology.orchestra.run.smithi157.stdout:1233 write 0xd12e600 thru 0xd13b3d2 (0xcdd3 bytes) 2024-09-13T09:12:32.414 INFO:teuthology.orchestra.run.smithi157.stdout:1236 write 0xad22600 thru 0xad23349 (0xd4a bytes) 2024-09-13T09:12:32.434 INFO:teuthology.orchestra.run.smithi157.stdout:1237 write 0xd759600 thru 0xd7598da (0x2db bytes) 2024-09-13T09:12:32.435 INFO:teuthology.orchestra.run.smithi157.stdout:1240 punch from 0xb890400 to 0xb89c000, (0xbc00 bytes) 2024-09-13T09:12:32.435 INFO:teuthology.orchestra.run.smithi157.stdout:1241 write 0x3473c00 thru 0x34814df (0xd8e0 bytes) 2024-09-13T09:12:32.435 INFO:teuthology.orchestra.run.smithi157.stdout:1243 write 0x897e000 thru 0x89879b5 (0x99b6 bytes) 2024-09-13T09:12:32.449 INFO:teuthology.orchestra.run.smithi157.stdout:1244 trunc from 0xd759a00 to 0x1a6d000 2024-09-13T09:12:32.460 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:32.460 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:32.473 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:32.478 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:32.478 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:32.495 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:32.496 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:32.502 INFO:teuthology.orchestra.run.smithi157.stdout:1247 clone 10 order 22 su 16384 sc 6 2024-09-13T09:12:32.508 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:32.506+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:32.512 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:32.512 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:32.528 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:32.528 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:32.544 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:32.544 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:32.561 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:32.561 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:32.580 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:32.580 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:32.580 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:32.580 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:32.589 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:32.612 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:32.610+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:32.691 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:32.691 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:32.691 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:32.691 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:32.691 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:32.700 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:32.709 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:32.708+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:32.804 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:11:59.289661+0000", 2024-09-13T09:12:32.804 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 33.511708333999998, 2024-09-13T09:12:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.93386499199999995, 2024-09-13T09:12:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1 2024-09-13T09:12:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289661+0000", 2024-09-13T09:12:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289661+0000", 2024-09-13T09:12:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289659+0000", 2024-09-13T09:12:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:32.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289669+0000", 2024-09-13T09:12:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:12:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289681+0000", 2024-09-13T09:12:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2316e-05 2024-09-13T09:12:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289693+0000", 2024-09-13T09:12:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2549e-05 2024-09-13T09:12:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289744+0000", 2024-09-13T09:12:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0365999999999999e-05 2024-09-13T09:12:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289752+0000", 2024-09-13T09:12:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5590000000000006e-06 2024-09-13T09:12:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.218301+0000", 2024-09-13T09:12:32.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.928549085 2024-09-13T09:12:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.218447+0000", 2024-09-13T09:12:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014537400000000001 2024-09-13T09:12:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.218615+0000", 2024-09-13T09:12:32.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000168195 2024-09-13T09:12:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.219277+0000", 2024-09-13T09:12:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066202600000000002 2024-09-13T09:12:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.223461+0000", 2024-09-13T09:12:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004184152 2024-09-13T09:12:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.223500+0000", 2024-09-13T09:12:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8733000000000003e-05 2024-09-13T09:12:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.223526+0000", 2024-09-13T09:12:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5612000000000001e-05 2024-09-13T09:12:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:32.812 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.553991828999997, 2024-09-13T09:12:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079358310000000008, 2024-09-13T09:12:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35 2024-09-13T09:12:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:32.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:32.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:32.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247410+0000", 2024-09-13T09:12:32.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3053999999999999e-05 2024-09-13T09:12:32.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247420+0000", 2024-09-13T09:12:32.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7189999999999996e-06 2024-09-13T09:12:32.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247429+0000", 2024-09-13T09:12:32.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5340000000000005e-06 2024-09-13T09:12:32.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.249500+0000", 2024-09-13T09:12:32.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020708990000000002 2024-09-13T09:12:32.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.249551+0000", 2024-09-13T09:12:32.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0762e-05 2024-09-13T09:12:32.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.249649+0000", 2024-09-13T09:12:32.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.768e-05 2024-09-13T09:12:32.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.251260+0000", 2024-09-13T09:12:32.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001611101 2024-09-13T09:12:32.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255178+0000", 2024-09-13T09:12:32.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039180930000000001 2024-09-13T09:12:32.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255256+0000", 2024-09-13T09:12:32.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8572000000000006e-05 2024-09-13T09:12:32.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255313+0000", 2024-09-13T09:12:32.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6625000000000001e-05 2024-09-13T09:12:32.818 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.819 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.819 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.819 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:32.819 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:32.819 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.553934650999999, 2024-09-13T09:12:32.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079428840000000007, 2024-09-13T09:12:32.819 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.819 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.819 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.819 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.819 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.819 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36 2024-09-13T09:12:32.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.820 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:32.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:32.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:32.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:32.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247438+0000", 2024-09-13T09:12:32.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7409999999999998e-06 2024-09-13T09:12:32.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247442+0000", 2024-09-13T09:12:32.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:12:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247444+0000", 2024-09-13T09:12:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8880000000000001e-06 2024-09-13T09:12:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250381+0000", 2024-09-13T09:12:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0029366800000000001 2024-09-13T09:12:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250446+0000", 2024-09-13T09:12:32.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5173999999999996e-05 2024-09-13T09:12:32.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250546+0000", 2024-09-13T09:12:32.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100142 2024-09-13T09:12:32.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.251514+0000", 2024-09-13T09:12:32.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00096772000000000004 2024-09-13T09:12:32.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255336+0000", 2024-09-13T09:12:32.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038214719999999998 2024-09-13T09:12:32.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255358+0000", 2024-09-13T09:12:32.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2141e-05 2024-09-13T09:12:32.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255377+0000", 2024-09-13T09:12:32.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9398999999999998e-05 2024-09-13T09:12:32.825 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.825 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.825 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.825 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:32.825 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:32.825 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.553920048999998, 2024-09-13T09:12:32.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079696799999999998, 2024-09-13T09:12:32.825 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.825 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.826 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.826 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.826 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37 2024-09-13T09:12:32.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.826 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:32.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:32.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:32.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:32.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247451+0000", 2024-09-13T09:12:32.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6000000000000001e-06 2024-09-13T09:12:32.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247463+0000", 2024-09-13T09:12:32.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1609e-05 2024-09-13T09:12:32.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247473+0000", 2024-09-13T09:12:32.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0536999999999999e-05 2024-09-13T09:12:32.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250802+0000", 2024-09-13T09:12:32.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033287270000000001 2024-09-13T09:12:32.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250853+0000", 2024-09-13T09:12:32.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1329000000000003e-05 2024-09-13T09:12:32.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250935+0000", 2024-09-13T09:12:32.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1396999999999993e-05 2024-09-13T09:12:32.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.251796+0000", 2024-09-13T09:12:32.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00086075099999999999 2024-09-13T09:12:32.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255382+0000", 2024-09-13T09:12:32.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003586334 2024-09-13T09:12:32.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255402+0000", 2024-09-13T09:12:32.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0392000000000001e-05 2024-09-13T09:12:32.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255419+0000", 2024-09-13T09:12:32.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6229e-05 2024-09-13T09:12:32.831 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.831 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.831 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.831 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:32.832 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:32.832 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.525985955000003, 2024-09-13T09:12:32.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077389640000000001, 2024-09-13T09:12:32.832 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.832 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.832 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 56 2024-09-13T09:12:32.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.832 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:32.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:32.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:32.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:32.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275385+0000", 2024-09-13T09:12:32.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:12:32.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275388+0000", 2024-09-13T09:12:32.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:12:32.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275396+0000", 2024-09-13T09:12:32.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0099999999999995e-06 2024-09-13T09:12:32.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.276666+0000", 2024-09-13T09:12:32.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00126968 2024-09-13T09:12:32.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.276710+0000", 2024-09-13T09:12:32.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4141999999999999e-05 2024-09-13T09:12:32.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.276785+0000", 2024-09-13T09:12:32.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4777000000000006e-05 2024-09-13T09:12:32.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.282050+0000", 2024-09-13T09:12:32.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0052647320000000003 2024-09-13T09:12:32.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283071+0000", 2024-09-13T09:12:32.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001021196 2024-09-13T09:12:32.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283102+0000", 2024-09-13T09:12:32.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0849999999999998e-05 2024-09-13T09:12:32.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283122+0000", 2024-09-13T09:12:32.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0190000000000002e-05 2024-09-13T09:12:32.837 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.837 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.837 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.837 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:58 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:32.837 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:32.837 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.518002439999997, 2024-09-13T09:12:32.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064599870000000004, 2024-09-13T09:12:32.838 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.838 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.838 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.838 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.838 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.838 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 58 2024-09-13T09:12:32.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.838 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:32.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:32.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:32.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:32.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283398+0000", 2024-09-13T09:12:32.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1646000000000002e-05 2024-09-13T09:12:32.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283403+0000", 2024-09-13T09:12:32.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0540000000000002e-06 2024-09-13T09:12:32.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283406+0000", 2024-09-13T09:12:32.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3110000000000001e-06 2024-09-13T09:12:32.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283841+0000", 2024-09-13T09:12:32.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000434549 2024-09-13T09:12:32.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283900+0000", 2024-09-13T09:12:32.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9153000000000001e-05 2024-09-13T09:12:32.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283975+0000", 2024-09-13T09:12:32.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4967000000000002e-05 2024-09-13T09:12:32.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.285844+0000", 2024-09-13T09:12:32.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001869435 2024-09-13T09:12:32.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289769+0000", 2024-09-13T09:12:32.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039245230000000001 2024-09-13T09:12:32.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289804+0000", 2024-09-13T09:12:32.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4999000000000002e-05 2024-09-13T09:12:32.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289826+0000", 2024-09-13T09:12:32.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2857000000000002e-05 2024-09-13T09:12:32.843 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.843 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.844 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:59 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:32.844 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.283413+0000", 2024-09-13T09:12:32.844 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.517956095000002, 2024-09-13T09:12:32.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064987589999999998, 2024-09-13T09:12:32.844 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.844 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.845 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 59 2024-09-13T09:12:32.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.845 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283413+0000", 2024-09-13T09:12:32.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283413+0000", 2024-09-13T09:12:32.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283412+0000", 2024-09-13T09:12:32.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:32.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283416+0000", 2024-09-13T09:12:32.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:12:32.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283420+0000", 2024-09-13T09:12:32.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6789999999999996e-06 2024-09-13T09:12:32.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283423+0000", 2024-09-13T09:12:32.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.672e-06 2024-09-13T09:12:32.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.284235+0000", 2024-09-13T09:12:32.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00081158900000000004 2024-09-13T09:12:32.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.284282+0000", 2024-09-13T09:12:32.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7213999999999999e-05 2024-09-13T09:12:32.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.284359+0000", 2024-09-13T09:12:32.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6814e-05 2024-09-13T09:12:32.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.285868+0000", 2024-09-13T09:12:32.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015097310000000001 2024-09-13T09:12:32.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289836+0000", 2024-09-13T09:12:32.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003967613 2024-09-13T09:12:32.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289877+0000", 2024-09-13T09:12:32.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1519e-05 2024-09-13T09:12:32.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289912+0000", 2024-09-13T09:12:32.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4088999999999998e-05 2024-09-13T09:12:32.850 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.850 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.850 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.850 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:32.850 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.298116+0000", 2024-09-13T09:12:32.850 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.503253291, 2024-09-13T09:12:32.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063667139999999999, 2024-09-13T09:12:32.851 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.851 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.851 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 69 2024-09-13T09:12:32.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.851 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298116+0000", 2024-09-13T09:12:32.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298116+0000", 2024-09-13T09:12:32.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298115+0000", 2024-09-13T09:12:32.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:32.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298193+0000", 2024-09-13T09:12:32.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7441999999999995e-05 2024-09-13T09:12:32.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298213+0000", 2024-09-13T09:12:32.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0829999999999999e-05 2024-09-13T09:12:32.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298224+0000", 2024-09-13T09:12:32.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0282000000000001e-05 2024-09-13T09:12:32.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299016+0000", 2024-09-13T09:12:32.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00079248999999999997 2024-09-13T09:12:32.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299098+0000", 2024-09-13T09:12:32.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1607999999999995e-05 2024-09-13T09:12:32.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299223+0000", 2024-09-13T09:12:32.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000124838 2024-09-13T09:12:32.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.300536+0000", 2024-09-13T09:12:32.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001312843 2024-09-13T09:12:32.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304428+0000", 2024-09-13T09:12:32.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038928000000000001 2024-09-13T09:12:32.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304463+0000", 2024-09-13T09:12:32.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4214999999999997e-05 2024-09-13T09:12:32.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304482+0000", 2024-09-13T09:12:32.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9800999999999999e-05 2024-09-13T09:12:32.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.856 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.856 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:32.856 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.298237+0000", 2024-09-13T09:12:32.857 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.503132323000003, 2024-09-13T09:12:32.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063589789999999998, 2024-09-13T09:12:32.857 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.857 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.857 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.860 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 70 2024-09-13T09:12:32.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.860 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298237+0000", 2024-09-13T09:12:32.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298237+0000", 2024-09-13T09:12:32.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298236+0000", 2024-09-13T09:12:32.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:32.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298242+0000", 2024-09-13T09:12:32.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5889999999999999e-06 2024-09-13T09:12:32.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298252+0000", 2024-09-13T09:12:32.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6539999999999994e-06 2024-09-13T09:12:32.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298257+0000", 2024-09-13T09:12:32.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4240000000000001e-06 2024-09-13T09:12:32.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299735+0000", 2024-09-13T09:12:32.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0014772120000000001 2024-09-13T09:12:32.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299851+0000", 2024-09-13T09:12:32.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000115998 2024-09-13T09:12:32.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299975+0000", 2024-09-13T09:12:32.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000124805 2024-09-13T09:12:32.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.300830+0000", 2024-09-13T09:12:32.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0008543 2024-09-13T09:12:32.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304519+0000", 2024-09-13T09:12:32.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036896849999999998 2024-09-13T09:12:32.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304560+0000", 2024-09-13T09:12:32.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1009e-05 2024-09-13T09:12:32.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304596+0000", 2024-09-13T09:12:32.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5114999999999998e-05 2024-09-13T09:12:32.865 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.865 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.865 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.866 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:85 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:32.866 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:32.866 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.489181922999997, 2024-09-13T09:12:32.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068319399999999999, 2024-09-13T09:12:32.866 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.866 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.866 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.866 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.866 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.866 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 85 2024-09-13T09:12:32.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.866 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:32.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:32.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:32.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:32.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312190+0000", 2024-09-13T09:12:32.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2389999999999998e-06 2024-09-13T09:12:32.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312198+0000", 2024-09-13T09:12:32.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8599999999999993e-06 2024-09-13T09:12:32.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312201+0000", 2024-09-13T09:12:32.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.399e-06 2024-09-13T09:12:32.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.313614+0000", 2024-09-13T09:12:32.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001413369 2024-09-13T09:12:32.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.313702+0000", 2024-09-13T09:12:32.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7556e-05 2024-09-13T09:12:32.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.313843+0000", 2024-09-13T09:12:32.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000141015 2024-09-13T09:12:32.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318968+0000", 2024-09-13T09:12:32.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0051251680000000003 2024-09-13T09:12:32.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318973+0000", 2024-09-13T09:12:32.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9520000000000002e-06 2024-09-13T09:12:32.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318999+0000", 2024-09-13T09:12:32.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5706000000000001e-05 2024-09-13T09:12:32.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319019+0000", 2024-09-13T09:12:32.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0024e-05 2024-09-13T09:12:32.871 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.871 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.871 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.871 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:32.871 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:32.872 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.489163902999998, 2024-09-13T09:12:32.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069390709999999998, 2024-09-13T09:12:32.872 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.872 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.872 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 86 2024-09-13T09:12:32.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.872 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:32.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:32.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:32.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:32.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312207+0000", 2024-09-13T09:12:32.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.243e-06 2024-09-13T09:12:32.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312211+0000", 2024-09-13T09:12:32.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1219999999999997e-06 2024-09-13T09:12:32.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312215+0000", 2024-09-13T09:12:32.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4680000000000001e-06 2024-09-13T09:12:32.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314378+0000", 2024-09-13T09:12:32.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021634309999999999 2024-09-13T09:12:32.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314426+0000", 2024-09-13T09:12:32.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7963000000000001e-05 2024-09-13T09:12:32.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314507+0000", 2024-09-13T09:12:32.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0619999999999997e-05 2024-09-13T09:12:32.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319024+0000", 2024-09-13T09:12:32.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045178140000000002 2024-09-13T09:12:32.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319065+0000", 2024-09-13T09:12:32.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0216999999999998e-05 2024-09-13T09:12:32.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319108+0000", 2024-09-13T09:12:32.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3618000000000001e-05 2024-09-13T09:12:32.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319144+0000", 2024-09-13T09:12:32.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5858999999999997e-05 2024-09-13T09:12:32.877 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.877 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.877 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.877 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:32.878 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:32.878 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.489149525000002, 2024-09-13T09:12:32.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011377922, 2024-09-13T09:12:32.878 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.878 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.878 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 87 2024-09-13T09:12:32.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.878 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:32.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:32.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:32.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:32.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312249+0000", 2024-09-13T09:12:32.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9377e-05 2024-09-13T09:12:32.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312255+0000", 2024-09-13T09:12:32.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0220000000000001e-06 2024-09-13T09:12:32.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312259+0000", 2024-09-13T09:12:32.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-13T09:12:32.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314845+0000", 2024-09-13T09:12:32.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0025858750000000001 2024-09-13T09:12:32.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314891+0000", 2024-09-13T09:12:32.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6347999999999997e-05 2024-09-13T09:12:32.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314968+0000", 2024-09-13T09:12:32.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7428999999999993e-05 2024-09-13T09:12:32.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319174+0000", 2024-09-13T09:12:32.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042056999999999997 2024-09-13T09:12:32.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323544+0000", 2024-09-13T09:12:32.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043701119999999998 2024-09-13T09:12:32.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323575+0000", 2024-09-13T09:12:32.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1285000000000003e-05 2024-09-13T09:12:32.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323597+0000", 2024-09-13T09:12:32.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1891000000000001e-05 2024-09-13T09:12:32.883 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.883 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.883 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.883 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:32.884 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:32.884 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.489106767999999, 2024-09-13T09:12:32.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011622512, 2024-09-13T09:12:32.884 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.884 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.884 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.884 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.884 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.884 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 88 2024-09-13T09:12:32.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.884 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:32.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:32.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:32.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:32.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312265+0000", 2024-09-13T09:12:32.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8380000000000002e-06 2024-09-13T09:12:32.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312268+0000", 2024-09-13T09:12:32.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2310000000000001e-06 2024-09-13T09:12:32.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312271+0000", 2024-09-13T09:12:32.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.551e-06 2024-09-13T09:12:32.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318458+0000", 2024-09-13T09:12:32.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061866710000000004 2024-09-13T09:12:32.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318540+0000", 2024-09-13T09:12:32.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1845e-05 2024-09-13T09:12:32.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318668+0000", 2024-09-13T09:12:32.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012830900000000001 2024-09-13T09:12:32.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319598+0000", 2024-09-13T09:12:32.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000929377 2024-09-13T09:12:32.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323840+0000", 2024-09-13T09:12:32.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042420180000000002 2024-09-13T09:12:32.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323866+0000", 2024-09-13T09:12:32.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6262999999999999e-05 2024-09-13T09:12:32.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323885+0000", 2024-09-13T09:12:32.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8675e-05 2024-09-13T09:12:32.889 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.889 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.889 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.890 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:32.890 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:32.890 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.328728773999998, 2024-09-13T09:12:32.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0062525000000000002, 2024-09-13T09:12:32.890 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.890 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.890 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.890 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.890 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.890 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113 2024-09-13T09:12:32.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.890 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:32.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:32.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:32.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:32.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472644+0000", 2024-09-13T09:12:32.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-13T09:12:32.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472650+0000", 2024-09-13T09:12:32.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:12:32.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472655+0000", 2024-09-13T09:12:32.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:12:32.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477258+0000", 2024-09-13T09:12:32.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004603762 2024-09-13T09:12:32.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477303+0000", 2024-09-13T09:12:32.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5031999999999998e-05 2024-09-13T09:12:32.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477375+0000", 2024-09-13T09:12:32.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1186000000000006e-05 2024-09-13T09:12:32.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478772+0000", 2024-09-13T09:12:32.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00139704 2024-09-13T09:12:32.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478817+0000", 2024-09-13T09:12:32.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5120999999999997e-05 2024-09-13T09:12:32.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478859+0000", 2024-09-13T09:12:32.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2567000000000001e-05 2024-09-13T09:12:32.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478893+0000", 2024-09-13T09:12:32.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3210000000000002e-05 2024-09-13T09:12:32.895 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.895 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.895 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.896 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:32.896 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:32.896 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.328705466000002, 2024-09-13T09:12:32.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063268899999999999, 2024-09-13T09:12:32.896 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.896 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.896 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 114 2024-09-13T09:12:32.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.896 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:32.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:32.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:32.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:32.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472667+0000", 2024-09-13T09:12:32.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9219999999999999e-06 2024-09-13T09:12:32.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472673+0000", 2024-09-13T09:12:32.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.63e-06 2024-09-13T09:12:32.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472678+0000", 2024-09-13T09:12:32.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6689999999999997e-06 2024-09-13T09:12:32.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477608+0000", 2024-09-13T09:12:32.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0049302870000000002 2024-09-13T09:12:32.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477685+0000", 2024-09-13T09:12:32.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6922000000000007e-05 2024-09-13T09:12:32.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477804+0000", 2024-09-13T09:12:32.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011865499999999999 2024-09-13T09:12:32.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478935+0000", 2024-09-13T09:12:32.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0011305099999999999 2024-09-13T09:12:32.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478942+0000", 2024-09-13T09:12:32.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3969999999999997e-06 2024-09-13T09:12:32.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478970+0000", 2024-09-13T09:12:32.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8447000000000001e-05 2024-09-13T09:12:32.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478990+0000", 2024-09-13T09:12:32.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9933e-05 2024-09-13T09:12:32.901 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.901 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.901 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.901 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:32.902 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472870+0000", 2024-09-13T09:12:32.902 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.328499049000001, 2024-09-13T09:12:32.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068457680000000003, 2024-09-13T09:12:32.902 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.902 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.902 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.902 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.902 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.902 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 115 2024-09-13T09:12:32.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.902 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472870+0000", 2024-09-13T09:12:32.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472870+0000", 2024-09-13T09:12:32.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472869+0000", 2024-09-13T09:12:32.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:32.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472933+0000", 2024-09-13T09:12:32.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3448000000000006e-05 2024-09-13T09:12:32.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472943+0000", 2024-09-13T09:12:32.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0234999999999999e-05 2024-09-13T09:12:32.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472948+0000", 2024-09-13T09:12:32.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:12:32.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478311+0000", 2024-09-13T09:12:32.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053631759999999999 2024-09-13T09:12:32.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478364+0000", 2024-09-13T09:12:32.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3174000000000003e-05 2024-09-13T09:12:32.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478441+0000", 2024-09-13T09:12:32.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6211000000000006e-05 2024-09-13T09:12:32.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479649+0000", 2024-09-13T09:12:32.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001208319 2024-09-13T09:12:32.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479673+0000", 2024-09-13T09:12:32.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4060999999999999e-05 2024-09-13T09:12:32.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479697+0000", 2024-09-13T09:12:32.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3958999999999999e-05 2024-09-13T09:12:32.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479716+0000", 2024-09-13T09:12:32.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8661e-05 2024-09-13T09:12:32.907 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.907 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.907 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.907 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:32.907 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472955+0000", 2024-09-13T09:12:32.908 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.328413994000002, 2024-09-13T09:12:32.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069091580000000003, 2024-09-13T09:12:32.908 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.908 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.908 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 116 2024-09-13T09:12:32.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.908 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472955+0000", 2024-09-13T09:12:32.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472955+0000", 2024-09-13T09:12:32.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472954+0000", 2024-09-13T09:12:32.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:32.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472959+0000", 2024-09-13T09:12:32.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1309999999999999e-06 2024-09-13T09:12:32.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472964+0000", 2024-09-13T09:12:32.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6659999999999996e-06 2024-09-13T09:12:32.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472968+0000", 2024-09-13T09:12:32.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.495e-06 2024-09-13T09:12:32.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479021+0000", 2024-09-13T09:12:32.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060531990000000004 2024-09-13T09:12:32.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479062+0000", 2024-09-13T09:12:32.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1436000000000001e-05 2024-09-13T09:12:32.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479131+0000", 2024-09-13T09:12:32.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8533000000000003e-05 2024-09-13T09:12:32.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479727+0000", 2024-09-13T09:12:32.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00059612 2024-09-13T09:12:32.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479827+0000", 2024-09-13T09:12:32.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9940999999999997e-05 2024-09-13T09:12:32.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479848+0000", 2024-09-13T09:12:32.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0877999999999999e-05 2024-09-13T09:12:32.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479864+0000", 2024-09-13T09:12:32.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6175999999999999e-05 2024-09-13T09:12:32.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.913 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.913 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:32.913 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472991+0000", 2024-09-13T09:12:32.913 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.328378387999997, 2024-09-13T09:12:32.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071846899999999997, 2024-09-13T09:12:32.914 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.914 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.914 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 117 2024-09-13T09:12:32.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.914 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472991+0000", 2024-09-13T09:12:32.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472991+0000", 2024-09-13T09:12:32.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472990+0000", 2024-09-13T09:12:32.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:32.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472994+0000", 2024-09-13T09:12:32.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4870000000000004e-06 2024-09-13T09:12:32.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.473001+0000", 2024-09-13T09:12:32.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:12:32.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.473005+0000", 2024-09-13T09:12:32.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-13T09:12:32.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479356+0000", 2024-09-13T09:12:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063509869999999998 2024-09-13T09:12:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479407+0000", 2024-09-13T09:12:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0349999999999997e-05 2024-09-13T09:12:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479488+0000", 2024-09-13T09:12:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0922999999999998e-05 2024-09-13T09:12:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479942+0000", 2024-09-13T09:12:32.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045411799999999999 2024-09-13T09:12:32.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.480136+0000", 2024-09-13T09:12:32.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000193909 2024-09-13T09:12:32.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.480158+0000", 2024-09-13T09:12:32.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2427999999999999e-05 2024-09-13T09:12:32.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.480175+0000", 2024-09-13T09:12:32.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7071e-05 2024-09-13T09:12:32.919 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.919 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.919 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.919 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:32.919 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:32.919 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.949030673999999, 2024-09-13T09:12:32.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066097600000000001, 2024-09-13T09:12:32.919 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.919 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.920 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.920 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.920 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.920 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 140 2024-09-13T09:12:32.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.920 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:32.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:32.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:32.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:32.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852387+0000", 2024-09-13T09:12:32.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9191999999999999e-05 2024-09-13T09:12:32.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852392+0000", 2024-09-13T09:12:32.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7299999999999996e-06 2024-09-13T09:12:32.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852395+0000", 2024-09-13T09:12:32.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9979999999999999e-06 2024-09-13T09:12:32.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857443+0000", 2024-09-13T09:12:32.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005048075 2024-09-13T09:12:32.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857503+0000", 2024-09-13T09:12:32.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9577999999999998e-05 2024-09-13T09:12:32.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857571+0000", 2024-09-13T09:12:32.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8320999999999999e-05 2024-09-13T09:12:32.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858400+0000", 2024-09-13T09:12:32.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00082945100000000004 2024-09-13T09:12:32.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858910+0000", 2024-09-13T09:12:32.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050916999999999998 2024-09-13T09:12:32.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858931+0000", 2024-09-13T09:12:32.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1742e-05 2024-09-13T09:12:32.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858948+0000", 2024-09-13T09:12:32.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6714999999999998e-05 2024-09-13T09:12:32.925 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.925 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.925 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.925 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:32.925 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.852399+0000", 2024-09-13T09:12:32.925 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.948970363000001, 2024-09-13T09:12:32.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068470969999999999, 2024-09-13T09:12:32.925 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:32.925 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:32.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:32.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:32.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:32.926 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 141 2024-09-13T09:12:32.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.926 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:32.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:32.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852399+0000", 2024-09-13T09:12:32.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:32.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852399+0000", 2024-09-13T09:12:32.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:32.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:32.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852398+0000", 2024-09-13T09:12:32.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:32.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:32.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852400+0000", 2024-09-13T09:12:32.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:12:32.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:32.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852404+0000", 2024-09-13T09:12:32.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:12:32.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:32.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852406+0000", 2024-09-13T09:12:32.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4559999999999999e-06 2024-09-13T09:12:32.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:32.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857928+0000", 2024-09-13T09:12:32.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005521593 2024-09-13T09:12:32.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:32.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857968+0000", 2024-09-13T09:12:32.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0586000000000001e-05 2024-09-13T09:12:32.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:32.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858029+0000", 2024-09-13T09:12:32.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1140000000000001e-05 2024-09-13T09:12:32.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:32.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859187+0000", 2024-09-13T09:12:32.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001157969 2024-09-13T09:12:32.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:32.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859198+0000", 2024-09-13T09:12:32.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1028e-05 2024-09-13T09:12:32.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:32.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859227+0000", 2024-09-13T09:12:32.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8370000000000001e-05 2024-09-13T09:12:32.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:32.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:32.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:32.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859246+0000", 2024-09-13T09:12:32.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8961e-05 2024-09-13T09:12:32.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.930 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:32.931 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:32.931 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:32.932 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:32.933 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:32.931+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:33.033 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:33.031+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:33.035 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:33.035 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:33.035 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:33.036 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:33.044 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:33.133 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:33.131+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:33.146 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:33.146 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:33.146 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:33.146 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:33.146 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:33.154 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:33.238 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:33.237+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:33.298 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:12:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:16.378424+0000", 2024-09-13T09:12:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 16.916976896000001, 2024-09-13T09:12:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061107690000000003, 2024-09-13T09:12:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:12:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:12:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9 2024-09-13T09:12:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378424+0000", 2024-09-13T09:12:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378424+0000", 2024-09-13T09:12:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378422+0000", 2024-09-13T09:12:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378429+0000", 2024-09-13T09:12:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6100000000000002e-06 2024-09-13T09:12:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378441+0000", 2024-09-13T09:12:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1581e-05 2024-09-13T09:12:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378449+0000", 2024-09-13T09:12:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2679999999999994e-06 2024-09-13T09:12:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378496+0000", 2024-09-13T09:12:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7286000000000001e-05 2024-09-13T09:12:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378501+0000", 2024-09-13T09:12:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0980000000000001e-06 2024-09-13T09:12:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.380081+0000", 2024-09-13T09:12:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015801879999999999 2024-09-13T09:12:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.380177+0000", 2024-09-13T09:12:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5311999999999998e-05 2024-09-13T09:12:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.380299+0000", 2024-09-13T09:12:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000121921 2024-09-13T09:12:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.381082+0000", 2024-09-13T09:12:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00078348100000000004 2024-09-13T09:12:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384464+0000", 2024-09-13T09:12:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033814349999999999 2024-09-13T09:12:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384495+0000", 2024-09-13T09:12:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1554999999999999e-05 2024-09-13T09:12:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384534+0000", 2024-09-13T09:12:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9320999999999998e-05 2024-09-13T09:12:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1499 3.11 3:8ebbb892:::rbd_data.11238d3f93be.0000000000000036:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.136984+0000", 2024-09-13T09:12:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.158417047, 2024-09-13T09:12:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064184419999999999, 2024-09-13T09:12:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1499 2024-09-13T09:12:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.136984+0000", 2024-09-13T09:12:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.136984+0000", 2024-09-13T09:12:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.136983+0000", 2024-09-13T09:12:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.136987+0000", 2024-09-13T09:12:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9659999999999998e-06 2024-09-13T09:12:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.137001+0000", 2024-09-13T09:12:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3832000000000001e-05 2024-09-13T09:12:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.137007+0000", 2024-09-13T09:12:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8549999999999998e-06 2024-09-13T09:12:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.137060+0000", 2024-09-13T09:12:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3171999999999998e-05 2024-09-13T09:12:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.137141+0000", 2024-09-13T09:12:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1116000000000001e-05 2024-09-13T09:12:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.137246+0000", 2024-09-13T09:12:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010518099999999999 2024-09-13T09:12:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.138787+0000", 2024-09-13T09:12:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015407789999999999 2024-09-13T09:12:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.143355+0000", 2024-09-13T09:12:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045682359999999998 2024-09-13T09:12:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.143381+0000", 2024-09-13T09:12:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5585999999999999e-05 2024-09-13T09:12:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.143402+0000", 2024-09-13T09:12:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1154000000000001e-05 2024-09-13T09:12:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4483.0:291 3.3 3:c3e921d9:::rbd_data.1123ffac6bc2.00000000000000ed:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.778379+0000", 2024-09-13T09:12:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.517021513, 2024-09-13T09:12:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0048318249999999997, 2024-09-13T09:12:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4483", 2024-09-13T09:12:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 291 2024-09-13T09:12:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778379+0000", 2024-09-13T09:12:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778379+0000", 2024-09-13T09:12:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778379+0000", 2024-09-13T09:12:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778382+0000", 2024-09-13T09:12:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1709999999999998e-06 2024-09-13T09:12:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778388+0000", 2024-09-13T09:12:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6529999999999997e-06 2024-09-13T09:12:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778394+0000", 2024-09-13T09:12:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3499999999999996e-06 2024-09-13T09:12:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778444+0000", 2024-09-13T09:12:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0303000000000003e-05 2024-09-13T09:12:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778524+0000", 2024-09-13T09:12:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0187999999999999e-05 2024-09-13T09:12:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.778589+0000", 2024-09-13T09:12:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4488999999999999e-05 2024-09-13T09:12:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.779159+0000", 2024-09-13T09:12:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00057035500000000004 2024-09-13T09:12:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.783165+0000", 2024-09-13T09:12:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0040055769999999997 2024-09-13T09:12:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.783189+0000", 2024-09-13T09:12:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4573999999999999e-05 2024-09-13T09:12:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.783211+0000", 2024-09-13T09:12:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1715000000000001e-05 2024-09-13T09:12:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4483.0:1054 3.3 3:c48fd897:::rbd_data.1123ffac6bc2.00000000000000a1:head [write 159744~27648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:20.140196+0000", 2024-09-13T09:12:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.155204841, 2024-09-13T09:12:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0047523959999999999, 2024-09-13T09:12:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4483", 2024-09-13T09:12:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1054 2024-09-13T09:12:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140196+0000", 2024-09-13T09:12:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140196+0000", 2024-09-13T09:12:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140195+0000", 2024-09-13T09:12:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140216+0000", 2024-09-13T09:12:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0672000000000001e-05 2024-09-13T09:12:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140223+0000", 2024-09-13T09:12:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5050000000000001e-06 2024-09-13T09:12:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140228+0000", 2024-09-13T09:12:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4950000000000002e-06 2024-09-13T09:12:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140259+0000", 2024-09-13T09:12:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0964999999999999e-05 2024-09-13T09:12:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140304+0000", 2024-09-13T09:12:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4922000000000001e-05 2024-09-13T09:12:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.140377+0000", 2024-09-13T09:12:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3819000000000006e-05 2024-09-13T09:12:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.143410+0000", 2024-09-13T09:12:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0030321290000000002 2024-09-13T09:12:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.144868+0000", 2024-09-13T09:12:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001458097 2024-09-13T09:12:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.144914+0000", 2024-09-13T09:12:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6715000000000003e-05 2024-09-13T09:12:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.144948+0000", 2024-09-13T09:12:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3760000000000002e-05 2024-09-13T09:12:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4483.0:1320 3.14 3:2b7a8de5:::rbd_data.1123ffac6bc2.00000000000000a9:head [set-alloc-hint object_size 1048576 write_size 1048576,write 790528~61440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:12:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:20.271531+0000", 2024-09-13T09:12:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.023869400000001, 2024-09-13T09:12:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0049277319999999998, 2024-09-13T09:12:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4483", 2024-09-13T09:12:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1320 2024-09-13T09:12:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271531+0000", 2024-09-13T09:12:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271531+0000", 2024-09-13T09:12:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271530+0000", 2024-09-13T09:12:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271586+0000", 2024-09-13T09:12:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6017999999999998e-05 2024-09-13T09:12:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271596+0000", 2024-09-13T09:12:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8949999999999993e-06 2024-09-13T09:12:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271601+0000", 2024-09-13T09:12:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9760000000000003e-06 2024-09-13T09:12:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271643+0000", 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1934000000000002e-05 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271776+0000", 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013290599999999999 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.271911+0000", 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000134566 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.276207+0000", 2024-09-13T09:12:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042967819999999999 2024-09-13T09:12:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.276367+0000", 2024-09-13T09:12:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00015928200000000001 2024-09-13T09:12:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.276419+0000", 2024-09-13T09:12:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2580000000000001e-05 2024-09-13T09:12:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:20.276459+0000", 2024-09-13T09:12:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9573999999999998e-05 2024-09-13T09:12:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:445 3.14 3:2dfb6081:::rbd_data.11238839e366.0000000000000001:6 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.283411+0000", 2024-09-13T09:12:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.0119898260000006, 2024-09-13T09:12:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0044893329999999999, 2024-09-13T09:12:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 445 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283411+0000", 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283411+0000", 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283410+0000", 2024-09-13T09:12:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283413+0000", 2024-09-13T09:12:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0800000000000002e-06 2024-09-13T09:12:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283419+0000", 2024-09-13T09:12:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9340000000000003e-06 2024-09-13T09:12:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283424+0000", 2024-09-13T09:12:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5600000000000004e-06 2024-09-13T09:12:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287773+0000", 2024-09-13T09:12:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004349571 2024-09-13T09:12:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287840+0000", 2024-09-13T09:12:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6424e-05 2024-09-13T09:12:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.287900+0000", 2024-09-13T09:12:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0340000000000002e-05 2024-09-13T09:12:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:490 3.9 3:901933f8:::rbd_data.11238839e366.0000000000000005:6 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.283739+0000", 2024-09-13T09:12:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.0116617380000008, 2024-09-13T09:12:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045407370000000004, 2024-09-13T09:12:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 490 2024-09-13T09:12:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283739+0000", 2024-09-13T09:12:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283739+0000", 2024-09-13T09:12:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283739+0000", 2024-09-13T09:12:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283740+0000", 2024-09-13T09:12:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8980000000000001e-06 2024-09-13T09:12:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283746+0000", 2024-09-13T09:12:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3970000000000002e-06 2024-09-13T09:12:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283750+0000", 2024-09-13T09:12:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9040000000000002e-06 2024-09-13T09:12:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.288085+0000", 2024-09-13T09:12:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043348659999999997 2024-09-13T09:12:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.288171+0000", 2024-09-13T09:12:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6200999999999997e-05 2024-09-13T09:12:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.288280+0000", 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000108795 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:491 3.9 3:901933f8:::rbd_data.11238839e366.0000000000000005:6 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.283754+0000", 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.0116464690000004, 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004723054, 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 491 2024-09-13T09:12:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283754+0000", 2024-09-13T09:12:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283754+0000", 2024-09-13T09:12:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283754+0000", 2024-09-13T09:12:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283756+0000", 2024-09-13T09:12:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.905e-06 2024-09-13T09:12:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283761+0000", 2024-09-13T09:12:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.383e-06 2024-09-13T09:12:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.283765+0000", 2024-09-13T09:12:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1829999999999996e-06 2024-09-13T09:12:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.288303+0000", 2024-09-13T09:12:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045381680000000004 2024-09-13T09:12:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.288384+0000", 2024-09-13T09:12:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0315999999999995e-05 2024-09-13T09:12:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.288477+0000", 2024-09-13T09:12:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3405000000000004e-05 2024-09-13T09:12:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:759 3.11 3:88c20a21:::rbd_data.1123b8b4a28a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.960872+0000", 2024-09-13T09:12:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.3345285479999998, 2024-09-13T09:12:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0047897319999999997, 2024-09-13T09:12:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 759 2024-09-13T09:12:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.960872+0000", 2024-09-13T09:12:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.960872+0000", 2024-09-13T09:12:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.960871+0000", 2024-09-13T09:12:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.960877+0000", 2024-09-13T09:12:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9780000000000002e-06 2024-09-13T09:12:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.960891+0000", 2024-09-13T09:12:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4059999999999999e-05 2024-09-13T09:12:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.960899+0000", 2024-09-13T09:12:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.276e-06 2024-09-13T09:12:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.960933+0000", 2024-09-13T09:12:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4107999999999999e-05 2024-09-13T09:12:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.961022+0000", 2024-09-13T09:12:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9028000000000002e-05 2024-09-13T09:12:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.961145+0000", 2024-09-13T09:12:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012239600000000001 2024-09-13T09:12:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.961577+0000", 2024-09-13T09:12:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043195199999999999 2024-09-13T09:12:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.965611+0000", 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0040338370000000002 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.965642+0000", 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1847999999999997e-05 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.965662+0000", 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9313999999999998e-05 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.346 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.347 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:1170 3.9 3:93ffaa8c:::rbd_data.1123b8b4a28a.0000000000000041:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:33.347 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.152303+0000", 2024-09-13T09:12:33.347 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.1430977250000005, 2024-09-13T09:12:33.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0050248350000000001, 2024-09-13T09:12:33.347 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.347 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.347 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.347 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:33.347 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:33.347 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1170 2024-09-13T09:12:33.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.347 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.152303+0000", 2024-09-13T09:12:33.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.152303+0000", 2024-09-13T09:12:33.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.152301+0000", 2024-09-13T09:12:33.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:33.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.152308+0000", 2024-09-13T09:12:33.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9909999999999996e-06 2024-09-13T09:12:33.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.152323+0000", 2024-09-13T09:12:33.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5075999999999999e-05 2024-09-13T09:12:33.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.152330+0000", 2024-09-13T09:12:33.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1380000000000004e-06 2024-09-13T09:12:33.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.152368+0000", 2024-09-13T09:12:33.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8860999999999999e-05 2024-09-13T09:12:33.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.152429+0000", 2024-09-13T09:12:33.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0541999999999997e-05 2024-09-13T09:12:33.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:33.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.152509+0000", 2024-09-13T09:12:33.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9954000000000001e-05 2024-09-13T09:12:33.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.153743+0000", 2024-09-13T09:12:33.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001233984 2024-09-13T09:12:33.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.157221+0000", 2024-09-13T09:12:33.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0034781450000000002 2024-09-13T09:12:33.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.157301+0000", 2024-09-13T09:12:33.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9913999999999994e-05 2024-09-13T09:12:33.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.157328+0000", 2024-09-13T09:12:33.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6712000000000001e-05 2024-09-13T09:12:33.352 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.352 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.352 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.353 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:1331 3.9 3:973d5796:::rbd_data.1123b8b4a28a.000000000000003f:head [set-alloc-hint object_size 2097152 write_size 2097152,write 294912~24064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:33.353 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.246249+0000", 2024-09-13T09:12:33.353 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.0491514970000004, 2024-09-13T09:12:33.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004791661, 2024-09-13T09:12:33.353 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.353 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.353 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.353 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:33.353 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:33.353 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1331 2024-09-13T09:12:33.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.353 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.246249+0000", 2024-09-13T09:12:33.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.246249+0000", 2024-09-13T09:12:33.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.246248+0000", 2024-09-13T09:12:33.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:33.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.246265+0000", 2024-09-13T09:12:33.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6308e-05 2024-09-13T09:12:33.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.246277+0000", 2024-09-13T09:12:33.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1924e-05 2024-09-13T09:12:33.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.246284+0000", 2024-09-13T09:12:33.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0060000000000003e-06 2024-09-13T09:12:33.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.246326+0000", 2024-09-13T09:12:33.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2230000000000001e-05 2024-09-13T09:12:33.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.246431+0000", 2024-09-13T09:12:33.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000105441 2024-09-13T09:12:33.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:33.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.246534+0000", 2024-09-13T09:12:33.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102704 2024-09-13T09:12:33.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.247309+0000", 2024-09-13T09:12:33.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00077521600000000001 2024-09-13T09:12:33.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.250997+0000", 2024-09-13T09:12:33.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036879730000000002 2024-09-13T09:12:33.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.251021+0000", 2024-09-13T09:12:33.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4238000000000001e-05 2024-09-13T09:12:33.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.251041+0000", 2024-09-13T09:12:33.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9361e-05 2024-09-13T09:12:33.358 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.358 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.358 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.358 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:1791 3.14 3:2e114262:::rbd_data.1123b8b4a28a.0000000000000009:head [set-alloc-hint object_size 2097152 write_size 2097152,write 28160~4608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:33.358 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.419462+0000", 2024-09-13T09:12:33.358 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.8759385630000001, 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0047530330000000003, 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1791 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.419462+0000", 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.419462+0000", 2024-09-13T09:12:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.419461+0000", 2024-09-13T09:12:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.419468+0000", 2024-09-13T09:12:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9260000000000003e-06 2024-09-13T09:12:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.419478+0000", 2024-09-13T09:12:33.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5349999999999993e-06 2024-09-13T09:12:33.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.419483+0000", 2024-09-13T09:12:33.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1159999999999998e-06 2024-09-13T09:12:33.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.419513+0000", 2024-09-13T09:12:33.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0392999999999999e-05 2024-09-13T09:12:33.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:33.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.419517+0000", 2024-09-13T09:12:33.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5640000000000001e-06 2024-09-13T09:12:33.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.423239+0000", 2024-09-13T09:12:33.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0037218580000000002 2024-09-13T09:12:33.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.423283+0000", 2024-09-13T09:12:33.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4279000000000001e-05 2024-09-13T09:12:33.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:33.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.423373+0000", 2024-09-13T09:12:33.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9716999999999995e-05 2024-09-13T09:12:33.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.423904+0000", 2024-09-13T09:12:33.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053167799999999995 2024-09-13T09:12:33.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.424154+0000", 2024-09-13T09:12:33.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000250034 2024-09-13T09:12:33.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.424182+0000", 2024-09-13T09:12:33.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7583e-05 2024-09-13T09:12:33.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.424215+0000", 2024-09-13T09:12:33.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3105000000000002e-05 2024-09-13T09:12:33.364 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.364 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.364 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.365 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:1996 3.11 3:8e1724e1:::rbd_data.1123b8b4a28a.0000000000000072:head [set-alloc-hint object_size 2097152 write_size 2097152,write 532480~24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:33.365 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.507169+0000", 2024-09-13T09:12:33.365 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7882320119999999, 2024-09-13T09:12:33.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0081220519999999994, 2024-09-13T09:12:33.365 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.365 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.365 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.365 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:33.365 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:33.365 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1996 2024-09-13T09:12:33.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.365 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507169+0000", 2024-09-13T09:12:33.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507169+0000", 2024-09-13T09:12:33.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507168+0000", 2024-09-13T09:12:33.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:33.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507193+0000", 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5505999999999999e-05 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507204+0000", 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1209999999999999e-05 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507212+0000", 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507472+0000", 2024-09-13T09:12:33.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025948799999999999 2024-09-13T09:12:33.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507516+0000", 2024-09-13T09:12:33.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4416999999999998e-05 2024-09-13T09:12:33.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:33.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507598+0000", 2024-09-13T09:12:33.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2081000000000002e-05 2024-09-13T09:12:33.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.508167+0000", 2024-09-13T09:12:33.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056849899999999998 2024-09-13T09:12:33.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.515246+0000", 2024-09-13T09:12:33.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0070798809999999997 2024-09-13T09:12:33.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.515272+0000", 2024-09-13T09:12:33.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.529e-05 2024-09-13T09:12:33.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.515291+0000", 2024-09-13T09:12:33.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8887000000000001e-05 2024-09-13T09:12:33.370 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.370 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.370 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.370 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:2110 3.3 3:c083ceea:::rbd_data.1123b8b4a28a.000000000000005d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:33.370 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.548044+0000", 2024-09-13T09:12:33.370 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7473566160000003, 2024-09-13T09:12:33.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005108502, 2024-09-13T09:12:33.371 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.371 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:33.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:33.371 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2110 2024-09-13T09:12:33.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.371 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548044+0000", 2024-09-13T09:12:33.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548044+0000", 2024-09-13T09:12:33.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548043+0000", 2024-09-13T09:12:33.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:33.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548048+0000", 2024-09-13T09:12:33.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4580000000000003e-06 2024-09-13T09:12:33.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548058+0000", 2024-09-13T09:12:33.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0494e-05 2024-09-13T09:12:33.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548063+0000", 2024-09-13T09:12:33.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:12:33.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548126+0000", 2024-09-13T09:12:33.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2761000000000004e-05 2024-09-13T09:12:33.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548201+0000", 2024-09-13T09:12:33.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5258000000000003e-05 2024-09-13T09:12:33.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:33.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548297+0000", 2024-09-13T09:12:33.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6409000000000005e-05 2024-09-13T09:12:33.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.549538+0000", 2024-09-13T09:12:33.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0012407830000000001 2024-09-13T09:12:33.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.553082+0000", 2024-09-13T09:12:33.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003544045 2024-09-13T09:12:33.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.553130+0000", 2024-09-13T09:12:33.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7664000000000003e-05 2024-09-13T09:12:33.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.553152+0000", 2024-09-13T09:12:33.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2546999999999999e-05 2024-09-13T09:12:33.376 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.376 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.376 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.376 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:2165 3.1a 3:5e5a1403:::rbd_data.1123b8b4a28a.000000000000003b:head [set-alloc-hint object_size 2097152 write_size 2097152,write 643584~11776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:33.376 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.572796+0000", 2024-09-13T09:12:33.376 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7226042640000001, 2024-09-13T09:12:33.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0047555870000000004, 2024-09-13T09:12:33.376 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.376 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.377 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.377 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:33.377 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:33.377 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2165 2024-09-13T09:12:33.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.377 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.572796+0000", 2024-09-13T09:12:33.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.572796+0000", 2024-09-13T09:12:33.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.572795+0000", 2024-09-13T09:12:33.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:33.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.572816+0000", 2024-09-13T09:12:33.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0483999999999999e-05 2024-09-13T09:12:33.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.572827+0000", 2024-09-13T09:12:33.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0825000000000001e-05 2024-09-13T09:12:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.572833+0000", 2024-09-13T09:12:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0430000000000004e-06 2024-09-13T09:12:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.572856+0000", 2024-09-13T09:12:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3655999999999998e-05 2024-09-13T09:12:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.572934+0000", 2024-09-13T09:12:33.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8040999999999994e-05 2024-09-13T09:12:33.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:33.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.573010+0000", 2024-09-13T09:12:33.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5187999999999999e-05 2024-09-13T09:12:33.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.573581+0000", 2024-09-13T09:12:33.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00057087399999999997 2024-09-13T09:12:33.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.577491+0000", 2024-09-13T09:12:33.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039106729999999999 2024-09-13T09:12:33.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.577521+0000", 2024-09-13T09:12:33.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9709e-05 2024-09-13T09:12:33.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.577552+0000", 2024-09-13T09:12:33.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1016999999999998e-05 2024-09-13T09:12:33.381 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.381 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.382 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:2311 3.9 3:93ffaa8c:::rbd_data.1123b8b4a28a.0000000000000041:head [set-alloc-hint object_size 2097152 write_size 2097152,write 684032~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:33.382 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.619294+0000", 2024-09-13T09:12:33.382 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.6761062970000005, 2024-09-13T09:12:33.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005528252, 2024-09-13T09:12:33.382 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.382 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:33.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:33.382 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2311 2024-09-13T09:12:33.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.383 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619294+0000", 2024-09-13T09:12:33.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619294+0000", 2024-09-13T09:12:33.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619293+0000", 2024-09-13T09:12:33.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:33.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619301+0000", 2024-09-13T09:12:33.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5780000000000001e-06 2024-09-13T09:12:33.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619309+0000", 2024-09-13T09:12:33.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9869999999999999e-06 2024-09-13T09:12:33.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619313+0000", 2024-09-13T09:12:33.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.386e-06 2024-09-13T09:12:33.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619346+0000", 2024-09-13T09:12:33.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2567000000000002e-05 2024-09-13T09:12:33.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619412+0000", 2024-09-13T09:12:33.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5654000000000005e-05 2024-09-13T09:12:33.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:33.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619544+0000", 2024-09-13T09:12:33.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013272699999999999 2024-09-13T09:12:33.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.620054+0000", 2024-09-13T09:12:33.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050981500000000005 2024-09-13T09:12:33.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.624764+0000", 2024-09-13T09:12:33.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0047095640000000003 2024-09-13T09:12:33.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.624802+0000", 2024-09-13T09:12:33.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7805999999999997e-05 2024-09-13T09:12:33.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.624823+0000", 2024-09-13T09:12:33.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0999e-05 2024-09-13T09:12:33.387 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.388 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.388 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.388 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:2450 3.3 3:c4c7f4d9:::rbd_data.1123b8b4a28a.000000000000005b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:33.388 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.674749+0000", 2024-09-13T09:12:33.388 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.6206513190000003, 2024-09-13T09:12:33.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0050554199999999997, 2024-09-13T09:12:33.388 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.388 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:33.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:33.388 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2450 2024-09-13T09:12:33.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.389 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.674749+0000", 2024-09-13T09:12:33.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.674749+0000", 2024-09-13T09:12:33.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.674749+0000", 2024-09-13T09:12:33.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.674752+0000", 2024-09-13T09:12:33.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.88e-06 2024-09-13T09:12:33.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.674759+0000", 2024-09-13T09:12:33.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3699999999999997e-06 2024-09-13T09:12:33.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.674763+0000", 2024-09-13T09:12:33.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1370000000000004e-06 2024-09-13T09:12:33.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.674804+0000", 2024-09-13T09:12:33.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0983999999999999e-05 2024-09-13T09:12:33.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.674851+0000", 2024-09-13T09:12:33.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.689e-05 2024-09-13T09:12:33.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:33.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.674921+0000", 2024-09-13T09:12:33.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9935e-05 2024-09-13T09:12:33.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.675590+0000", 2024-09-13T09:12:33.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000668876 2024-09-13T09:12:33.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.679733+0000", 2024-09-13T09:12:33.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004142888 2024-09-13T09:12:33.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.679772+0000", 2024-09-13T09:12:33.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9514000000000001e-05 2024-09-13T09:12:33.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.679805+0000", 2024-09-13T09:12:33.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2496000000000002e-05 2024-09-13T09:12:33.394 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.394 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.394 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.394 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:2454 3.9 3:96917efd:::rbd_data.1123b8b4a28a.0000000000000066:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:33.394 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.676741+0000", 2024-09-13T09:12:33.394 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.6186594669999996, 2024-09-13T09:12:33.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053986470000000003, 2024-09-13T09:12:33.394 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.394 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:33.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:33.395 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2454 2024-09-13T09:12:33.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.395 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676741+0000", 2024-09-13T09:12:33.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676741+0000", 2024-09-13T09:12:33.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676741+0000", 2024-09-13T09:12:33.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676744+0000", 2024-09-13T09:12:33.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:12:33.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676751+0000", 2024-09-13T09:12:33.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5140000000000004e-06 2024-09-13T09:12:33.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676755+0000", 2024-09-13T09:12:33.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9990000000000002e-06 2024-09-13T09:12:33.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676788+0000", 2024-09-13T09:12:33.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3181e-05 2024-09-13T09:12:33.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676861+0000", 2024-09-13T09:12:33.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2924999999999997e-05 2024-09-13T09:12:33.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:33.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676966+0000", 2024-09-13T09:12:33.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000104535 2024-09-13T09:12:33.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.677384+0000", 2024-09-13T09:12:33.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00041842600000000001 2024-09-13T09:12:33.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.681986+0000", 2024-09-13T09:12:33.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0046015250000000004 2024-09-13T09:12:33.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.682054+0000", 2024-09-13T09:12:33.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8235e-05 2024-09-13T09:12:33.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.682140+0000", 2024-09-13T09:12:33.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5840999999999993e-05 2024-09-13T09:12:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:2479 3.11 3:8e1724e1:::rbd_data.1123b8b4a28a.0000000000000072:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.679877+0000", 2024-09-13T09:12:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.6155232450000003, 2024-09-13T09:12:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045199259999999996, 2024-09-13T09:12:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2479 2024-09-13T09:12:33.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.401 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.679877+0000", 2024-09-13T09:12:33.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.679877+0000", 2024-09-13T09:12:33.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.679877+0000", 2024-09-13T09:12:33.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:33.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.679881+0000", 2024-09-13T09:12:33.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1710000000000004e-06 2024-09-13T09:12:33.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.679891+0000", 2024-09-13T09:12:33.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.041e-05 2024-09-13T09:12:33.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.679899+0000", 2024-09-13T09:12:33.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8800000000000008e-06 2024-09-13T09:12:33.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.679929+0000", 2024-09-13T09:12:33.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9563000000000001e-05 2024-09-13T09:12:33.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.679970+0000", 2024-09-13T09:12:33.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0939000000000002e-05 2024-09-13T09:12:33.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:33.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680032+0000", 2024-09-13T09:12:33.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2816000000000003e-05 2024-09-13T09:12:33.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.683380+0000", 2024-09-13T09:12:33.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033471600000000001 2024-09-13T09:12:33.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.684324+0000", 2024-09-13T09:12:33.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00094440300000000002 2024-09-13T09:12:33.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.684364+0000", 2024-09-13T09:12:33.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0148000000000003e-05 2024-09-13T09:12:33.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.684397+0000", 2024-09-13T09:12:33.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3216000000000002e-05 2024-09-13T09:12:33.405 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.405 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.406 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.406 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:2487 3.3 3:c560f27e:::rbd_data.1123b8b4a28a.0000000000000076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:33.406 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.680361+0000", 2024-09-13T09:12:33.406 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.6150392770000002, 2024-09-13T09:12:33.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057142149999999999, 2024-09-13T09:12:33.406 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.406 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:33.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:33.406 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2487 2024-09-13T09:12:33.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.407 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680361+0000", 2024-09-13T09:12:33.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680361+0000", 2024-09-13T09:12:33.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680361+0000", 2024-09-13T09:12:33.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:33.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680365+0000", 2024-09-13T09:12:33.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6260000000000003e-06 2024-09-13T09:12:33.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680373+0000", 2024-09-13T09:12:33.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2029999999999992e-06 2024-09-13T09:12:33.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680378+0000", 2024-09-13T09:12:33.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2710000000000003e-06 2024-09-13T09:12:33.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680419+0000", 2024-09-13T09:12:33.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1267000000000003e-05 2024-09-13T09:12:33.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680478+0000", 2024-09-13T09:12:33.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8955000000000001e-05 2024-09-13T09:12:33.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:33.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680537+0000", 2024-09-13T09:12:33.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9001e-05 2024-09-13T09:12:33.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.684320+0000", 2024-09-13T09:12:33.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0037828860000000001 2024-09-13T09:12:33.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.686008+0000", 2024-09-13T09:12:33.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0016885660000000001 2024-09-13T09:12:33.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.686045+0000", 2024-09-13T09:12:33.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6279000000000003e-05 2024-09-13T09:12:33.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.686076+0000", 2024-09-13T09:12:33.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0945000000000003e-05 2024-09-13T09:12:33.411 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.411 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.412 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.412 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.412 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.412 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:33.413 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:33.414 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:33.412+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:33.489 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:12:33.534 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:33.532+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:33.565 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:33.565 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:33.565 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:33.566 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:33.575 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:33.614 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:33.612+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:33.679 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:33.680 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:33.680 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:33.680 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:33.680 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:33.688 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:33.732 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:33.731+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:33.797 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4221.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:12:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:04.320224+0000", 2024-09-13T09:12:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.474662897000002, 2024-09-13T09:12:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010683929, 2024-09-13T09:12:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4221", 2024-09-13T09:12:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.44:0/558773684", 2024-09-13T09:12:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1 2024-09-13T09:12:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320224+0000", 2024-09-13T09:12:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320224+0000", 2024-09-13T09:12:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320220+0000", 2024-09-13T09:12:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999967 2024-09-13T09:12:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320230+0000", 2024-09-13T09:12:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0275000000000001e-05 2024-09-13T09:12:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320241+0000", 2024-09-13T09:12:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0154e-05 2024-09-13T09:12:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320256+0000", 2024-09-13T09:12:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5068999999999999e-05 2024-09-13T09:12:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320328+0000", 2024-09-13T09:12:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2507000000000002e-05 2024-09-13T09:12:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.320336+0000", 2024-09-13T09:12:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:12:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.322475+0000", 2024-09-13T09:12:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021399430000000001 2024-09-13T09:12:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.322484+0000", 2024-09-13T09:12:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5779999999999999e-06 2024-09-13T09:12:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.325065+0000", 2024-09-13T09:12:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002581323 2024-09-13T09:12:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.325211+0000", 2024-09-13T09:12:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014534899999999999 2024-09-13T09:12:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.325374+0000", 2024-09-13T09:12:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000162898 2024-09-13T09:12:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.326528+0000", 2024-09-13T09:12:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001154415 2024-09-13T09:12:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.330811+0000", 2024-09-13T09:12:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042825090000000003 2024-09-13T09:12:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.330862+0000", 2024-09-13T09:12:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1430000000000001e-05 2024-09-13T09:12:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:04.330908+0000", 2024-09-13T09:12:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5636000000000001e-05 2024-09-13T09:12:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4546.0:2175 3.10 3:0eca404c:::rbd_data.1123ffac6bc2.0000000000000080:head [sparse-read 4096~126976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.048825+0000", 2024-09-13T09:12:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.746061504, 2024-09-13T09:12:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026988175999999999, 2024-09-13T09:12:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4546", 2024-09-13T09:12:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2955085030", 2024-09-13T09:12:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2175 2024-09-13T09:12:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048825+0000", 2024-09-13T09:12:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048825+0000", 2024-09-13T09:12:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048825+0000", 2024-09-13T09:12:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048828+0000", 2024-09-13T09:12:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2160000000000002e-06 2024-09-13T09:12:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048837+0000", 2024-09-13T09:12:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6180000000000001e-06 2024-09-13T09:12:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048841+0000", 2024-09-13T09:12:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5389999999999999e-06 2024-09-13T09:12:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.048952+0000", 2024-09-13T09:12:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011115 2024-09-13T09:12:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.075813+0000", 2024-09-13T09:12:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026861182000000001 2024-09-13T09:12:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4546.0:2178 3.13 3:c89627a6:::rbd_data.1123ffac6bc2.0000000000000082:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.049364+0000", 2024-09-13T09:12:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.7455224719999993, 2024-09-13T09:12:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026490942999999999, 2024-09-13T09:12:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4546", 2024-09-13T09:12:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2955085030", 2024-09-13T09:12:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2178 2024-09-13T09:12:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.049364+0000", 2024-09-13T09:12:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.049364+0000", 2024-09-13T09:12:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.049363+0000", 2024-09-13T09:12:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.049367+0000", 2024-09-13T09:12:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4960000000000001e-06 2024-09-13T09:12:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.075707+0000", 2024-09-13T09:12:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026339912 2024-09-13T09:12:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.075727+0000", 2024-09-13T09:12:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9898000000000002e-05 2024-09-13T09:12:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.075778+0000", 2024-09-13T09:12:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1617999999999999e-05 2024-09-13T09:12:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.075855+0000", 2024-09-13T09:12:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6644999999999995e-05 2024-09-13T09:12:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1553 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510425+0000", 2024-09-13T09:12:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.2844616260000006, 2024-09-13T09:12:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0089516189999999992, 2024-09-13T09:12:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1553 2024-09-13T09:12:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510425+0000", 2024-09-13T09:12:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510425+0000", 2024-09-13T09:12:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510425+0000", 2024-09-13T09:12:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510426+0000", 2024-09-13T09:12:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.72e-06 2024-09-13T09:12:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510434+0000", 2024-09-13T09:12:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7649999999999993e-06 2024-09-13T09:12:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510441+0000", 2024-09-13T09:12:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8449999999999997e-06 2024-09-13T09:12:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.514480+0000", 2024-09-13T09:12:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0040390199999999999 2024-09-13T09:12:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.514545+0000", 2024-09-13T09:12:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4834999999999998e-05 2024-09-13T09:12:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.514776+0000", 2024-09-13T09:12:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023143 2024-09-13T09:12:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519286+0000", 2024-09-13T09:12:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045099720000000001 2024-09-13T09:12:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519296+0000", 2024-09-13T09:12:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-06 2024-09-13T09:12:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519342+0000", 2024-09-13T09:12:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6436000000000001e-05 2024-09-13T09:12:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519377+0000", 2024-09-13T09:12:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4269e-05 2024-09-13T09:12:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1554 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510444+0000", 2024-09-13T09:12:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.284442812, 2024-09-13T09:12:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0090383259999999993, 2024-09-13T09:12:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1554 2024-09-13T09:12:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510444+0000", 2024-09-13T09:12:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510444+0000", 2024-09-13T09:12:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510444+0000", 2024-09-13T09:12:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510445+0000", 2024-09-13T09:12:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.951e-06 2024-09-13T09:12:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510449+0000", 2024-09-13T09:12:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3569999999999998e-06 2024-09-13T09:12:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510456+0000", 2024-09-13T09:12:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6599999999999998e-06 2024-09-13T09:12:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.515200+0000", 2024-09-13T09:12:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0047443889999999999 2024-09-13T09:12:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.515286+0000", 2024-09-13T09:12:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5626999999999999e-05 2024-09-13T09:12:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.515685+0000", 2024-09-13T09:12:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039960899999999999 2024-09-13T09:12:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519386+0000", 2024-09-13T09:12:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0037010659999999998 2024-09-13T09:12:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519426+0000", 2024-09-13T09:12:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9929999999999999e-05 2024-09-13T09:12:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519458+0000", 2024-09-13T09:12:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2189e-05 2024-09-13T09:12:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519482+0000", 2024-09-13T09:12:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3779999999999999e-05 2024-09-13T09:12:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1555 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510460+0000", 2024-09-13T09:12:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.2844269720000003, 2024-09-13T09:12:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0091250399999999992, 2024-09-13T09:12:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1555 2024-09-13T09:12:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510460+0000", 2024-09-13T09:12:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510460+0000", 2024-09-13T09:12:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510459+0000", 2024-09-13T09:12:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510464+0000", 2024-09-13T09:12:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8709999999999996e-06 2024-09-13T09:12:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510469+0000", 2024-09-13T09:12:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:12:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510472+0000", 2024-09-13T09:12:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.813e-06 2024-09-13T09:12:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516420+0000", 2024-09-13T09:12:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059477749999999998 2024-09-13T09:12:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516523+0000", 2024-09-13T09:12:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103222 2024-09-13T09:12:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.516877+0000", 2024-09-13T09:12:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00035438100000000002 2024-09-13T09:12:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519391+0000", 2024-09-13T09:12:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002514166 2024-09-13T09:12:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519520+0000", 2024-09-13T09:12:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000128582 2024-09-13T09:12:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519566+0000", 2024-09-13T09:12:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6224999999999999e-05 2024-09-13T09:12:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519585+0000", 2024-09-13T09:12:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8391000000000001e-05 2024-09-13T09:12:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1556 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510474+0000", 2024-09-13T09:12:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.2844121499999996, 2024-09-13T09:12:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0091831980000000001, 2024-09-13T09:12:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1556 2024-09-13T09:12:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510474+0000", 2024-09-13T09:12:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510474+0000", 2024-09-13T09:12:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510474+0000", 2024-09-13T09:12:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510476+0000", 2024-09-13T09:12:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1799999999999999e-06 2024-09-13T09:12:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510479+0000", 2024-09-13T09:12:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1159999999999999e-06 2024-09-13T09:12:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510482+0000", 2024-09-13T09:12:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:12:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.517292+0000", 2024-09-13T09:12:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068100239999999996 2024-09-13T09:12:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.517344+0000", 2024-09-13T09:12:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2092999999999997e-05 2024-09-13T09:12:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.517558+0000", 2024-09-13T09:12:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021441900000000001 2024-09-13T09:12:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519397+0000", 2024-09-13T09:12:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001838891 2024-09-13T09:12:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519617+0000", 2024-09-13T09:12:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021970400000000001 2024-09-13T09:12:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519639+0000", 2024-09-13T09:12:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1773999999999999e-05 2024-09-13T09:12:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519658+0000", 2024-09-13T09:12:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8787000000000002e-05 2024-09-13T09:12:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1557 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510490+0000", 2024-09-13T09:12:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.2843965540000006, 2024-09-13T09:12:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0092238319999999995, 2024-09-13T09:12:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1557 2024-09-13T09:12:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510490+0000", 2024-09-13T09:12:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510490+0000", 2024-09-13T09:12:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510490+0000", 2024-09-13T09:12:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510492+0000", 2024-09-13T09:12:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-13T09:12:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510495+0000", 2024-09-13T09:12:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5020000000000001e-06 2024-09-13T09:12:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510498+0000", 2024-09-13T09:12:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6809999999999998e-06 2024-09-13T09:12:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.517928+0000", 2024-09-13T09:12:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0074299259999999999 2024-09-13T09:12:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.518005+0000", 2024-09-13T09:12:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6642000000000002e-05 2024-09-13T09:12:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.518340+0000", 2024-09-13T09:12:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00033502400000000001 2024-09-13T09:12:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519592+0000", 2024-09-13T09:12:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001252647 2024-09-13T09:12:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519677+0000", 2024-09-13T09:12:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4751000000000002e-05 2024-09-13T09:12:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519698+0000", 2024-09-13T09:12:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0792999999999999e-05 2024-09-13T09:12:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519714+0000", 2024-09-13T09:12:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6008e-05 2024-09-13T09:12:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4537.0:1558 3.5 3:a240ac0f:::rbd_object_map.1123a0dcea4a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:24.510501+0000", 2024-09-13T09:12:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.2843855249999994, 2024-09-13T09:12:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0094031559999999993, 2024-09-13T09:12:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4537", 2024-09-13T09:12:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1558 2024-09-13T09:12:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510501+0000", 2024-09-13T09:12:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510501+0000", 2024-09-13T09:12:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510501+0000", 2024-09-13T09:12:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510503+0000", 2024-09-13T09:12:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.72e-06 2024-09-13T09:12:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510506+0000", 2024-09-13T09:12:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.286e-06 2024-09-13T09:12:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.510508+0000", 2024-09-13T09:12:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-13T09:12:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.518739+0000", 2024-09-13T09:12:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0082309080000000003 2024-09-13T09:12:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.518787+0000", 2024-09-13T09:12:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8214999999999998e-05 2024-09-13T09:12:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519003+0000", 2024-09-13T09:12:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021570199999999999 2024-09-13T09:12:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519720+0000", 2024-09-13T09:12:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00071655099999999995 2024-09-13T09:12:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519868+0000", 2024-09-13T09:12:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014865000000000001 2024-09-13T09:12:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519888+0000", 2024-09-13T09:12:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0177e-05 2024-09-13T09:12:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:24.519904+0000", 2024-09-13T09:12:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5767999999999999e-05 2024-09-13T09:12:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4589.0:376 3.16 3:6bc7d1be:::rbd_object_map.1123b8d418eb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:12:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:30.005039+0000", 2024-09-13T09:12:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.7898478789999999, 2024-09-13T09:12:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0086753839999999995, 2024-09-13T09:12:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4589", 2024-09-13T09:12:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1943449923", 2024-09-13T09:12:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 376 2024-09-13T09:12:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005039+0000", 2024-09-13T09:12:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005039+0000", 2024-09-13T09:12:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005038+0000", 2024-09-13T09:12:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005041+0000", 2024-09-13T09:12:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2729999999999998e-06 2024-09-13T09:12:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005048+0000", 2024-09-13T09:12:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8120000000000002e-06 2024-09-13T09:12:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005053+0000", 2024-09-13T09:12:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8740000000000003e-06 2024-09-13T09:12:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.006410+0000", 2024-09-13T09:12:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0013570629999999999 2024-09-13T09:12:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.006509+0000", 2024-09-13T09:12:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9055999999999995e-05 2024-09-13T09:12:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.006862+0000", 2024-09-13T09:12:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00035223299999999997 2024-09-13T09:12:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.008718+0000", 2024-09-13T09:12:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001856688 2024-09-13T09:12:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.013640+0000", 2024-09-13T09:12:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0049218420000000001 2024-09-13T09:12:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.013666+0000", 2024-09-13T09:12:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6075999999999999e-05 2024-09-13T09:12:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.013714+0000", 2024-09-13T09:12:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7961999999999999e-05 2024-09-13T09:12:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4589.0:377 3.16 3:6bc7d1be:::rbd_object_map.1123b8d418eb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:12:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:30.005331+0000", 2024-09-13T09:12:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.7895551350000001, 2024-09-13T09:12:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.008464704, 2024-09-13T09:12:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4589", 2024-09-13T09:12:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1943449923", 2024-09-13T09:12:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 377 2024-09-13T09:12:33.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.858 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005331+0000", 2024-09-13T09:12:33.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005331+0000", 2024-09-13T09:12:33.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005331+0000", 2024-09-13T09:12:33.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:33.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005335+0000", 2024-09-13T09:12:33.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5070000000000002e-06 2024-09-13T09:12:33.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005343+0000", 2024-09-13T09:12:33.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7510000000000008e-06 2024-09-13T09:12:33.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005349+0000", 2024-09-13T09:12:33.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3300000000000004e-06 2024-09-13T09:12:33.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.007380+0000", 2024-09-13T09:12:33.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002031176 2024-09-13T09:12:33.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.007429+0000", 2024-09-13T09:12:33.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8130000000000002e-05 2024-09-13T09:12:33.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:33.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.007643+0000", 2024-09-13T09:12:33.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021404599999999999 2024-09-13T09:12:33.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.009782+0000", 2024-09-13T09:12:33.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021395379999999999 2024-09-13T09:12:33.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.013725+0000", 2024-09-13T09:12:33.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039429770000000003 2024-09-13T09:12:33.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.013769+0000", 2024-09-13T09:12:33.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4240000000000003e-05 2024-09-13T09:12:33.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.013796+0000", 2024-09-13T09:12:33.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6804999999999998e-05 2024-09-13T09:12:33.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.862 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.863 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4589.0:378 3.16 3:6bc7d1be:::rbd_object_map.1123b8d418eb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:12:33.863 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:30.005360+0000", 2024-09-13T09:12:33.863 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.789526714, 2024-09-13T09:12:33.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0084969060000000003, 2024-09-13T09:12:33.863 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.863 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4589", 2024-09-13T09:12:33.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1943449923", 2024-09-13T09:12:33.863 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 378 2024-09-13T09:12:33.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.864 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005360+0000", 2024-09-13T09:12:33.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005360+0000", 2024-09-13T09:12:33.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005359+0000", 2024-09-13T09:12:33.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005363+0000", 2024-09-13T09:12:33.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1810000000000001e-06 2024-09-13T09:12:33.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005369+0000", 2024-09-13T09:12:33.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2620000000000004e-06 2024-09-13T09:12:33.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005374+0000", 2024-09-13T09:12:33.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.023e-06 2024-09-13T09:12:33.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.007908+0000", 2024-09-13T09:12:33.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0025339719999999998 2024-09-13T09:12:33.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.007987+0000", 2024-09-13T09:12:33.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9633000000000001e-05 2024-09-13T09:12:33.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:33.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.008335+0000", 2024-09-13T09:12:33.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00034711099999999999 2024-09-13T09:12:33.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.009860+0000", 2024-09-13T09:12:33.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015250109999999999 2024-09-13T09:12:33.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.013802+0000", 2024-09-13T09:12:33.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039427309999999997 2024-09-13T09:12:33.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.013827+0000", 2024-09-13T09:12:33.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4284999999999999e-05 2024-09-13T09:12:33.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.013857+0000", 2024-09-13T09:12:33.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0151e-05 2024-09-13T09:12:33.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.869 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.869 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4589.0:379 3.16 3:6bc7d1be:::rbd_object_map.1123b8d418eb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:12:33.869 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:30.005465+0000", 2024-09-13T09:12:33.869 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.7894216979999999, 2024-09-13T09:12:33.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0089794039999999999, 2024-09-13T09:12:33.869 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.869 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.869 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.869 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4589", 2024-09-13T09:12:33.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1943449923", 2024-09-13T09:12:33.870 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 379 2024-09-13T09:12:33.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.870 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005465+0000", 2024-09-13T09:12:33.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005465+0000", 2024-09-13T09:12:33.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005464+0000", 2024-09-13T09:12:33.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005468+0000", 2024-09-13T09:12:33.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8630000000000001e-06 2024-09-13T09:12:33.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005474+0000", 2024-09-13T09:12:33.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:12:33.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005479+0000", 2024-09-13T09:12:33.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2109999999999997e-06 2024-09-13T09:12:33.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.008746+0000", 2024-09-13T09:12:33.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0032663060000000001 2024-09-13T09:12:33.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.008812+0000", 2024-09-13T09:12:33.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5846999999999995e-05 2024-09-13T09:12:33.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:33.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.009019+0000", 2024-09-13T09:12:33.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000207324 2024-09-13T09:12:33.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.011053+0000", 2024-09-13T09:12:33.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002033928 2024-09-13T09:12:33.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.014361+0000", 2024-09-13T09:12:33.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003308002 2024-09-13T09:12:33.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.014410+0000", 2024-09-13T09:12:33.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9055999999999999e-05 2024-09-13T09:12:33.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.014444+0000", 2024-09-13T09:12:33.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4459999999999999e-05 2024-09-13T09:12:33.875 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.875 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.875 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.875 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4589.0:380 3.16 3:6bc7d1be:::rbd_object_map.1123b8d418eb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:12:33.875 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:30.005489+0000", 2024-09-13T09:12:33.875 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.789397975, 2024-09-13T09:12:33.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0090426099999999995, 2024-09-13T09:12:33.876 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.876 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4589", 2024-09-13T09:12:33.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1943449923", 2024-09-13T09:12:33.876 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 380 2024-09-13T09:12:33.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.876 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005489+0000", 2024-09-13T09:12:33.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005489+0000", 2024-09-13T09:12:33.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005488+0000", 2024-09-13T09:12:33.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005491+0000", 2024-09-13T09:12:33.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:12:33.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005495+0000", 2024-09-13T09:12:33.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0629999999999999e-06 2024-09-13T09:12:33.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005498+0000", 2024-09-13T09:12:33.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6800000000000002e-06 2024-09-13T09:12:33.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.009275+0000", 2024-09-13T09:12:33.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0037770019999999998 2024-09-13T09:12:33.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.009321+0000", 2024-09-13T09:12:33.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6699999999999997e-05 2024-09-13T09:12:33.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:33.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.009537+0000", 2024-09-13T09:12:33.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021603200000000001 2024-09-13T09:12:33.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.011097+0000", 2024-09-13T09:12:33.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015596620000000001 2024-09-13T09:12:33.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.014460+0000", 2024-09-13T09:12:33.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033627179999999998 2024-09-13T09:12:33.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.014500+0000", 2024-09-13T09:12:33.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9891999999999997e-05 2024-09-13T09:12:33.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.014531+0000", 2024-09-13T09:12:33.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1664e-05 2024-09-13T09:12:33.881 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.881 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.881 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.882 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4589.0:381 3.16 3:6bc7d1be:::rbd_object_map.1123b8d418eb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:12:33.882 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:30.005637+0000", 2024-09-13T09:12:33.882 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.7892500469999999, 2024-09-13T09:12:33.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0089713429999999997, 2024-09-13T09:12:33.882 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.882 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4589", 2024-09-13T09:12:33.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1943449923", 2024-09-13T09:12:33.882 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 381 2024-09-13T09:12:33.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.883 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005637+0000", 2024-09-13T09:12:33.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005637+0000", 2024-09-13T09:12:33.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005636+0000", 2024-09-13T09:12:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005640+0000", 2024-09-13T09:12:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.374e-06 2024-09-13T09:12:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005647+0000", 2024-09-13T09:12:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3049999999999998e-06 2024-09-13T09:12:33.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005651+0000", 2024-09-13T09:12:33.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:12:33.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.009884+0000", 2024-09-13T09:12:33.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004233194 2024-09-13T09:12:33.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.009938+0000", 2024-09-13T09:12:33.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4546000000000003e-05 2024-09-13T09:12:33.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:33.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.010143+0000", 2024-09-13T09:12:33.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020462400000000001 2024-09-13T09:12:33.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.011141+0000", 2024-09-13T09:12:33.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00099756499999999995 2024-09-13T09:12:33.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.014538+0000", 2024-09-13T09:12:33.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033975889999999999 2024-09-13T09:12:33.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.014577+0000", 2024-09-13T09:12:33.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8460000000000001e-05 2024-09-13T09:12:33.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.014608+0000", 2024-09-13T09:12:33.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1167000000000002e-05 2024-09-13T09:12:33.887 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.887 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.888 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.888 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4589.0:382 3.16 3:6bc7d1be:::rbd_object_map.1123b8d418eb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:12:33.888 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:30.005757+0000", 2024-09-13T09:12:33.888 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.789129215, 2024-09-13T09:12:33.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0089280279999999993, 2024-09-13T09:12:33.888 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.888 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4589", 2024-09-13T09:12:33.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1943449923", 2024-09-13T09:12:33.889 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 382 2024-09-13T09:12:33.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.889 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005757+0000", 2024-09-13T09:12:33.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005757+0000", 2024-09-13T09:12:33.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005757+0000", 2024-09-13T09:12:33.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005759+0000", 2024-09-13T09:12:33.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4899999999999999e-06 2024-09-13T09:12:33.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005766+0000", 2024-09-13T09:12:33.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7819999999999996e-06 2024-09-13T09:12:33.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005769+0000", 2024-09-13T09:12:33.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.19e-06 2024-09-13T09:12:33.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.010387+0000", 2024-09-13T09:12:33.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0046178620000000004 2024-09-13T09:12:33.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.010431+0000", 2024-09-13T09:12:33.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:12:33.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:33.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.010706+0000", 2024-09-13T09:12:33.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00027503400000000001 2024-09-13T09:12:33.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.013908+0000", 2024-09-13T09:12:33.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0032021200000000001 2024-09-13T09:12:33.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.014615+0000", 2024-09-13T09:12:33.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00070676099999999998 2024-09-13T09:12:33.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.014652+0000", 2024-09-13T09:12:33.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7086999999999999e-05 2024-09-13T09:12:33.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.014685+0000", 2024-09-13T09:12:33.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3256999999999997e-05 2024-09-13T09:12:33.894 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.894 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.894 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.894 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4589.0:383 3.16 3:6bc7d1be:::rbd_object_map.1123b8d418eb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:12:33.894 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:30.005808+0000", 2024-09-13T09:12:33.894 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.7890782239999998, 2024-09-13T09:12:33.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011423302999999999, 2024-09-13T09:12:33.894 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.894 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.894 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4589", 2024-09-13T09:12:33.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1943449923", 2024-09-13T09:12:33.895 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 383 2024-09-13T09:12:33.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.895 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005808+0000", 2024-09-13T09:12:33.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005808+0000", 2024-09-13T09:12:33.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005808+0000", 2024-09-13T09:12:33.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:33.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005811+0000", 2024-09-13T09:12:33.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.543e-06 2024-09-13T09:12:33.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005815+0000", 2024-09-13T09:12:33.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3120000000000004e-06 2024-09-13T09:12:33.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005818+0000", 2024-09-13T09:12:33.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8629999999999999e-06 2024-09-13T09:12:33.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.011165+0000", 2024-09-13T09:12:33.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053474849999999999 2024-09-13T09:12:33.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.011243+0000", 2024-09-13T09:12:33.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7333e-05 2024-09-13T09:12:33.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:33.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.011597+0000", 2024-09-13T09:12:33.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00035468000000000001 2024-09-13T09:12:33.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.013955+0000", 2024-09-13T09:12:33.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002358172 2024-09-13T09:12:33.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.017148+0000", 2024-09-13T09:12:33.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003192257 2024-09-13T09:12:33.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.017197+0000", 2024-09-13T09:12:33.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9001000000000001e-05 2024-09-13T09:12:33.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.017232+0000", 2024-09-13T09:12:33.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5043000000000003e-05 2024-09-13T09:12:33.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.900 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.900 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4589.0:384 3.16 3:6bc7d1be:::rbd_object_map.1123b8d418eb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:12:33.900 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:30.005916+0000", 2024-09-13T09:12:33.900 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.7889706419999998, 2024-09-13T09:12:33.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011402439, 2024-09-13T09:12:33.901 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.901 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4589", 2024-09-13T09:12:33.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1943449923", 2024-09-13T09:12:33.901 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 384 2024-09-13T09:12:33.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.901 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005916+0000", 2024-09-13T09:12:33.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005916+0000", 2024-09-13T09:12:33.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005915+0000", 2024-09-13T09:12:33.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:33.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005921+0000", 2024-09-13T09:12:33.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4180000000000002e-06 2024-09-13T09:12:33.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005930+0000", 2024-09-13T09:12:33.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6309999999999992e-06 2024-09-13T09:12:33.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.005936+0000", 2024-09-13T09:12:33.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2639999999999997e-06 2024-09-13T09:12:33.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.011981+0000", 2024-09-13T09:12:33.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060450180000000001 2024-09-13T09:12:33.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.012030+0000", 2024-09-13T09:12:33.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8406999999999999e-05 2024-09-13T09:12:33.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:33.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.012234+0000", 2024-09-13T09:12:33.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020398700000000001 2024-09-13T09:12:33.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.014011+0000", 2024-09-13T09:12:33.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0017777240000000001 2024-09-13T09:12:33.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.017241+0000", 2024-09-13T09:12:33.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0032292670000000001 2024-09-13T09:12:33.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.017280+0000", 2024-09-13T09:12:33.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9688999999999999e-05 2024-09-13T09:12:33.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.017318+0000", 2024-09-13T09:12:33.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8074000000000002e-05 2024-09-13T09:12:33.906 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.906 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.906 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.906 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4589.0:385 3.16 3:6bc7d1be:::rbd_object_map.1123b8d418eb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:12:33.906 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:30.006044+0000", 2024-09-13T09:12:33.906 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.7888425579999998, 2024-09-13T09:12:33.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011352434, 2024-09-13T09:12:33.907 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.907 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.907 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.907 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4589", 2024-09-13T09:12:33.907 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1943449923", 2024-09-13T09:12:33.907 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 385 2024-09-13T09:12:33.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.907 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.006044+0000", 2024-09-13T09:12:33.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.006044+0000", 2024-09-13T09:12:33.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.006043+0000", 2024-09-13T09:12:33.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:33.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.006048+0000", 2024-09-13T09:12:33.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9150000000000004e-06 2024-09-13T09:12:33.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.006059+0000", 2024-09-13T09:12:33.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1294999999999999e-05 2024-09-13T09:12:33.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.006066+0000", 2024-09-13T09:12:33.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7020000000000004e-06 2024-09-13T09:12:33.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.012526+0000", 2024-09-13T09:12:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064603580000000002 2024-09-13T09:12:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.012576+0000", 2024-09-13T09:12:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9069000000000001e-05 2024-09-13T09:12:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.012781+0000", 2024-09-13T09:12:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020551900000000001 2024-09-13T09:12:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.014078+0000", 2024-09-13T09:12:33.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0012970620000000001 2024-09-13T09:12:33.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.017326+0000", 2024-09-13T09:12:33.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003247509 2024-09-13T09:12:33.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.017364+0000", 2024-09-13T09:12:33.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.837e-05 2024-09-13T09:12:33.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.017396+0000", 2024-09-13T09:12:33.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2468999999999997e-05 2024-09-13T09:12:33.912 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.912 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.912 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.912 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4589.0:386 3.16 3:6bc7d1be:::rbd_object_map.1123b8d418eb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:12:33.912 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:30.006226+0000", 2024-09-13T09:12:33.912 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.7886608000000002, 2024-09-13T09:12:33.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011256258999999999, 2024-09-13T09:12:33.913 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:33.913 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:33.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:33.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4589", 2024-09-13T09:12:33.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1943449923", 2024-09-13T09:12:33.913 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 386 2024-09-13T09:12:33.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.913 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:33.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:33.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.006226+0000", 2024-09-13T09:12:33.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:33.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.006226+0000", 2024-09-13T09:12:33.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:33.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:33.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.006225+0000", 2024-09-13T09:12:33.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:33.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:33.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.006230+0000", 2024-09-13T09:12:33.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1989999999999997e-06 2024-09-13T09:12:33.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:33.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.006239+0000", 2024-09-13T09:12:33.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5990000000000002e-06 2024-09-13T09:12:33.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:33.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.006247+0000", 2024-09-13T09:12:33.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3229999999999998e-06 2024-09-13T09:12:33.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:33.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.013096+0000", 2024-09-13T09:12:33.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068488350000000002 2024-09-13T09:12:33.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:33.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.013147+0000", 2024-09-13T09:12:33.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1508999999999998e-05 2024-09-13T09:12:33.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:33.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.013375+0000", 2024-09-13T09:12:33.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000227216 2024-09-13T09:12:33.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:33.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.016330+0000", 2024-09-13T09:12:33.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002955283 2024-09-13T09:12:33.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:33.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.017403+0000", 2024-09-13T09:12:33.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010733450000000001 2024-09-13T09:12:33.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:33.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.017442+0000", 2024-09-13T09:12:33.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9011999999999999e-05 2024-09-13T09:12:33.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:33.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:33.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:33.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:30.017482+0000", 2024-09-13T09:12:33.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9719999999999999e-05 2024-09-13T09:12:33.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.918 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:33.918 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:33.918 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:33.919 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-13T09:12:33.920 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:33.918+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:33.920 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:34.014 INFO:teuthology.orchestra.run.smithi157.stdout:1248 punch from 0x5b9c00 to 0x5bec00, (0x5000 bytes) 2024-09-13T09:12:34.020 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:34.018+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:34.030 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:34.030 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:34.031 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:34.031 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:34.039 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:34.040 INFO:teuthology.orchestra.run.smithi157.stdout:1249 trunc from 0x1a6d000 to 0xb140400 2024-09-13T09:12:34.050 INFO:teuthology.orchestra.run.smithi157.stdout:1251 write 0x1c34600 thru 0x1c41b67 (0xd568 bytes) 2024-09-13T09:12:34.061 INFO:teuthology.orchestra.run.smithi157.stdout:1252 trunc from 0xb140400 to 0x284bc00 2024-09-13T09:12:34.102 INFO:teuthology.orchestra.run.smithi157.stdout:1255 write 0xb710800 thru 0xb71479a (0x3f9b bytes) 2024-09-13T09:12:34.112 INFO:teuthology.orchestra.run.smithi157.stdout:1256 trunc from 0xb714800 to 0x2a54200 2024-09-13T09:12:34.124 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:34.122+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:34.130 INFO:teuthology.orchestra.run.smithi157.stdout:1259 punch from 0x1d44e00 to 0x1d45c00, (0xe00 bytes) 2024-09-13T09:12:34.131 INFO:teuthology.orchestra.run.smithi157.stdout:1260 read 0x15bfc00 thru 0x15ccd32 (0xd133 bytes) 2024-09-13T09:12:34.134 INFO:teuthology.orchestra.run.smithi157.stdout:1261 punch from 0x724600 to 0x72a200, (0x5c00 bytes) 2024-09-13T09:12:34.134 INFO:teuthology.orchestra.run.smithi157.stdout:1262 read 0x24f4a00 thru 0x24f76ce (0x2ccf bytes) 2024-09-13T09:12:34.135 INFO:teuthology.orchestra.run.smithi157.stdout:1264 read 0xa30a00 thru 0xa3716e (0x676f bytes) 2024-09-13T09:12:34.145 INFO:teuthology.orchestra.run.smithi157.stdout:1269 trunc from 0x2a54200 to 0xc633200 2024-09-13T09:12:34.154 INFO:teuthology.orchestra.run.smithi157.stdout:1270 read 0x31c1200 thru 0x31c1f2a (0xd2b bytes) 2024-09-13T09:12:34.161 INFO:teuthology.orchestra.run.smithi157.stdout:1271 trunc from 0xc633200 to 0x7899600 2024-09-13T09:12:34.170 INFO:teuthology.orchestra.run.smithi157.stdout:1272 read 0x5f7be00 thru 0x5f8b61a (0xf81b bytes) 2024-09-13T09:12:34.181 INFO:teuthology.orchestra.run.smithi157.stdout:1273 trunc from 0x7899600 to 0x3a6bc00 2024-09-13T09:12:34.193 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:34.193 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:34.194 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:34.194 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:34.194 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:34.203 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:34.208 INFO:teuthology.orchestra.run.smithi157.stdout:1274 write 0xae4de00 thru 0xae57500 (0x9701 bytes) 2024-09-13T09:12:34.214 INFO:teuthology.orchestra.run.smithi157.stdout:1275 punch from 0x3e53c00 to 0x3e54e00, (0x1200 bytes) 2024-09-13T09:12:34.218 INFO:teuthology.orchestra.run.smithi157.stdout:1279 write 0xac6b000 thru 0xac6ba5b (0xa5c bytes) 2024-09-13T09:12:34.219 INFO:teuthology.orchestra.run.smithi157.stdout:1281 read 0x3a78a00 thru 0x3a80eda (0x84db bytes) 2024-09-13T09:12:34.219 INFO:teuthology.orchestra.run.smithi157.stdout:1282 write 0x130200 thru 0x13b8c7 (0xb6c8 bytes) 2024-09-13T09:12:34.220 INFO:teuthology.orchestra.run.smithi157.stdout:1284 read 0x88d8600 thru 0x88debf9 (0x65fa bytes) 2024-09-13T09:12:34.225 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:34.221+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:34.243 INFO:teuthology.orchestra.run.smithi157.stdout:1285 write 0xbfd8e00 thru 0xbfe47ef (0xb9f0 bytes) 2024-09-13T09:12:34.250 INFO:teuthology.orchestra.run.smithi157.stdout:1289 read 0x5ce9200 thru 0x5cf71b9 (0xdfba bytes) 2024-09-13T09:12:34.250 INFO:teuthology.orchestra.run.smithi157.stdout:1292 write 0x1733e00 thru 0x1741eba (0xe0bb bytes) 2024-09-13T09:12:34.261 INFO:teuthology.orchestra.run.smithi157.stdout:1293 trunc from 0xbfe4800 to 0x2b7c000 2024-09-13T09:12:34.291 INFO:teuthology.orchestra.run.smithi157.stdout:1295 read 0x1c74600 thru 0x1c7d06f (0x8a70 bytes) 2024-09-13T09:12:34.317 INFO:teuthology.orchestra.run.smithi157.stdout:1298 write 0xcb02000 thru 0xcb0fbc3 (0xdbc4 bytes) 2024-09-13T09:12:34.318 INFO:teuthology.orchestra.run.smithi157.stdout:skipping zero length punch hole 2024-09-13T09:12:34.318 INFO:teuthology.orchestra.run.smithi157.stdout:1302 punch from 0x2344c00 to 0x234c800, (0x7c00 bytes) 2024-09-13T09:12:34.319 INFO:teuthology.orchestra.run.smithi157.stdout:1303 flatten 2024-09-13T09:12:34.323 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:34.321+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:34.334 INFO:teuthology.orchestra.run.smithi157.stdout:1306 write 0x4ef4200 thru 0x4efc777 (0x8578 bytes) 2024-09-13T09:12:34.335 INFO:teuthology.orchestra.run.smithi157.stdout:1309 write 0x73d2600 thru 0x73daca8 (0x86a9 bytes) 2024-09-13T09:12:34.335 INFO:teuthology.orchestra.run.smithi157.stdout:1312 write 0x410a800 thru 0x411a5d1 (0xfdd2 bytes) 2024-09-13T09:12:34.335 INFO:teuthology.orchestra.run.smithi157.stdout:1313 trunc from 0xcb0fc00 to 0xbe32c00 2024-09-13T09:12:34.359 INFO:teuthology.orchestra.run.smithi157.stdout:1314 write 0xbd77000 thru 0xbd7eb22 (0x7b23 bytes) 2024-09-13T09:12:34.359 INFO:teuthology.orchestra.run.smithi157.stdout:1315 punch from 0x75b6600 to 0x75bf400, (0x8e00 bytes) 2024-09-13T09:12:34.360 INFO:teuthology.orchestra.run.smithi157.stdout:1316 write 0x50b1800 thru 0x50c14de (0xfcdf bytes) 2024-09-13T09:12:34.360 INFO:teuthology.orchestra.run.smithi157.stdout:1317 read 0xbaf7400 thru 0xbb052d4 (0xded5 bytes) 2024-09-13T09:12:34.360 INFO:teuthology.orchestra.run.smithi157.stdout:1318 write 0x5362c00 thru 0x5371727 (0xeb28 bytes) 2024-09-13T09:12:34.361 INFO:teuthology.orchestra.run.smithi157.stdout:1322 punch from 0xa641c00 to 0xa650000, (0xe400 bytes) 2024-09-13T09:12:34.361 INFO:teuthology.orchestra.run.smithi157.stdout:1324 read 0x7bf2200 thru 0x7bfe3a5 (0xc1a6 bytes) 2024-09-13T09:12:34.362 INFO:teuthology.orchestra.run.smithi157.stdout:1325 read 0x2519c00 thru 0x251b392 (0x1793 bytes) 2024-09-13T09:12:34.369 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:34.370 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:34.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:34.370 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:34.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.370 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1321 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:34.370 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.242418+0000", 2024-09-13T09:12:34.370 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.1236299580000004, 2024-09-13T09:12:34.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0087640029999999994, 2024-09-13T09:12:34.370 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.371 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.371 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.371 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.371 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.371 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1321 2024-09-13T09:12:34.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.371 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242418+0000", 2024-09-13T09:12:34.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242418+0000", 2024-09-13T09:12:34.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242418+0000", 2024-09-13T09:12:34.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242419+0000", 2024-09-13T09:12:34.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7740000000000001e-06 2024-09-13T09:12:34.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242424+0000", 2024-09-13T09:12:34.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4839999999999998e-06 2024-09-13T09:12:34.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242427+0000", 2024-09-13T09:12:34.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8660000000000002e-06 2024-09-13T09:12:34.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.244978+0000", 2024-09-13T09:12:34.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002551139 2024-09-13T09:12:34.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.245025+0000", 2024-09-13T09:12:34.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6832999999999997e-05 2024-09-13T09:12:34.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.245221+0000", 2024-09-13T09:12:34.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000195954 2024-09-13T09:12:34.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.246365+0000", 2024-09-13T09:12:34.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011444669999999999 2024-09-13T09:12:34.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.251115+0000", 2024-09-13T09:12:34.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047497779999999996 2024-09-13T09:12:34.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.378 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.251150+0000", 2024-09-13T09:12:34.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4687999999999997e-05 2024-09-13T09:12:34.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.378 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.251182+0000", 2024-09-13T09:12:34.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2264000000000001e-05 2024-09-13T09:12:34.378 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.378 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.379 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.379 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1550 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:34.379 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.329689+0000", 2024-09-13T09:12:34.379 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.0363592839999995, 2024-09-13T09:12:34.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.009203391, 2024-09-13T09:12:34.379 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.379 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.380 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.380 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.380 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.380 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1550 2024-09-13T09:12:34.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.380 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.380 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.380 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329689+0000", 2024-09-13T09:12:34.380 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329689+0000", 2024-09-13T09:12:34.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329688+0000", 2024-09-13T09:12:34.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329690+0000", 2024-09-13T09:12:34.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7e-06 2024-09-13T09:12:34.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.383 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329695+0000", 2024-09-13T09:12:34.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4880000000000001e-06 2024-09-13T09:12:34.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.383 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329697+0000", 2024-09-13T09:12:34.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.74e-06 2024-09-13T09:12:34.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.333003+0000", 2024-09-13T09:12:34.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003305451 2024-09-13T09:12:34.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.333084+0000", 2024-09-13T09:12:34.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1062000000000004e-05 2024-09-13T09:12:34.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.333427+0000", 2024-09-13T09:12:34.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000342932 2024-09-13T09:12:34.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338607+0000", 2024-09-13T09:12:34.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051806580000000003 2024-09-13T09:12:34.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338821+0000", 2024-09-13T09:12:34.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00021341099999999999 2024-09-13T09:12:34.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338860+0000", 2024-09-13T09:12:34.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9320000000000003e-05 2024-09-13T09:12:34.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338892+0000", 2024-09-13T09:12:34.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1868e-05 2024-09-13T09:12:34.386 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.386 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.387 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1551 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:34.387 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.329715+0000", 2024-09-13T09:12:34.387 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.036332432, 2024-09-13T09:12:34.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0093095490000000003, 2024-09-13T09:12:34.387 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.387 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.387 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.387 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.388 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.388 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1551 2024-09-13T09:12:34.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.388 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.388 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.388 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329715+0000", 2024-09-13T09:12:34.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329715+0000", 2024-09-13T09:12:34.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329715+0000", 2024-09-13T09:12:34.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329718+0000", 2024-09-13T09:12:34.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8279999999999999e-06 2024-09-13T09:12:34.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329726+0000", 2024-09-13T09:12:34.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0420000000000003e-06 2024-09-13T09:12:34.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329729+0000", 2024-09-13T09:12:34.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1099999999999999e-06 2024-09-13T09:12:34.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.333843+0000", 2024-09-13T09:12:34.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0041141499999999996 2024-09-13T09:12:34.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.333899+0000", 2024-09-13T09:12:34.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6273000000000001e-05 2024-09-13T09:12:34.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.392 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.392 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.334107+0000", 2024-09-13T09:12:34.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00020787400000000001 2024-09-13T09:12:34.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.393 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338610+0000", 2024-09-13T09:12:34.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004502944 2024-09-13T09:12:34.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.393 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338943+0000", 2024-09-13T09:12:34.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033260000000000001 2024-09-13T09:12:34.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338980+0000", 2024-09-13T09:12:34.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7382999999999998e-05 2024-09-13T09:12:34.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.339025+0000", 2024-09-13T09:12:34.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4765999999999998e-05 2024-09-13T09:12:34.394 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.394 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.394 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.394 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1990 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.395 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.504086+0000", 2024-09-13T09:12:34.395 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.8619614930000008, 2024-09-13T09:12:34.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0098999020000000004, 2024-09-13T09:12:34.395 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.395 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.395 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.395 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.395 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.395 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1990 2024-09-13T09:12:34.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.396 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.396 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504086+0000", 2024-09-13T09:12:34.396 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.396 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504086+0000", 2024-09-13T09:12:34.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.397 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504086+0000", 2024-09-13T09:12:34.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.398 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.398 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504090+0000", 2024-09-13T09:12:34.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000004e-06 2024-09-13T09:12:34.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.398 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.398 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504097+0000", 2024-09-13T09:12:34.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4630000000000004e-06 2024-09-13T09:12:34.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504103+0000", 2024-09-13T09:12:34.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4589999999999997e-06 2024-09-13T09:12:34.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.505791+0000", 2024-09-13T09:12:34.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0016881260000000001 2024-09-13T09:12:34.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.400 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.505955+0000", 2024-09-13T09:12:34.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016378200000000001 2024-09-13T09:12:34.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.400 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.506366+0000", 2024-09-13T09:12:34.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000411667 2024-09-13T09:12:34.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.401 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.401 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.513522+0000", 2024-09-13T09:12:34.401 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071554360000000003 2024-09-13T09:12:34.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.401 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.401 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.513662+0000", 2024-09-13T09:12:34.401 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000140195 2024-09-13T09:12:34.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.402 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.402 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.513949+0000", 2024-09-13T09:12:34.402 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000286908 2024-09-13T09:12:34.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.402 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.402 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.513986+0000", 2024-09-13T09:12:34.402 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7444999999999998e-05 2024-09-13T09:12:34.402 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.403 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.403 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.403 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1991 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.403 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.504108+0000", 2024-09-13T09:12:34.403 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.8619396110000004, 2024-09-13T09:12:34.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011285132, 2024-09-13T09:12:34.403 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.403 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.404 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.404 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.404 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.404 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1991 2024-09-13T09:12:34.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.404 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.404 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.404 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504108+0000", 2024-09-13T09:12:34.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.405 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504108+0000", 2024-09-13T09:12:34.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.405 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504108+0000", 2024-09-13T09:12:34.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504111+0000", 2024-09-13T09:12:34.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4070000000000002e-06 2024-09-13T09:12:34.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504117+0000", 2024-09-13T09:12:34.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:12:34.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504123+0000", 2024-09-13T09:12:34.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:12:34.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507071+0000", 2024-09-13T09:12:34.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029475579999999999 2024-09-13T09:12:34.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507151+0000", 2024-09-13T09:12:34.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0000000000000007e-05 2024-09-13T09:12:34.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507556+0000", 2024-09-13T09:12:34.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00040499900000000002 2024-09-13T09:12:34.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515305+0000", 2024-09-13T09:12:34.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0077488820000000003 2024-09-13T09:12:34.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515312+0000", 2024-09-13T09:12:34.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1640000000000004e-06 2024-09-13T09:12:34.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515358+0000", 2024-09-13T09:12:34.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.604e-05 2024-09-13T09:12:34.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515393+0000", 2024-09-13T09:12:34.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5670999999999998e-05 2024-09-13T09:12:34.410 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.410 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.410 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.411 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1992 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.411 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.504132+0000", 2024-09-13T09:12:34.411 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.8619157000000008, 2024-09-13T09:12:34.411 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011378241000000001, 2024-09-13T09:12:34.411 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.411 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.411 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.411 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.411 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.411 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1992 2024-09-13T09:12:34.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.412 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504132+0000", 2024-09-13T09:12:34.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504132+0000", 2024-09-13T09:12:34.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504132+0000", 2024-09-13T09:12:34.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504135+0000", 2024-09-13T09:12:34.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4580000000000002e-06 2024-09-13T09:12:34.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504143+0000", 2024-09-13T09:12:34.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0590000000000003e-06 2024-09-13T09:12:34.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504149+0000", 2024-09-13T09:12:34.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7649999999999999e-06 2024-09-13T09:12:34.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.415 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.508003+0000", 2024-09-13T09:12:34.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003854176 2024-09-13T09:12:34.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.415 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.508075+0000", 2024-09-13T09:12:34.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2197e-05 2024-09-13T09:12:34.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.415 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.508374+0000", 2024-09-13T09:12:34.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00029824099999999998 2024-09-13T09:12:34.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515402+0000", 2024-09-13T09:12:34.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0070279380000000001 2024-09-13T09:12:34.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.417 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515437+0000", 2024-09-13T09:12:34.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5274000000000001e-05 2024-09-13T09:12:34.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.417 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.417 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515475+0000", 2024-09-13T09:12:34.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8296000000000001e-05 2024-09-13T09:12:34.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.417 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.417 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515510+0000", 2024-09-13T09:12:34.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5261e-05 2024-09-13T09:12:34.417 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.418 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.418 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.418 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1993 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.418 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.504155+0000", 2024-09-13T09:12:34.418 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.8618926709999997, 2024-09-13T09:12:34.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01145854, 2024-09-13T09:12:34.418 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.418 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.418 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.418 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.418 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.419 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1993 2024-09-13T09:12:34.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.419 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504155+0000", 2024-09-13T09:12:34.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504155+0000", 2024-09-13T09:12:34.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504155+0000", 2024-09-13T09:12:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504158+0000", 2024-09-13T09:12:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3100000000000001e-06 2024-09-13T09:12:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504165+0000", 2024-09-13T09:12:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7519999999999999e-06 2024-09-13T09:12:34.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504170+0000", 2024-09-13T09:12:34.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5350000000000004e-06 2024-09-13T09:12:34.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.508797+0000", 2024-09-13T09:12:34.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046267319999999997 2024-09-13T09:12:34.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.508893+0000", 2024-09-13T09:12:34.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5356e-05 2024-09-13T09:12:34.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.509224+0000", 2024-09-13T09:12:34.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033109799999999999 2024-09-13T09:12:34.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515406+0000", 2024-09-13T09:12:34.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061827640000000003 2024-09-13T09:12:34.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515544+0000", 2024-09-13T09:12:34.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013761599999999999 2024-09-13T09:12:34.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515581+0000", 2024-09-13T09:12:34.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6902e-05 2024-09-13T09:12:34.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515614+0000", 2024-09-13T09:12:34.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2848000000000001e-05 2024-09-13T09:12:34.423 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.423 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.424 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.424 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2156 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.424 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.568613+0000", 2024-09-13T09:12:34.424 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.7974349049999994, 2024-09-13T09:12:34.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0087999440000000005, 2024-09-13T09:12:34.424 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.424 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.424 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.424 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.424 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.424 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2156 2024-09-13T09:12:34.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.425 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568613+0000", 2024-09-13T09:12:34.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568613+0000", 2024-09-13T09:12:34.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568612+0000", 2024-09-13T09:12:34.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568616+0000", 2024-09-13T09:12:34.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9380000000000002e-06 2024-09-13T09:12:34.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568625+0000", 2024-09-13T09:12:34.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5080000000000005e-06 2024-09-13T09:12:34.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568630+0000", 2024-09-13T09:12:34.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2920000000000003e-06 2024-09-13T09:12:34.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.571451+0000", 2024-09-13T09:12:34.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028206749999999999 2024-09-13T09:12:34.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.571530+0000", 2024-09-13T09:12:34.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9571000000000001e-05 2024-09-13T09:12:34.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.571881+0000", 2024-09-13T09:12:34.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035083500000000002 2024-09-13T09:12:34.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.575089+0000", 2024-09-13T09:12:34.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003207831 2024-09-13T09:12:34.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.577325+0000", 2024-09-13T09:12:34.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022359139999999999 2024-09-13T09:12:34.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.577374+0000", 2024-09-13T09:12:34.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8667e-05 2024-09-13T09:12:34.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.577413+0000", 2024-09-13T09:12:34.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9211000000000001e-05 2024-09-13T09:12:34.429 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.429 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.430 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.430 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2280 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.430 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.614829+0000", 2024-09-13T09:12:34.430 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.7512191389999998, 2024-09-13T09:12:34.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010493406, 2024-09-13T09:12:34.430 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.430 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.430 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.430 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.430 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.430 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2280 2024-09-13T09:12:34.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.431 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614829+0000", 2024-09-13T09:12:34.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614829+0000", 2024-09-13T09:12:34.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614828+0000", 2024-09-13T09:12:34.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614848+0000", 2024-09-13T09:12:34.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9653e-05 2024-09-13T09:12:34.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614880+0000", 2024-09-13T09:12:34.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2008999999999998e-05 2024-09-13T09:12:34.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614887+0000", 2024-09-13T09:12:34.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7750000000000004e-06 2024-09-13T09:12:34.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617693+0000", 2024-09-13T09:12:34.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028061119999999999 2024-09-13T09:12:34.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617770+0000", 2024-09-13T09:12:34.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7286999999999994e-05 2024-09-13T09:12:34.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618140+0000", 2024-09-13T09:12:34.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000370355 2024-09-13T09:12:34.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.622954+0000", 2024-09-13T09:12:34.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048138829999999997 2024-09-13T09:12:34.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625235+0000", 2024-09-13T09:12:34.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022808160000000002 2024-09-13T09:12:34.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625289+0000", 2024-09-13T09:12:34.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3604999999999999e-05 2024-09-13T09:12:34.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625322+0000", 2024-09-13T09:12:34.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3491999999999997e-05 2024-09-13T09:12:34.436 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.437 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.437 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.437 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2281 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.437 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.614893+0000", 2024-09-13T09:12:34.437 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.7511551329999993, 2024-09-13T09:12:34.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010532901000000001, 2024-09-13T09:12:34.437 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.437 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.437 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.437 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.438 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.438 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2281 2024-09-13T09:12:34.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.438 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.438 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.438 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614893+0000", 2024-09-13T09:12:34.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614893+0000", 2024-09-13T09:12:34.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614892+0000", 2024-09-13T09:12:34.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614900+0000", 2024-09-13T09:12:34.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2119999999999997e-06 2024-09-13T09:12:34.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614924+0000", 2024-09-13T09:12:34.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4260000000000002e-05 2024-09-13T09:12:34.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614934+0000", 2024-09-13T09:12:34.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7659999999999993e-06 2024-09-13T09:12:34.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619174+0000", 2024-09-13T09:12:34.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042408890000000003 2024-09-13T09:12:34.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.442 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.442 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619255+0000", 2024-09-13T09:12:34.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0189000000000001e-05 2024-09-13T09:12:34.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.442 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.442 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619609+0000", 2024-09-13T09:12:34.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035430500000000002 2024-09-13T09:12:34.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.443 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.622991+0000", 2024-09-13T09:12:34.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033823220000000001 2024-09-13T09:12:34.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.443 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625333+0000", 2024-09-13T09:12:34.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023419309999999998 2024-09-13T09:12:34.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.444 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625385+0000", 2024-09-13T09:12:34.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2194999999999997e-05 2024-09-13T09:12:34.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.444 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625426+0000", 2024-09-13T09:12:34.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0278999999999998e-05 2024-09-13T09:12:34.445 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.445 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.445 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.445 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2282 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.445 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.614942+0000", 2024-09-13T09:12:34.445 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.7511060119999993, 2024-09-13T09:12:34.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010565567999999999, 2024-09-13T09:12:34.445 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.446 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.446 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.446 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.446 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.446 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2282 2024-09-13T09:12:34.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.446 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614942+0000", 2024-09-13T09:12:34.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614942+0000", 2024-09-13T09:12:34.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614941+0000", 2024-09-13T09:12:34.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614947+0000", 2024-09-13T09:12:34.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3789999999999997e-06 2024-09-13T09:12:34.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614956+0000", 2024-09-13T09:12:34.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2969999999999992e-06 2024-09-13T09:12:34.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614961+0000", 2024-09-13T09:12:34.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:12:34.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620209+0000", 2024-09-13T09:12:34.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052470429999999998 2024-09-13T09:12:34.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620286+0000", 2024-09-13T09:12:34.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7061e-05 2024-09-13T09:12:34.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.450 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620646+0000", 2024-09-13T09:12:34.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000360164 2024-09-13T09:12:34.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.450 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.623024+0000", 2024-09-13T09:12:34.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002378186 2024-09-13T09:12:34.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625434+0000", 2024-09-13T09:12:34.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0024102630000000002 2024-09-13T09:12:34.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625475+0000", 2024-09-13T09:12:34.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0867e-05 2024-09-13T09:12:34.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625507+0000", 2024-09-13T09:12:34.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2394000000000002e-05 2024-09-13T09:12:34.451 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.452 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.452 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.452 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2283 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.452 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.614977+0000", 2024-09-13T09:12:34.452 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.7510712939999991, 2024-09-13T09:12:34.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011508387, 2024-09-13T09:12:34.452 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.452 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.452 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.452 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.453 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.453 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2283 2024-09-13T09:12:34.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.453 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614977+0000", 2024-09-13T09:12:34.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614977+0000", 2024-09-13T09:12:34.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614976+0000", 2024-09-13T09:12:34.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.455 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.455 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614980+0000", 2024-09-13T09:12:34.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8659999999999999e-06 2024-09-13T09:12:34.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.455 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614997+0000", 2024-09-13T09:12:34.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6946e-05 2024-09-13T09:12:34.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.456 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615003+0000", 2024-09-13T09:12:34.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4829999999999996e-06 2024-09-13T09:12:34.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.621063+0000", 2024-09-13T09:12:34.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0060601860000000004 2024-09-13T09:12:34.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.621415+0000", 2024-09-13T09:12:34.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035131399999999998 2024-09-13T09:12:34.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.621769+0000", 2024-09-13T09:12:34.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035409500000000002 2024-09-13T09:12:34.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625219+0000", 2024-09-13T09:12:34.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034497970000000001 2024-09-13T09:12:34.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626400+0000", 2024-09-13T09:12:34.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011811910000000001 2024-09-13T09:12:34.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626448+0000", 2024-09-13T09:12:34.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8615000000000001e-05 2024-09-13T09:12:34.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626485+0000", 2024-09-13T09:12:34.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6491e-05 2024-09-13T09:12:34.460 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.460 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.460 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.460 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2286 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.460 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.615037+0000", 2024-09-13T09:12:34.461 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.7510112239999991, 2024-09-13T09:12:34.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013941546000000001, 2024-09-13T09:12:34.461 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.461 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.461 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.461 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.461 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.461 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2286 2024-09-13T09:12:34.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.461 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615037+0000", 2024-09-13T09:12:34.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615037+0000", 2024-09-13T09:12:34.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615036+0000", 2024-09-13T09:12:34.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.463 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615039+0000", 2024-09-13T09:12:34.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1829999999999998e-06 2024-09-13T09:12:34.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.463 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615047+0000", 2024-09-13T09:12:34.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:12:34.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.464 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615052+0000", 2024-09-13T09:12:34.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2780000000000001e-06 2024-09-13T09:12:34.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.464 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.623049+0000", 2024-09-13T09:12:34.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0079966499999999992 2024-09-13T09:12:34.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.623128+0000", 2024-09-13T09:12:34.465 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9531999999999997e-05 2024-09-13T09:12:34.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.623468+0000", 2024-09-13T09:12:34.465 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033982700000000001 2024-09-13T09:12:34.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.466 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625551+0000", 2024-09-13T09:12:34.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020836140000000001 2024-09-13T09:12:34.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.466 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628931+0000", 2024-09-13T09:12:34.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033793709999999999 2024-09-13T09:12:34.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628958+0000", 2024-09-13T09:12:34.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6928e-05 2024-09-13T09:12:34.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.467 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628978+0000", 2024-09-13T09:12:34.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0426999999999999e-05 2024-09-13T09:12:34.467 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.468 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.468 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.468 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2293 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.468 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.616955+0000", 2024-09-13T09:12:34.468 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.7490926309999999, 2024-09-13T09:12:34.468 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014655674, 2024-09-13T09:12:34.468 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.468 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.468 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.468 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.468 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.469 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2293 2024-09-13T09:12:34.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.469 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.469 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.469 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616955+0000", 2024-09-13T09:12:34.469 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616955+0000", 2024-09-13T09:12:34.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616954+0000", 2024-09-13T09:12:34.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:34.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616960+0000", 2024-09-13T09:12:34.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1809999999999998e-06 2024-09-13T09:12:34.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616973+0000", 2024-09-13T09:12:34.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3001e-05 2024-09-13T09:12:34.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.472 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.472 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.472 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616981+0000", 2024-09-13T09:12:34.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5089999999999996e-06 2024-09-13T09:12:34.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.472 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.472 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.472 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625596+0000", 2024-09-13T09:12:34.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086152260000000001 2024-09-13T09:12:34.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.473 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.473 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.473 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625680+0000", 2024-09-13T09:12:34.473 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4098999999999996e-05 2024-09-13T09:12:34.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.473 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.473 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626054+0000", 2024-09-13T09:12:34.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00037370300000000002 2024-09-13T09:12:34.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.474 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.629114+0000", 2024-09-13T09:12:34.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0030600279999999998 2024-09-13T09:12:34.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631561+0000", 2024-09-13T09:12:34.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0024473009999999998 2024-09-13T09:12:34.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631590+0000", 2024-09-13T09:12:34.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9033000000000001e-05 2024-09-13T09:12:34.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631611+0000", 2024-09-13T09:12:34.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0665999999999998e-05 2024-09-13T09:12:34.476 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.476 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.476 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.477 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2297 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.477 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.617105+0000", 2024-09-13T09:12:34.477 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.7489432950000001, 2024-09-13T09:12:34.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01465442, 2024-09-13T09:12:34.477 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.477 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.477 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.477 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.477 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.477 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2297 2024-09-13T09:12:34.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.478 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.478 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617105+0000", 2024-09-13T09:12:34.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.478 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617105+0000", 2024-09-13T09:12:34.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617104+0000", 2024-09-13T09:12:34.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:34.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617109+0000", 2024-09-13T09:12:34.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7579999999999998e-06 2024-09-13T09:12:34.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617121+0000", 2024-09-13T09:12:34.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1439e-05 2024-09-13T09:12:34.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617137+0000", 2024-09-13T09:12:34.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6481999999999999e-05 2024-09-13T09:12:34.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626526+0000", 2024-09-13T09:12:34.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0093890499999999995 2024-09-13T09:12:34.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626617+0000", 2024-09-13T09:12:34.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0116000000000003e-05 2024-09-13T09:12:34.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626962+0000", 2024-09-13T09:12:34.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034533300000000002 2024-09-13T09:12:34.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.629177+0000", 2024-09-13T09:12:34.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022147529999999999 2024-09-13T09:12:34.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631706+0000", 2024-09-13T09:12:34.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025289829999999998 2024-09-13T09:12:34.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631735+0000", 2024-09-13T09:12:34.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9309e-05 2024-09-13T09:12:34.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631759+0000", 2024-09-13T09:12:34.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4066e-05 2024-09-13T09:12:34.484 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.484 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.484 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.485 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2298 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.485 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.617145+0000", 2024-09-13T09:12:34.485 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.7489029340000002, 2024-09-13T09:12:34.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014745108, 2024-09-13T09:12:34.485 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.485 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.485 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.485 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.486 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.486 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2298 2024-09-13T09:12:34.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.486 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617145+0000", 2024-09-13T09:12:34.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617145+0000", 2024-09-13T09:12:34.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617144+0000", 2024-09-13T09:12:34.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617148+0000", 2024-09-13T09:12:34.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6830000000000001e-06 2024-09-13T09:12:34.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617156+0000", 2024-09-13T09:12:34.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1610000000000003e-06 2024-09-13T09:12:34.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617162+0000", 2024-09-13T09:12:34.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1600000000000003e-06 2024-09-13T09:12:34.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.627419+0000", 2024-09-13T09:12:34.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010256517 2024-09-13T09:12:34.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.627516+0000", 2024-09-13T09:12:34.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7600000000000001e-05 2024-09-13T09:12:34.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.627872+0000", 2024-09-13T09:12:34.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035532400000000002 2024-09-13T09:12:34.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631665+0000", 2024-09-13T09:12:34.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037928200000000001 2024-09-13T09:12:34.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631771+0000", 2024-09-13T09:12:34.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000106288 2024-09-13T09:12:34.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631807+0000", 2024-09-13T09:12:34.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5695999999999998e-05 2024-09-13T09:12:34.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631890+0000", 2024-09-13T09:12:34.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3473000000000005e-05 2024-09-13T09:12:34.492 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.492 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.492 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.492 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2308 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.492 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.618672+0000", 2024-09-13T09:12:34.492 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.7473756379999994, 2024-09-13T09:12:34.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013301396, 2024-09-13T09:12:34.493 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.493 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.493 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.493 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.493 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.493 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2308 2024-09-13T09:12:34.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.493 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618672+0000", 2024-09-13T09:12:34.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618672+0000", 2024-09-13T09:12:34.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618672+0000", 2024-09-13T09:12:34.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618681+0000", 2024-09-13T09:12:34.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3819999999999993e-06 2024-09-13T09:12:34.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618690+0000", 2024-09-13T09:12:34.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2420000000000006e-06 2024-09-13T09:12:34.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618696+0000", 2024-09-13T09:12:34.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2859999999999996e-06 2024-09-13T09:12:34.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628271+0000", 2024-09-13T09:12:34.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.009574249 2024-09-13T09:12:34.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628314+0000", 2024-09-13T09:12:34.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3467999999999998e-05 2024-09-13T09:12:34.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628672+0000", 2024-09-13T09:12:34.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000357726 2024-09-13T09:12:34.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631699+0000", 2024-09-13T09:12:34.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003026759 2024-09-13T09:12:34.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631902+0000", 2024-09-13T09:12:34.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000202862 2024-09-13T09:12:34.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631943+0000", 2024-09-13T09:12:34.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1084000000000002e-05 2024-09-13T09:12:34.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631974+0000", 2024-09-13T09:12:34.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1007000000000003e-05 2024-09-13T09:12:34.500 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.500 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.500 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.500 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2309 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.500 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.619333+0000", 2024-09-13T09:12:34.500 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.7467149749999997, 2024-09-13T09:12:34.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012775455999999999, 2024-09-13T09:12:34.501 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.501 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.501 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.501 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.501 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.501 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2309 2024-09-13T09:12:34.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.501 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619333+0000", 2024-09-13T09:12:34.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619333+0000", 2024-09-13T09:12:34.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619332+0000", 2024-09-13T09:12:34.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:34.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619338+0000", 2024-09-13T09:12:34.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.648e-06 2024-09-13T09:12:34.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619349+0000", 2024-09-13T09:12:34.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1545e-05 2024-09-13T09:12:34.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619367+0000", 2024-09-13T09:12:34.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7419e-05 2024-09-13T09:12:34.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.629223+0000", 2024-09-13T09:12:34.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0098561680000000002 2024-09-13T09:12:34.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.629311+0000", 2024-09-13T09:12:34.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8057000000000001e-05 2024-09-13T09:12:34.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.630009+0000", 2024-09-13T09:12:34.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00069777100000000002 2024-09-13T09:12:34.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631985+0000", 2024-09-13T09:12:34.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019758919999999999 2024-09-13T09:12:34.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.632022+0000", 2024-09-13T09:12:34.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7410000000000003e-05 2024-09-13T09:12:34.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.632075+0000", 2024-09-13T09:12:34.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3248000000000002e-05 2024-09-13T09:12:34.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.632108+0000", 2024-09-13T09:12:34.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3191000000000001e-05 2024-09-13T09:12:34.507 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.507 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.507 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.507 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2310 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:34.507 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.619384+0000", 2024-09-13T09:12:34.507 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.7466639540000006, 2024-09-13T09:12:34.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014600315000000001, 2024-09-13T09:12:34.507 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.507 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:34.507 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.508 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:34.508 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:34.508 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2310 2024-09-13T09:12:34.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.508 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619384+0000", 2024-09-13T09:12:34.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619384+0000", 2024-09-13T09:12:34.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619383+0000", 2024-09-13T09:12:34.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619387+0000", 2024-09-13T09:12:34.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1200000000000004e-06 2024-09-13T09:12:34.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619398+0000", 2024-09-13T09:12:34.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0841e-05 2024-09-13T09:12:34.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619404+0000", 2024-09-13T09:12:34.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8329999999999999e-06 2024-09-13T09:12:34.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.630863+0000", 2024-09-13T09:12:34.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011459278 2024-09-13T09:12:34.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.630971+0000", 2024-09-13T09:12:34.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010728100000000001 2024-09-13T09:12:34.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:34.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631296+0000", 2024-09-13T09:12:34.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00032547400000000002 2024-09-13T09:12:34.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:34.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.632133+0000", 2024-09-13T09:12:34.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00083720599999999995 2024-09-13T09:12:34.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:34.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.633830+0000", 2024-09-13T09:12:34.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001696342 2024-09-13T09:12:34.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:34.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.633950+0000", 2024-09-13T09:12:34.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120444 2024-09-13T09:12:34.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.633984+0000", 2024-09-13T09:12:34.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4082000000000003e-05 2024-09-13T09:12:34.513 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.513 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.513 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.513 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4816.0:719 3.1f 3:f983ed45:::rbd_data.1123df4df335.0000000000000008:d [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T09:12:34.513 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:34.218407+0000", 2024-09-13T09:12:34.513 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.14764090099999999, 2024-09-13T09:12:34.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01050302, 2024-09-13T09:12:34.514 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.514 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:34.514 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:34.514 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4816", 2024-09-13T09:12:34.514 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2210529289", 2024-09-13T09:12:34.514 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 719 2024-09-13T09:12:34.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.514 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.514 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.514 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.218407+0000", 2024-09-13T09:12:34.514 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.218407+0000", 2024-09-13T09:12:34.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.218406+0000", 2024-09-13T09:12:34.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:34.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.218409+0000", 2024-09-13T09:12:34.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7219999999999999e-06 2024-09-13T09:12:34.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.218416+0000", 2024-09-13T09:12:34.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:12:34.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.218422+0000", 2024-09-13T09:12:34.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.455e-06 2024-09-13T09:12:34.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.226431+0000", 2024-09-13T09:12:34.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0080093630000000002 2024-09-13T09:12:34.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.226484+0000", 2024-09-13T09:12:34.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2258e-05 2024-09-13T09:12:34.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:34.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.228910+0000", 2024-09-13T09:12:34.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00242625 2024-09-13T09:12:34.517 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.517 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.517 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.518 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.518 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.518 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:34.520 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:34.522 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-09-13T09:12:34.522 INFO:teuthology.orchestra.run.smithi157.stdout:1326 trunc from 0xbe32c00 to 0x1ae2a00 2024-09-13T09:12:34.522 INFO:teuthology.orchestra.run.smithi157.stdout:1328 trunc from 0x1ae2a00 to 0x6b0ea00 2024-09-13T09:12:34.522 INFO:teuthology.orchestra.run.smithi157.stdout:1329 write 0xc923000 thru 0xc92f4b8 (0xc4b9 bytes) 2024-09-13T09:12:34.522 INFO:teuthology.orchestra.run.smithi157.stdout:1330 trunc from 0xc92f600 to 0x440ee00 2024-09-13T09:12:34.522 INFO:teuthology.orchestra.run.smithi157.stdout:1331 write 0xd776000 thru 0xd77fffa (0x9ffb bytes) 2024-09-13T09:12:34.522 INFO:teuthology.orchestra.run.smithi157.stdout:1333 read 0xcdb200 thru 0xcea5dc (0xf3dd bytes) 2024-09-13T09:12:34.522 INFO:teuthology.orchestra.run.smithi157.stdout:1336 write 0x72e3a00 thru 0x72f25b5 (0xebb6 bytes) 2024-09-13T09:12:34.522 INFO:teuthology.orchestra.run.smithi157.stdout:1343 write 0x5386600 thru 0x5395c4b (0xf64c bytes) 2024-09-13T09:12:34.522 INFO:teuthology.orchestra.run.smithi157.stdout:1345 read 0xb1b0400 thru 0xb1b1724 (0x1325 bytes) 2024-09-13T09:12:34.522 INFO:teuthology.orchestra.run.smithi157.stdout:1349 write 0xd2a9400 thru 0xd2b8054 (0xec55 bytes) 2024-09-13T09:12:34.523 INFO:teuthology.orchestra.run.smithi157.stdout:1351 write 0x1c8a200 thru 0x1c95a84 (0xb885 bytes) 2024-09-13T09:12:34.523 INFO:teuthology.orchestra.run.smithi157.stdout:1352 trunc from 0xd780000 to 0x5e37e00 2024-09-13T09:12:34.523 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:34.517+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:34.536 INFO:teuthology.orchestra.run.smithi157.stdout:1354 read 0x5d93800 thru 0x5d97a0b (0x420c bytes) 2024-09-13T09:12:34.544 INFO:teuthology.orchestra.run.smithi157.stdout:1357 read 0x4b18e00 thru 0x4b22907 (0x9b08 bytes) 2024-09-13T09:12:34.544 INFO:teuthology.orchestra.run.smithi157.stdout:1360 read 0xde0200 thru 0xded605 (0xd406 bytes) 2024-09-13T09:12:34.545 INFO:teuthology.orchestra.run.smithi157.stdout:1361 punch from 0x5a7e400 to 0x5a82200, (0x3e00 bytes) 2024-09-13T09:12:34.545 INFO:teuthology.orchestra.run.smithi157.stdout:1362 read 0x1821800 thru 0x1827d49 (0x654a bytes) 2024-09-13T09:12:34.546 INFO:teuthology.orchestra.run.smithi157.stdout:1365 read 0x317de00 thru 0x31844cd (0x66ce bytes) 2024-09-13T09:12:34.546 INFO:teuthology.orchestra.run.smithi157.stdout:1368 read 0x905000 thru 0x90f290 (0xa291 bytes) 2024-09-13T09:12:34.546 INFO:teuthology.orchestra.run.smithi157.stdout:1369 read 0x22a7400 thru 0x22b2ec3 (0xbac4 bytes) 2024-09-13T09:12:34.547 INFO:teuthology.orchestra.run.smithi157.stdout:1372 write 0x43400 thru 0x4e56d (0xb16e bytes) 2024-09-13T09:12:34.567 INFO:teuthology.orchestra.run.smithi157.stdout:1373 write 0xe33c000 thru 0xe33e4ab (0x24ac bytes) 2024-09-13T09:12:34.572 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:12:34.575 INFO:teuthology.orchestra.run.smithi157.stdout:1375 read 0x6b05e00 thru 0x6b13352 (0xd553 bytes) 2024-09-13T09:12:34.575 INFO:teuthology.orchestra.run.smithi157.stdout:1378 read 0x3880c00 thru 0x388a9c7 (0x9dc8 bytes) 2024-09-13T09:12:34.575 INFO:teuthology.orchestra.run.smithi157.stdout:1379 read 0xb0db800 thru 0xb0dcd96 (0x1597 bytes) 2024-09-13T09:12:34.575 INFO:teuthology.orchestra.run.smithi157.stdout:1380 punch from 0x6b8f200 to 0x6b99600, (0xa400 bytes) 2024-09-13T09:12:34.576 INFO:teuthology.orchestra.run.smithi157.stdout:1381 read 0xa29bc00 thru 0xa2a2a2c (0x6e2d bytes) 2024-09-13T09:12:34.576 INFO:teuthology.orchestra.run.smithi157.stdout:1383 punch from 0x69a6e00 to 0x69b1c00, (0xae00 bytes) 2024-09-13T09:12:34.576 INFO:teuthology.orchestra.run.smithi157.stdout:1384 punch from 0x9bf1c00 to 0x9bf2a00, (0xe00 bytes) 2024-09-13T09:12:34.576 INFO:teuthology.orchestra.run.smithi157.stdout:1385 read 0xb620200 thru 0xb62b773 (0xb574 bytes) 2024-09-13T09:12:34.585 INFO:teuthology.orchestra.run.smithi157.stdout:1386 trunc from 0xe33e600 to 0x5860c00 2024-09-13T09:12:34.601 INFO:teuthology.orchestra.run.smithi157.stdout:1387 punch from 0x2a25000 to 0x2a28800, (0x3800 bytes) 2024-09-13T09:12:34.602 INFO:teuthology.orchestra.run.smithi157.stdout:1390 punch from 0x5121200 to 0x5127800, (0x6600 bytes) 2024-09-13T09:12:34.609 INFO:teuthology.orchestra.run.smithi157.stdout:1393 trunc from 0x5860c00 to 0xc28c200 2024-09-13T09:12:34.618 INFO:teuthology.orchestra.run.smithi157.stdout:1395 read 0x6dc6c00 thru 0x6dd6459 (0xf85a bytes) 2024-09-13T09:12:34.618 INFO:teuthology.orchestra.run.smithi157.stdout:1396 read 0xb3d9400 thru 0xb3daca8 (0x18a9 bytes) 2024-09-13T09:12:34.618 INFO:teuthology.orchestra.run.smithi157.stdout:1398 read 0x7346400 thru 0x7354908 (0xe509 bytes) 2024-09-13T09:12:34.618 INFO:teuthology.orchestra.run.smithi157.stdout:1399 write 0x5ecfe00 thru 0x5ed6eb2 (0x70b3 bytes) 2024-09-13T09:12:34.619 INFO:teuthology.orchestra.run.smithi157.stdout:1403 read 0x40a5800 thru 0x40a842d (0x2c2e bytes) 2024-09-13T09:12:34.619 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:34.618+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:34.629 INFO:teuthology.orchestra.run.smithi157.stdout:1406 trunc from 0xc28c200 to 0x43c3a00 2024-09-13T09:12:34.643 INFO:teuthology.orchestra.run.smithi157.stdout:1410 read 0x1023c00 thru 0x1026e97 (0x3298 bytes) 2024-09-13T09:12:34.651 INFO:teuthology.orchestra.run.smithi157.stdout:1411 trunc from 0x43c3a00 to 0x9e40e00 2024-09-13T09:12:34.660 INFO:teuthology.orchestra.run.smithi157.stdout:1412 write 0x552d200 thru 0x553c01f (0xee20 bytes) 2024-09-13T09:12:34.661 INFO:teuthology.orchestra.run.smithi157.stdout:1414 write 0x2fd6c00 thru 0x2fd8acd (0x1ece bytes) 2024-09-13T09:12:34.662 INFO:teuthology.orchestra.run.smithi157.stdout:1415 punch from 0x5fb7a00 to 0x5fc3600, (0xbc00 bytes) 2024-09-13T09:12:34.662 INFO:teuthology.orchestra.run.smithi157.stdout:1416 read 0x87a3400 thru 0x87a8531 (0x5132 bytes) 2024-09-13T09:12:34.666 INFO:teuthology.orchestra.run.smithi157.stdout:1417 trunc from 0x9e40e00 to 0xd7f5c00 2024-09-13T09:12:34.678 INFO:teuthology.orchestra.run.smithi157.stdout:1419 write 0xc357600 thru 0xc35f4d3 (0x7ed4 bytes) 2024-09-13T09:12:34.679 INFO:teuthology.orchestra.run.smithi157.stdout:1420 read 0x4178400 thru 0x41827d5 (0xa3d6 bytes) 2024-09-13T09:12:34.679 INFO:teuthology.orchestra.run.smithi157.stdout:1421 punch from 0x51200 to 0x5de00, (0xcc00 bytes) 2024-09-13T09:12:34.679 INFO:teuthology.orchestra.run.smithi157.stdout:1423 write 0x8c2d400 thru 0x8c32ba1 (0x57a2 bytes) 2024-09-13T09:12:34.679 INFO:teuthology.orchestra.run.smithi157.stdout:1424 write 0x4d90400 thru 0x4d9951c (0x911d bytes) 2024-09-13T09:12:34.681 INFO:teuthology.orchestra.run.smithi157.stdout:1425 punch from 0xad67c00 to 0xad74a00, (0xce00 bytes) 2024-09-13T09:12:34.688 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:34.688 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:34.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.689 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4816.0:2322 3.12 e41/22 3:4b0dfad0:::rbd_data.1123b259dc65.000000000000002a:head v 41'34, mlcod=41'32)", 2024-09-13T09:12:34.689 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:34.687358+0000", 2024-09-13T09:12:34.689 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.000739863, 2024-09-13T09:12:34.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000756279, 2024-09-13T09:12:34.689 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:34.689 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:34.689 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:34.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:34.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.687358+0000", 2024-09-13T09:12:34.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.687358+0000", 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.687357+0000", 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.687373+0000", 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6195e-05 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.687387+0000", 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3295999999999999e-05 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.687394+0000", 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3769999999999998e-06 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.687439+0000", 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4807e-05 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:34.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:34.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.687458+0000", 2024-09-13T09:12:34.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8984e-05 2024-09-13T09:12:34.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.692 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:34.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:34.692 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:12:34.692 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 1 2024-09-13T09:12:34.692 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:34.692 INFO:teuthology.orchestra.run.smithi157.stdout:1426 trunc from 0xd7f5c00 to 0x5031e00 2024-09-13T09:12:34.700 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:34.720 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:34.718+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:34.727 INFO:teuthology.orchestra.run.smithi157.stdout:1427 write 0xb44d600 thru 0xb458161 (0xab62 bytes) 2024-09-13T09:12:34.728 INFO:teuthology.orchestra.run.smithi157.stdout:1431 punch from 0x38fe00 to 0x392000, (0x2200 bytes) 2024-09-13T09:12:34.732 INFO:teuthology.orchestra.run.smithi157.stdout:1434 punch from 0x6654000 to 0x665a600, (0x6600 bytes) 2024-09-13T09:12:34.743 INFO:teuthology.orchestra.run.smithi157.stdout:1435 trunc from 0xb458200 to 0x2486400 2024-09-13T09:12:34.749 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:34.750 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:34.768 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:34.768 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:34.769 INFO:teuthology.orchestra.run.smithi157.stdout:1436 trunc from 0x2486400 to 0x9797600 2024-09-13T09:12:34.779 INFO:teuthology.orchestra.run.smithi157.stdout:1439 punch from 0x2e76400 to 0x2e82c00, (0xc800 bytes) 2024-09-13T09:12:34.780 INFO:teuthology.orchestra.run.smithi157.stdout:1441 punch from 0x7d7c00 to 0x7e3000, (0xb400 bytes) 2024-09-13T09:12:34.780 INFO:teuthology.orchestra.run.smithi157.stdout:1443 write 0x2e08a00 thru 0x2e0a091 (0x1692 bytes) 2024-09-13T09:12:34.782 INFO:teuthology.orchestra.run.smithi157.stdout:1446 read 0x3c62000 thru 0x3c6c596 (0xa597 bytes) 2024-09-13T09:12:34.782 INFO:teuthology.orchestra.run.smithi157.stdout:1447 punch from 0x4106a00 to 0x4115600, (0xec00 bytes) 2024-09-13T09:12:34.782 INFO:teuthology.orchestra.run.smithi157.stdout:1448 read 0xb44800 thru 0xb5333b (0xeb3c bytes) 2024-09-13T09:12:34.783 INFO:teuthology.orchestra.run.smithi157.stdout:1451 read 0x216ae00 thru 0x2177e9a (0xd09b bytes) 2024-09-13T09:12:34.783 INFO:teuthology.orchestra.run.smithi157.stdout:1452 read 0x1edf400 thru 0x1ee3acf (0x46d0 bytes) 2024-09-13T09:12:34.784 INFO:teuthology.orchestra.run.smithi157.stdout:1453 write 0x38b1c00 thru 0x38b57a8 (0x3ba9 bytes) 2024-09-13T09:12:34.789 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:34.789 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:34.801 INFO:teuthology.orchestra.run.smithi157.stdout:1455 write 0xb57ce00 thru 0xb57dfbf (0x11c0 bytes) 2024-09-13T09:12:34.804 INFO:teuthology.orchestra.run.smithi157.stdout:1456 trunc from 0xb57e000 to 0xd3b2c00 2024-09-13T09:12:34.807 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:34.807 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:34.814 INFO:teuthology.orchestra.run.smithi157.stdout:1457 write 0x41fde00 thru 0x41fff94 (0x2195 bytes) 2024-09-13T09:12:34.820 INFO:teuthology.orchestra.run.smithi157.stdout:1459 read 0xac7000 thru 0xad4647 (0xd648 bytes) 2024-09-13T09:12:34.820 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:34.820+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:34.821 INFO:teuthology.orchestra.run.smithi157.stdout:1461 punch from 0x5c97a00 to 0x5c9f600, (0x7c00 bytes) 2024-09-13T09:12:34.821 INFO:teuthology.orchestra.run.smithi157.stdout:1462 punch from 0xb2dc200 to 0xb2e6a00, (0xa800 bytes) 2024-09-13T09:12:34.821 INFO:teuthology.orchestra.run.smithi157.stdout:1469 read 0xcf61800 thru 0xcf6934e (0x7b4f bytes) 2024-09-13T09:12:34.822 INFO:teuthology.orchestra.run.smithi157.stdout:1470 write 0x56d4e00 thru 0x56e05be (0xb7bf bytes) 2024-09-13T09:12:34.822 INFO:teuthology.orchestra.run.smithi157.stdout:1472 write 0x6c70600 thru 0x6c7e0fe (0xdaff bytes) 2024-09-13T09:12:34.822 INFO:teuthology.orchestra.run.smithi157.stdout:1473 punch from 0xa34c000 to 0xa35b800, (0xf800 bytes) 2024-09-13T09:12:34.822 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:34.823 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:34.823 INFO:teuthology.orchestra.run.smithi157.stdout:1476 read 0x6ea9200 thru 0x6ea95fc (0x3fd bytes) 2024-09-13T09:12:34.823 INFO:teuthology.orchestra.run.smithi157.stdout:1477 read 0x63fd000 thru 0x6401720 (0x4721 bytes) 2024-09-13T09:12:34.823 INFO:teuthology.orchestra.run.smithi157.stdout:1478 trunc from 0xd3b2c00 to 0xc802800 2024-09-13T09:12:34.826 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:34.826 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:34.836 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:34.847 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:34.847 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:34.848 INFO:teuthology.orchestra.run.smithi157.stdout:1480 write 0x2f64000 thru 0x2f6e177 (0xa178 bytes) 2024-09-13T09:12:34.854 INFO:teuthology.orchestra.run.smithi157.stdout:1481 read 0x80bd400 thru 0x80c2da7 (0x59a8 bytes) 2024-09-13T09:12:34.854 INFO:teuthology.orchestra.run.smithi157.stdout:1482 read 0x5efda00 thru 0x5f0d2a9 (0xf8aa bytes) 2024-09-13T09:12:34.854 INFO:teuthology.orchestra.run.smithi157.stdout:1486 write 0x498a400 thru 0x4992539 (0x813a bytes) 2024-09-13T09:12:34.855 INFO:teuthology.orchestra.run.smithi157.stdout:1488 read 0x32a1800 thru 0x32a856a (0x6d6b bytes) 2024-09-13T09:12:34.856 INFO:teuthology.orchestra.run.smithi157.stdout:1490 write 0x87ff200 thru 0x880adaa (0xbbab bytes) 2024-09-13T09:12:34.856 INFO:teuthology.orchestra.run.smithi157.stdout:1492 write 0x7763600 thru 0x77670f2 (0x3af3 bytes) 2024-09-13T09:12:34.856 INFO:teuthology.orchestra.run.smithi157.stdout:1494 punch from 0xad52000 to 0xad54c00, (0x2c00 bytes) 2024-09-13T09:12:34.856 INFO:teuthology.orchestra.run.smithi157.stdout:1495 write 0x8ed9c00 thru 0x8eddf93 (0x4394 bytes) 2024-09-13T09:12:34.857 INFO:teuthology.orchestra.run.smithi157.stdout:1496 read 0x8c4be00 thru 0x8c5aa9d (0xec9e bytes) 2024-09-13T09:12:34.876 INFO:teuthology.orchestra.run.smithi157.stdout:1497 write 0xe5ab000 thru 0xe5b405a (0x905b bytes) 2024-09-13T09:12:34.883 INFO:teuthology.orchestra.run.smithi157.stdout:1499 read 0xc4cf200 thru 0xc4d69e8 (0x77e9 bytes) 2024-09-13T09:12:34.883 INFO:teuthology.orchestra.run.smithi157.stdout:1501 punch from 0x71b7200 to 0x71c1c00, (0xaa00 bytes) 2024-09-13T09:12:34.883 INFO:teuthology.orchestra.run.smithi157.stdout:1503 write 0x5f5f200 thru 0x5f66a63 (0x7864 bytes) 2024-09-13T09:12:34.889 INFO:teuthology.orchestra.run.smithi157.stdout:1505 trunc from 0xe5b4200 to 0x9e9b600 2024-09-13T09:12:34.921 INFO:teuthology.orchestra.run.smithi157.stdout:1506 trunc from 0x9e9b600 to 0x1f29000 2024-09-13T09:12:34.921 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:34.919+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:34.942 INFO:teuthology.orchestra.run.smithi157.stdout:1507 read 0xc8b600 thru 0xc93afb (0x84fc bytes) 2024-09-13T09:12:34.947 INFO:teuthology.orchestra.run.smithi157.stdout:1508 read 0x3400 thru 0x7253 (0x3e54 bytes) 2024-09-13T09:12:34.948 INFO:teuthology.orchestra.run.smithi157.stdout:1509 punch from 0x1583200 to 0x1587400, (0x4200 bytes) 2024-09-13T09:12:34.948 INFO:teuthology.orchestra.run.smithi157.stdout:1510 read 0x137d600 thru 0x137f3c8 (0x1dc9 bytes) 2024-09-13T09:12:34.956 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:34.956 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:34.966 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:34.967 INFO:teuthology.orchestra.run.smithi157.stdout:1511 write 0x9f17600 thru 0x9f269e8 (0xf3e9 bytes) 2024-09-13T09:12:34.967 INFO:teuthology.orchestra.run.smithi157.stdout:1512 write 0x630be00 thru 0x631a61c (0xe81d bytes) 2024-09-13T09:12:34.967 INFO:teuthology.orchestra.run.smithi157.stdout:1514 write 0x2a3c600 thru 0x2a41668 (0x5069 bytes) 2024-09-13T09:12:34.967 INFO:teuthology.orchestra.run.smithi157.stdout:1517 read 0xd15c00 thru 0xd19316 (0x3717 bytes) 2024-09-13T09:12:34.968 INFO:teuthology.orchestra.run.smithi157.stdout:1523 write 0x9e3de00 thru 0x9e43d25 (0x5f26 bytes) 2024-09-13T09:12:34.994 INFO:teuthology.orchestra.run.smithi157.stdout:1524 write 0xd368e00 thru 0xd372f3c (0xa13d bytes) 2024-09-13T09:12:34.995 INFO:teuthology.orchestra.run.smithi157.stdout:1525 read 0x590400 thru 0x59a78b (0xa38c bytes) 2024-09-13T09:12:34.996 INFO:teuthology.orchestra.run.smithi157.stdout:1526 punch from 0x8e1b200 to 0x8e23600, (0x8400 bytes) 2024-09-13T09:12:34.997 INFO:teuthology.orchestra.run.smithi157.stdout:1527 trunc from 0xd373000 to 0xe92a200 2024-09-13T09:12:35.031 INFO:teuthology.orchestra.run.smithi157.stdout:1529 trunc from 0xe92a200 to 0x2d53c00 2024-09-13T09:12:35.049 INFO:teuthology.orchestra.run.smithi157.stdout:1530 punch from 0x279e400 to 0x27ab000, (0xcc00 bytes) 2024-09-13T09:12:35.050 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:35.048+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:35.051 INFO:teuthology.orchestra.run.smithi157.stdout:1531 punch from 0x1ea5a00 to 0x1eb0a00, (0xb000 bytes) 2024-09-13T09:12:35.079 INFO:teuthology.orchestra.run.smithi157.stdout:1533 write 0xe4b4800 thru 0xe4bef49 (0xa74a bytes) 2024-09-13T09:12:35.079 INFO:teuthology.orchestra.run.smithi157.stdout:1534 read 0x464f200 thru 0x46545ec (0x53ed bytes) 2024-09-13T09:12:35.080 INFO:teuthology.orchestra.run.smithi157.stdout:1536 punch from 0xc57d400 to 0xc588200, (0xae00 bytes) 2024-09-13T09:12:35.080 INFO:teuthology.orchestra.run.smithi157.stdout:1538 punch from 0x13f0800 to 0x13f3a00, (0x3200 bytes) 2024-09-13T09:12:35.080 INFO:teuthology.orchestra.run.smithi157.stdout:1539 read 0x843ee00 thru 0x8448942 (0x9b43 bytes) 2024-09-13T09:12:35.092 INFO:teuthology.orchestra.run.smithi157.stdout:1540 trunc from 0xe4bf000 to 0x3646000 2024-09-13T09:12:35.115 INFO:teuthology.orchestra.run.smithi157.stdout:1542 trunc from 0x3646000 to 0x2a86800 2024-09-13T09:12:35.122 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:35.123 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:35.123 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:35.121+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:35.132 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:35.138 INFO:teuthology.orchestra.run.smithi157.stdout:1543 write 0x3a4b800 thru 0x3a4da7d (0x227e bytes) 2024-09-13T09:12:35.138 INFO:teuthology.orchestra.run.smithi157.stdout:1546 punch from 0x559000 to 0x55ba00, (0x2a00 bytes) 2024-09-13T09:12:35.139 INFO:teuthology.orchestra.run.smithi157.stdout:1547 punch from 0x26d6400 to 0x26e2e00, (0xca00 bytes) 2024-09-13T09:12:35.143 INFO:teuthology.orchestra.run.smithi157.stdout:1548 trunc from 0x3a4dc00 to 0xb75600 2024-09-13T09:12:35.175 INFO:teuthology.orchestra.run.smithi157.stdout:1549 write 0x245cc00 thru 0x24612be (0x46bf bytes) 2024-09-13T09:12:35.175 INFO:teuthology.orchestra.run.smithi157.stdout:1552 punch from 0x1db8200 to 0x1dbfa00, (0x7800 bytes) 2024-09-13T09:12:35.192 INFO:teuthology.orchestra.run.smithi157.stdout:1553 write 0x5b2dc00 thru 0x5b303bf (0x27c0 bytes) 2024-09-13T09:12:35.201 INFO:teuthology.orchestra.run.smithi157.stdout:1554 trunc from 0x5b30400 to 0xc8e7600 2024-09-13T09:12:35.217 INFO:teuthology.orchestra.run.smithi157.stdout:1557 write 0x3233e00 thru 0x32416aa (0xd8ab bytes) 2024-09-13T09:12:35.224 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:35.223+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:35.232 INFO:teuthology.orchestra.run.smithi157.stdout:1558 trunc from 0xc8e7600 to 0x37e000 2024-09-13T09:12:35.259 INFO:teuthology.orchestra.run.smithi157.stdout:1562 read 0x320600 thru 0x32b452 (0xae53 bytes) 2024-09-13T09:12:35.261 INFO:teuthology.orchestra.run.smithi157.stdout:1565 read 0x75a00 thru 0x7c5fc (0x6bfd bytes) 2024-09-13T09:12:35.274 INFO:teuthology.orchestra.run.smithi157.stdout:1567 trunc from 0x37e000 to 0xabb8000 2024-09-13T09:12:35.282 INFO:teuthology.orchestra.run.smithi157.stdout:1570 read 0x110a400 thru 0x111251e (0x811f bytes) 2024-09-13T09:12:35.283 INFO:teuthology.orchestra.run.smithi157.stdout:1571 punch from 0x9c8e800 to 0x9c8f800, (0x1000 bytes) 2024-09-13T09:12:35.283 INFO:teuthology.orchestra.run.smithi157.stdout:1572 write 0x942a600 thru 0x9434872 (0xa273 bytes) 2024-09-13T09:12:35.283 INFO:teuthology.orchestra.run.smithi157.stdout:1573 read 0x2c8e200 thru 0x2c99510 (0xb311 bytes) 2024-09-13T09:12:35.289 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:35.289 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:35.299 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:35.301 INFO:teuthology.orchestra.run.smithi157.stdout:1574 write 0xe7dcc00 thru 0xe7ec3c0 (0xf7c1 bytes) 2024-09-13T09:12:35.302 INFO:teuthology.orchestra.run.smithi157.stdout:1579 punch from 0xb801e00 to 0xb811c00, (0xfe00 bytes) 2024-09-13T09:12:35.302 INFO:teuthology.orchestra.run.smithi157.stdout:1580 read 0x42b400 thru 0x42d6b8 (0x22b9 bytes) 2024-09-13T09:12:35.303 INFO:teuthology.orchestra.run.smithi157.stdout:1581 read 0x24d0000 thru 0x24d87c0 (0x87c1 bytes) 2024-09-13T09:12:35.303 INFO:teuthology.orchestra.run.smithi157.stdout:1582 read 0xfd0c00 thru 0xfd67f3 (0x5bf4 bytes) 2024-09-13T09:12:35.303 INFO:teuthology.orchestra.run.smithi157.stdout:1583 write 0x7aea200 thru 0x7af9db4 (0xfbb5 bytes) 2024-09-13T09:12:35.303 INFO:teuthology.orchestra.run.smithi157.stdout:1584 punch from 0xcf88c00 to 0xcf98a00, (0xfe00 bytes) 2024-09-13T09:12:35.303 INFO:teuthology.orchestra.run.smithi157.stdout:1586 write 0x1084800 thru 0x108b5bd (0x6dbe bytes) 2024-09-13T09:12:35.304 INFO:teuthology.orchestra.run.smithi157.stdout:1588 read 0xa103e00 thru 0xa10db96 (0x9d97 bytes) 2024-09-13T09:12:35.304 INFO:teuthology.orchestra.run.smithi157.stdout:1590 write 0x547c000 thru 0x54830f5 (0x70f6 bytes) 2024-09-13T09:12:35.305 INFO:teuthology.orchestra.run.smithi157.stdout:1591 write 0x2dc3c00 thru 0x2dc7080 (0x3481 bytes) 2024-09-13T09:12:35.305 INFO:teuthology.orchestra.run.smithi157.stdout:1593 read 0x1415e00 thru 0x1418a7e (0x2c7f bytes) 2024-09-13T09:12:35.309 INFO:teuthology.orchestra.run.smithi157.stdout:1594 trunc from 0xe7ec400 to 0xb8b2c00 2024-09-13T09:12:35.344 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:35.342+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:35.349 INFO:teuthology.orchestra.run.smithi157.stdout:1595 punch from 0x40c7000 to 0x40d6e00, (0xfe00 bytes) 2024-09-13T09:12:35.349 INFO:teuthology.orchestra.run.smithi157.stdout:1598 read 0x3b44600 thru 0x3b4af15 (0x6916 bytes) 2024-09-13T09:12:35.349 INFO:teuthology.orchestra.run.smithi157.stdout:1599 read 0xafc3800 thru 0xafc4020 (0x821 bytes) 2024-09-13T09:12:35.349 INFO:teuthology.orchestra.run.smithi157.stdout:1600 read 0x7082000 thru 0x708618a (0x418b bytes) 2024-09-13T09:12:35.354 INFO:teuthology.orchestra.run.smithi157.stdout:1608 trunc from 0xb8b2c00 to 0x7bd1200 2024-09-13T09:12:35.379 INFO:teuthology.orchestra.run.smithi157.stdout:1610 read 0x3db7a00 thru 0x3dc528d (0xd88e bytes) 2024-09-13T09:12:35.380 INFO:teuthology.orchestra.run.smithi157.stdout:1611 punch from 0x3fe7200 to 0x3ff0200, (0x9000 bytes) 2024-09-13T09:12:35.380 INFO:teuthology.orchestra.run.smithi157.stdout:1612 write 0x3667800 thru 0x3676a52 (0xf253 bytes) 2024-09-13T09:12:35.382 INFO:teuthology.orchestra.run.smithi157.stdout:1613 write 0x13d5c00 thru 0x13e110a (0xb50b bytes) 2024-09-13T09:12:35.383 INFO:teuthology.orchestra.run.smithi157.stdout:1614 write 0x2379600 thru 0x2380bd4 (0x75d5 bytes) 2024-09-13T09:12:35.406 INFO:teuthology.orchestra.run.smithi157.stdout:1616 write 0xe200000 thru 0xe20bd4d (0xbd4e bytes) 2024-09-13T09:12:35.406 INFO:teuthology.orchestra.run.smithi157.stdout:1617 read 0x9401400 thru 0x9407830 (0x6431 bytes) 2024-09-13T09:12:35.406 INFO:teuthology.orchestra.run.smithi157.stdout:1618 write 0x8eeda00 thru 0x8ef6ba1 (0x91a2 bytes) 2024-09-13T09:12:35.411 INFO:teuthology.orchestra.run.smithi157.stdout:1619 write 0xabefa00 thru 0xabfe827 (0xee28 bytes) 2024-09-13T09:12:35.411 INFO:teuthology.orchestra.run.smithi157.stdout:1622 read 0x9ea9600 thru 0x9eab168 (0x1b69 bytes) 2024-09-13T09:12:35.411 INFO:teuthology.orchestra.run.smithi157.stdout:1625 read 0x6ee6000 thru 0x6ee945b (0x345c bytes) 2024-09-13T09:12:35.411 INFO:teuthology.orchestra.run.smithi157.stdout:1626 punch from 0x8841e00 to 0x884e800, (0xca00 bytes) 2024-09-13T09:12:35.411 INFO:teuthology.orchestra.run.smithi157.stdout:1627 write 0x9142200 thru 0x91457b4 (0x35b5 bytes) 2024-09-13T09:12:35.424 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:35.424 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:35.425 INFO:teuthology.orchestra.run.smithi157.stdout:1628 trunc from 0xe20be00 to 0x2beaa00 2024-09-13T09:12:35.425 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:35.424+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:35.433 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:35.467 INFO:teuthology.orchestra.run.smithi157.stdout:1630 write 0x5bbf800 thru 0x5bcd5a9 (0xddaa bytes) 2024-09-13T09:12:35.468 INFO:teuthology.orchestra.run.smithi157.stdout:1631 write 0x7c7400 thru 0x7cad09 (0x390a bytes) 2024-09-13T09:12:35.468 INFO:teuthology.orchestra.run.smithi157.stdout:1634 punch from 0x1cc1000 to 0x1cc9800, (0x8800 bytes) 2024-09-13T09:12:35.469 INFO:teuthology.orchestra.run.smithi157.stdout:1636 read 0x14c5a00 thru 0x14d0aa1 (0xb0a2 bytes) 2024-09-13T09:12:35.476 INFO:teuthology.orchestra.run.smithi157.stdout:1637 read 0xa42a00 thru 0xa4b670 (0x8c71 bytes) 2024-09-13T09:12:35.485 INFO:teuthology.orchestra.run.smithi157.stdout:1638 trunc from 0x5bcd600 to 0xd7bda00 2024-09-13T09:12:35.495 INFO:teuthology.orchestra.run.smithi157.stdout:1640 write 0x1f08200 thru 0x1f1394d (0xb74e bytes) 2024-09-13T09:12:35.496 INFO:teuthology.orchestra.run.smithi157.stdout:1642 read 0x5d07800 thru 0x5d1297b (0xb17c bytes) 2024-09-13T09:12:35.496 INFO:teuthology.orchestra.run.smithi157.stdout:1643 read 0xac62600 thru 0xac712ad (0xecae bytes) 2024-09-13T09:12:35.508 INFO:teuthology.orchestra.run.smithi157.stdout:1644 trunc from 0xd7bda00 to 0x3a4b200 2024-09-13T09:12:35.523 INFO:teuthology.orchestra.run.smithi157.stdout:1648 punch from 0x2d5ac00 to 0x2d67600, (0xca00 bytes) 2024-09-13T09:12:35.525 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:35.525+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:35.539 INFO:teuthology.orchestra.run.smithi157.stdout:1650 write 0x5e37200 thru 0x5e3bb0d (0x490e bytes) 2024-09-13T09:12:35.539 INFO:teuthology.orchestra.run.smithi157.stdout:1652 write 0x4294e00 thru 0x429ec5b (0x9e5c bytes) 2024-09-13T09:12:35.539 INFO:teuthology.orchestra.run.smithi157.stdout:1653 punch from 0x3dc7400 to 0x3dcc200, (0x4e00 bytes) 2024-09-13T09:12:35.543 INFO:teuthology.orchestra.run.smithi157.stdout:1654 read 0x5882800 thru 0x5888c7e (0x647f bytes) 2024-09-13T09:12:35.549 INFO:teuthology.orchestra.run.smithi157.stdout:1655 trunc from 0x5e3bc00 to 0x289cc00 2024-09-13T09:12:35.552 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:35.552 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:35.563 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:35.567 INFO:teuthology.orchestra.run.smithi157.stdout:1659 trunc from 0x289cc00 to 0x2073c00 2024-09-13T09:12:35.580 INFO:teuthology.orchestra.run.smithi157.stdout:1662 punch from 0xb87a00 to 0xb89600, (0x1c00 bytes) 2024-09-13T09:12:35.603 INFO:teuthology.orchestra.run.smithi157.stdout:1663 write 0xded9000 thru 0xdee5afc (0xcafd bytes) 2024-09-13T09:12:35.604 INFO:teuthology.orchestra.run.smithi157.stdout:1664 punch from 0x4329c00 to 0x4337800, (0xdc00 bytes) 2024-09-13T09:12:35.604 INFO:teuthology.orchestra.run.smithi157.stdout:1666 read 0x6d31a00 thru 0x6d3d257 (0xb858 bytes) 2024-09-13T09:12:35.614 INFO:teuthology.orchestra.run.smithi157.stdout:1668 trunc from 0xdee5c00 to 0x533c600 2024-09-13T09:12:35.643 INFO:teuthology.orchestra.run.smithi157.stdout:1671 write 0xbed1600 thru 0xbedce47 (0xb848 bytes) 2024-09-13T09:12:35.647 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:35.647+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:35.650 INFO:teuthology.orchestra.run.smithi157.stdout:1673 read 0x62d2a00 thru 0x62d5fa9 (0x35aa bytes) 2024-09-13T09:12:35.650 INFO:teuthology.orchestra.run.smithi157.stdout:1675 read 0x439ca00 thru 0x439d4a8 (0xaa9 bytes) 2024-09-13T09:12:35.652 INFO:teuthology.orchestra.run.smithi157.stdout:1678 trunc from 0xbedd000 to 0x9f36600 2024-09-13T09:12:35.676 INFO:teuthology.orchestra.run.smithi157.stdout:1682 punch from 0x5f59200 to 0x5f65600, (0xc400 bytes) 2024-09-13T09:12:35.684 INFO:teuthology.orchestra.run.smithi157.stdout:1684 write 0x9633200 thru 0x96425f2 (0xf3f3 bytes) 2024-09-13T09:12:35.684 INFO:teuthology.orchestra.run.smithi157.stdout:1685 punch from 0x5af0000 to 0x5af1200, (0x1200 bytes) 2024-09-13T09:12:35.684 INFO:teuthology.orchestra.run.smithi157.stdout:1688 punch from 0x3801400 to 0x380aa00, (0x9600 bytes) 2024-09-13T09:12:35.684 INFO:teuthology.orchestra.run.smithi157.stdout:1689 write 0x2682e00 thru 0x2687416 (0x4617 bytes) 2024-09-13T09:12:35.686 INFO:teuthology.orchestra.run.smithi157.stdout:1690 write 0x9ed9400 thru 0x9eda88f (0x1490 bytes) 2024-09-13T09:12:35.686 INFO:teuthology.orchestra.run.smithi157.stdout:1692 punch from 0x6597e00 to 0x659be00, (0x4000 bytes) 2024-09-13T09:12:35.706 INFO:teuthology.orchestra.run.smithi157.stdout:1694 write 0xe9a7e00 thru 0xe9ab8bd (0x3abe bytes) 2024-09-13T09:12:35.706 INFO:teuthology.orchestra.run.smithi157.stdout:1695 write 0x1396400 thru 0x13a4a4a (0xe64b bytes) 2024-09-13T09:12:35.707 INFO:teuthology.orchestra.run.smithi157.stdout:1697 punch from 0x11eba00 to 0x11eec00, (0x3200 bytes) 2024-09-13T09:12:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:1698 punch from 0x4faac00 to 0x4fb5a00, (0xae00 bytes) 2024-09-13T09:12:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:1699 punch from 0xad16c00 to 0xad26400, (0xf800 bytes) 2024-09-13T09:12:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:1700 write 0xe2ebc00 thru 0xe2ecc81 (0x1082 bytes) 2024-09-13T09:12:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:1701 punch from 0x1900c00 to 0x190bc00, (0xb000 bytes) 2024-09-13T09:12:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:1702 punch from 0xcb0f600 to 0xcb15000, (0x5a00 bytes) 2024-09-13T09:12:35.721 INFO:teuthology.orchestra.run.smithi157.stdout:1703 trunc from 0xe9aba00 to 0x3fda800 2024-09-13T09:12:35.723 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:35.723 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:35.727 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:35.725+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:35.734 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:35.748 INFO:teuthology.orchestra.run.smithi157.stdout:1706 write 0x39cf400 thru 0x39cf976 (0x577 bytes) 2024-09-13T09:12:35.760 INFO:teuthology.orchestra.run.smithi157.stdout:1708 trunc from 0x3fda800 to 0x86cb600 2024-09-13T09:12:35.770 INFO:teuthology.orchestra.run.smithi157.stdout:1709 punch from 0x859ac00 to 0x859b000, (0x400 bytes) 2024-09-13T09:12:35.777 INFO:teuthology.orchestra.run.smithi157.stdout:1712 write 0x345fc00 thru 0x346a52d (0xa92e bytes) 2024-09-13T09:12:35.783 INFO:teuthology.orchestra.run.smithi157.stdout:1713 trunc from 0x86cb600 to 0x3956e00 2024-09-13T09:12:35.800 INFO:teuthology.orchestra.run.smithi157.stdout:1714 read 0x2cc7c00 thru 0x2cd51f2 (0xd5f3 bytes) 2024-09-13T09:12:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:1717 read 0x95fc00 thru 0x966b2f (0x6f30 bytes) 2024-09-13T09:12:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:1719 read 0x458200 thru 0x45a34b (0x214c bytes) 2024-09-13T09:12:35.809 INFO:teuthology.orchestra.run.smithi157.stdout:1720 write 0x22a2800 thru 0x22b11cf (0xe9d0 bytes) 2024-09-13T09:12:35.809 INFO:teuthology.orchestra.run.smithi157.stdout:1721 punch from 0x2da6a00 to 0x2db3200, (0xc800 bytes) 2024-09-13T09:12:35.810 INFO:teuthology.orchestra.run.smithi157.stdout:1727 write 0x5e1e00 thru 0x5e870e (0x690f bytes) 2024-09-13T09:12:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:1729 read 0x10aa800 thru 0x10b66a3 (0xbea4 bytes) 2024-09-13T09:12:35.811 INFO:teuthology.orchestra.run.smithi157.stdout:1732 read 0x18ef600 thru 0x18f43b8 (0x4db9 bytes) 2024-09-13T09:12:35.813 INFO:teuthology.orchestra.run.smithi157.stdout:1733 trunc from 0x3956e00 to 0x354ae00 2024-09-13T09:12:35.825 INFO:teuthology.orchestra.run.smithi157.stdout:1736 read 0x120e800 thru 0x1215491 (0x6c92 bytes) 2024-09-13T09:12:35.828 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:35.826+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:35.840 INFO:teuthology.orchestra.run.smithi157.stdout:1737 write 0x55b9c00 thru 0x55bb06b (0x146c bytes) 2024-09-13T09:12:35.841 INFO:teuthology.orchestra.run.smithi157.stdout:1739 read 0xbaa400 thru 0xbb209b (0x7c9c bytes) 2024-09-13T09:12:35.861 INFO:teuthology.orchestra.run.smithi157.stdout:1741 write 0x8c97400 thru 0x8c9ae94 (0x3a95 bytes) 2024-09-13T09:12:35.876 INFO:teuthology.orchestra.run.smithi157.stdout:1743 write 0xaa65800 thru 0xaa65852 (0x53 bytes) 2024-09-13T09:12:35.877 INFO:teuthology.orchestra.run.smithi157.stdout:1744 punch from 0x3903000 to 0x390bc00, (0x8c00 bytes) 2024-09-13T09:12:35.889 INFO:teuthology.orchestra.run.smithi157.stdout:1750 write 0xbca9400 thru 0xbcb3272 (0x9e73 bytes) 2024-09-13T09:12:35.889 INFO:teuthology.orchestra.run.smithi157.stdout:1752 write 0x9a72800 thru 0x9a7ba44 (0x9245 bytes) 2024-09-13T09:12:35.890 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:35.890 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:35.890 INFO:teuthology.orchestra.run.smithi157.stdout:1754 trunc from 0xbcb3400 to 0xc7bba00 2024-09-13T09:12:35.902 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:35.911 INFO:teuthology.orchestra.run.smithi157.stdout:1755 read 0xb224400 thru 0xb232318 (0xdf19 bytes) 2024-09-13T09:12:35.917 INFO:teuthology.orchestra.run.smithi157.stdout:1756 trunc from 0xc7bba00 to 0x7ead000 2024-09-13T09:12:35.944 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:35.944+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:35.950 INFO:teuthology.orchestra.run.smithi157.stdout:1759 write 0x9528600 thru 0x9536d7e (0xe77f bytes) 2024-09-13T09:12:35.950 INFO:teuthology.orchestra.run.smithi157.stdout:1763 read 0x6a2e00 thru 0x6affe4 (0xd1e5 bytes) 2024-09-13T09:12:35.953 INFO:teuthology.orchestra.run.smithi157.stdout:1765 trunc from 0x9536e00 to 0xbaddc00 2024-09-13T09:12:35.970 INFO:teuthology.orchestra.run.smithi157.stdout:1767 punch from 0x556c800 to 0x5578800, (0xc000 bytes) 2024-09-13T09:12:35.970 INFO:teuthology.orchestra.run.smithi157.stdout:1769 punch from 0x6008c00 to 0x600c200, (0x3600 bytes) 2024-09-13T09:12:35.971 INFO:teuthology.orchestra.run.smithi157.stdout:1771 read 0x51a6400 thru 0x51b2460 (0xc061 bytes) 2024-09-13T09:12:35.972 INFO:teuthology.orchestra.run.smithi157.stdout:1772 write 0x4bc0c00 thru 0x4bccabb (0xbebc bytes) 2024-09-13T09:12:35.973 INFO:teuthology.orchestra.run.smithi157.stdout:1775 read 0xb0bf000 thru 0xb0c88da (0x98db bytes) 2024-09-13T09:12:35.973 INFO:teuthology.orchestra.run.smithi157.stdout:1779 read 0x58fc200 thru 0x58fda2a (0x182b bytes) 2024-09-13T09:12:35.973 INFO:teuthology.orchestra.run.smithi157.stdout:1780 punch from 0x75e4e00 to 0x75e6c00, (0x1e00 bytes) 2024-09-13T09:12:35.973 INFO:teuthology.orchestra.run.smithi157.stdout:1781 write 0x510a600 thru 0x510bbf8 (0x15f9 bytes) 2024-09-13T09:12:35.978 INFO:teuthology.orchestra.run.smithi157.stdout:1782 punch from 0x8641600 to 0x864de00, (0xc800 bytes) 2024-09-13T09:12:35.978 INFO:teuthology.orchestra.run.smithi157.stdout:1785 write 0x72e4200 thru 0x72e60e7 (0x1ee8 bytes) 2024-09-13T09:12:35.985 INFO:teuthology.orchestra.run.smithi157.stdout:1790 trunc from 0xbaddc00 to 0x5e8ba00 2024-09-13T09:12:36.004 INFO:teuthology.orchestra.run.smithi157.stdout:1792 punch from 0xdda200 to 0xddd000, (0x2e00 bytes) 2024-09-13T09:12:36.009 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-13T09:12:36.009 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:12:36.009 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-13T09:12:36.009 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:12:36.009 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:12:36.010 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-13T09:12:36.011 INFO:teuthology.orchestra.run.smithi157.stdout:1795 write 0x26bcc00 thru 0x26c0c67 (0x4068 bytes) 2024-09-13T09:12:36.012 INFO:teuthology.orchestra.run.smithi157.stdout:1796 read 0x3694a00 thru 0x369765c (0x2c5d bytes) 2024-09-13T09:12:36.013 INFO:teuthology.orchestra.run.smithi157.stdout:1797 read 0x3f1a000 thru 0x3f26c3a (0xcc3b bytes) 2024-09-13T09:12:36.020 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:36.021 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:36.024 INFO:teuthology.orchestra.run.smithi157.stdout:1798 write 0x7448400 thru 0x744b89f (0x34a0 bytes) 2024-09-13T09:12:36.025 INFO:teuthology.orchestra.run.smithi157.stdout:1799 read 0x6ddea00 thru 0x6de2a2d (0x402e bytes) 2024-09-13T09:12:36.031 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:36.033 INFO:teuthology.orchestra.run.smithi157.stdout:1800 trunc from 0x744ba00 to 0xe654000 2024-09-13T09:12:36.055 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:36.053+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:36.055 INFO:teuthology.orchestra.run.smithi157.stdout:1802 trunc from 0xe654000 to 0x62c6000 2024-09-13T09:12:36.083 INFO:teuthology.orchestra.run.smithi157.stdout:1804 write 0xe1bf000 thru 0xe1c3009 (0x400a bytes) 2024-09-13T09:12:36.091 INFO:teuthology.orchestra.run.smithi157.stdout:1806 trunc from 0xe1c3200 to 0x84d9a00 2024-09-13T09:12:36.110 INFO:teuthology.orchestra.run.smithi157.stdout:1807 trunc from 0x84d9a00 to 0x792c000 2024-09-13T09:12:36.118 INFO:teuthology.orchestra.run.smithi157.stdout:1812 clone 11 order 25 su 16777216 sc 8 2024-09-13T09:12:36.125 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:36.125 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:12:36.125 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:36.130 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:36.130+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:36.180 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:36.181 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:36.189 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:36.231 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:36.231+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:36.331 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:36.332 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:36.332 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:36.331+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:36.340 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:36.432 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:36.431+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:36.484 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:36.484 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:36.492 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:36.523 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:12:36.568 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:36.566+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:36.600 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:36.600 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:36.609 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:36.653 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:36.652+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:36.714 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:36.714 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:36.723 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:36.766 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:36.764+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:36.833 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:36.834 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:36.836 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:36.836+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:36.842 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:36.860 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:12:36.880 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-13T09:12:36.937 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:36.935+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:37.011 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:37.012 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:37.021 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:37.026 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:37.027 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:37.028 INFO:teuthology.orchestra.run.smithi157.stdout:1813 punch from 0x3c01e00 to 0x3c08a00, (0x6c00 bytes) 2024-09-13T09:12:37.037 INFO:teuthology.orchestra.run.smithi157.stdout:1814 read 0x4e70c00 thru 0x4e7ad93 (0xa194 bytes) 2024-09-13T09:12:37.038 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:37.037+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:37.038 INFO:teuthology.orchestra.run.smithi157.stdout:1816 write 0x6511600 thru 0x65145dd (0x2fde bytes) 2024-09-13T09:12:37.038 INFO:teuthology.orchestra.run.smithi157.stdout:1817 punch from 0x2de9400 to 0x2df3800, (0xa400 bytes) 2024-09-13T09:12:37.046 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:37.046 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:37.047 INFO:teuthology.orchestra.run.smithi157.stdout:1818 write 0x20fae00 thru 0x21068b8 (0xbab9 bytes) 2024-09-13T09:12:37.048 INFO:teuthology.orchestra.run.smithi157.stdout:1819 write 0x1394a00 thru 0x1397c0a (0x320b bytes) 2024-09-13T09:12:37.050 INFO:teuthology.orchestra.run.smithi157.stdout:1820 write 0x7242000 thru 0x725196f (0xf970 bytes) 2024-09-13T09:12:37.054 INFO:teuthology.orchestra.run.smithi157.stdout:1821 trunc from 0x792c000 to 0xafef800 2024-09-13T09:12:37.064 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:37.064 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:37.071 INFO:teuthology.orchestra.run.smithi157.stdout:1822 punch from 0x6ef3600 to 0x6eff000, (0xba00 bytes) 2024-09-13T09:12:37.072 INFO:teuthology.orchestra.run.smithi157.stdout:1823 write 0xab90600 thru 0xab92c82 (0x2683 bytes) 2024-09-13T09:12:37.083 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:37.083 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:37.085 INFO:teuthology.orchestra.run.smithi157.stdout:1824 trunc from 0xafef800 to 0xbc3000 2024-09-13T09:12:37.139 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:37.138+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:37.139 INFO:teuthology.orchestra.run.smithi157.stdout:1826 read 0x3df600 thru 0x3edb67 (0xe568 bytes) 2024-09-13T09:12:37.140 INFO:teuthology.orchestra.run.smithi157.stdout:1827 read 0x258e00 thru 0x25ec0c (0x5e0d bytes) 2024-09-13T09:12:37.146 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:37.146 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:37.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:37.146 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:37.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.147 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1321 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:37.147 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.242418+0000", 2024-09-13T09:12:37.147 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.901837735999999, 2024-09-13T09:12:37.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0087640029999999994, 2024-09-13T09:12:37.147 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.147 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.147 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.147 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.147 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.147 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1321 2024-09-13T09:12:37.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.147 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242418+0000", 2024-09-13T09:12:37.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242418+0000", 2024-09-13T09:12:37.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242418+0000", 2024-09-13T09:12:37.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242419+0000", 2024-09-13T09:12:37.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7740000000000001e-06 2024-09-13T09:12:37.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242424+0000", 2024-09-13T09:12:37.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4839999999999998e-06 2024-09-13T09:12:37.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.242427+0000", 2024-09-13T09:12:37.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8660000000000002e-06 2024-09-13T09:12:37.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.244978+0000", 2024-09-13T09:12:37.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002551139 2024-09-13T09:12:37.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.245025+0000", 2024-09-13T09:12:37.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6832999999999997e-05 2024-09-13T09:12:37.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.245221+0000", 2024-09-13T09:12:37.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000195954 2024-09-13T09:12:37.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.246365+0000", 2024-09-13T09:12:37.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011444669999999999 2024-09-13T09:12:37.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.251115+0000", 2024-09-13T09:12:37.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047497779999999996 2024-09-13T09:12:37.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.251150+0000", 2024-09-13T09:12:37.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4687999999999997e-05 2024-09-13T09:12:37.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.251182+0000", 2024-09-13T09:12:37.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2264000000000001e-05 2024-09-13T09:12:37.152 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.152 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.152 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.152 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1550 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:37.153 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.329689+0000", 2024-09-13T09:12:37.153 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.814567062, 2024-09-13T09:12:37.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.009203391, 2024-09-13T09:12:37.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.153 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.153 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.153 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.153 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.153 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1550 2024-09-13T09:12:37.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.153 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329689+0000", 2024-09-13T09:12:37.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329689+0000", 2024-09-13T09:12:37.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329688+0000", 2024-09-13T09:12:37.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329690+0000", 2024-09-13T09:12:37.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7e-06 2024-09-13T09:12:37.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329695+0000", 2024-09-13T09:12:37.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4880000000000001e-06 2024-09-13T09:12:37.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329697+0000", 2024-09-13T09:12:37.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.74e-06 2024-09-13T09:12:37.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.333003+0000", 2024-09-13T09:12:37.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003305451 2024-09-13T09:12:37.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.333084+0000", 2024-09-13T09:12:37.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1062000000000004e-05 2024-09-13T09:12:37.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.333427+0000", 2024-09-13T09:12:37.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000342932 2024-09-13T09:12:37.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338607+0000", 2024-09-13T09:12:37.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051806580000000003 2024-09-13T09:12:37.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338821+0000", 2024-09-13T09:12:37.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00021341099999999999 2024-09-13T09:12:37.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338860+0000", 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9320000000000003e-05 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338892+0000", 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1868e-05 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1551 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.329715+0000", 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.814540210000001, 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0093095490000000003, 2024-09-13T09:12:37.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.159 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.159 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.159 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.159 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.159 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1551 2024-09-13T09:12:37.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.159 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329715+0000", 2024-09-13T09:12:37.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329715+0000", 2024-09-13T09:12:37.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329715+0000", 2024-09-13T09:12:37.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329718+0000", 2024-09-13T09:12:37.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8279999999999999e-06 2024-09-13T09:12:37.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329726+0000", 2024-09-13T09:12:37.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0420000000000003e-06 2024-09-13T09:12:37.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.329729+0000", 2024-09-13T09:12:37.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1099999999999999e-06 2024-09-13T09:12:37.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.333843+0000", 2024-09-13T09:12:37.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0041141499999999996 2024-09-13T09:12:37.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.333899+0000", 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6273000000000001e-05 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.334107+0000", 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00020787400000000001 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338610+0000", 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004502944 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338943+0000", 2024-09-13T09:12:37.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033260000000000001 2024-09-13T09:12:37.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.338980+0000", 2024-09-13T09:12:37.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7382999999999998e-05 2024-09-13T09:12:37.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.339025+0000", 2024-09-13T09:12:37.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4765999999999998e-05 2024-09-13T09:12:37.163 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.164 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.164 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.164 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1990 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.164 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.504086+0000", 2024-09-13T09:12:37.164 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.640169271, 2024-09-13T09:12:37.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0098999020000000004, 2024-09-13T09:12:37.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.164 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.164 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.164 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.164 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.164 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1990 2024-09-13T09:12:37.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.165 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504086+0000", 2024-09-13T09:12:37.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504086+0000", 2024-09-13T09:12:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504086+0000", 2024-09-13T09:12:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504090+0000", 2024-09-13T09:12:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000004e-06 2024-09-13T09:12:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504097+0000", 2024-09-13T09:12:37.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4630000000000004e-06 2024-09-13T09:12:37.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504103+0000", 2024-09-13T09:12:37.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4589999999999997e-06 2024-09-13T09:12:37.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.505791+0000", 2024-09-13T09:12:37.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0016881260000000001 2024-09-13T09:12:37.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.505955+0000", 2024-09-13T09:12:37.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016378200000000001 2024-09-13T09:12:37.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.506366+0000", 2024-09-13T09:12:37.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000411667 2024-09-13T09:12:37.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.513522+0000", 2024-09-13T09:12:37.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071554360000000003 2024-09-13T09:12:37.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.513662+0000", 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000140195 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.513949+0000", 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000286908 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.513986+0000", 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7444999999999998e-05 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.169 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1991 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.504108+0000", 2024-09-13T09:12:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.640147388999999, 2024-09-13T09:12:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011285132, 2024-09-13T09:12:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1991 2024-09-13T09:12:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.170 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504108+0000", 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504108+0000", 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504108+0000", 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504111+0000", 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4070000000000002e-06 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504117+0000", 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504123+0000", 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507071+0000", 2024-09-13T09:12:37.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029475579999999999 2024-09-13T09:12:37.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507151+0000", 2024-09-13T09:12:37.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0000000000000007e-05 2024-09-13T09:12:37.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507556+0000", 2024-09-13T09:12:37.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00040499900000000002 2024-09-13T09:12:37.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515305+0000", 2024-09-13T09:12:37.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0077488820000000003 2024-09-13T09:12:37.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515312+0000", 2024-09-13T09:12:37.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1640000000000004e-06 2024-09-13T09:12:37.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515358+0000", 2024-09-13T09:12:37.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.604e-05 2024-09-13T09:12:37.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515393+0000", 2024-09-13T09:12:37.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5670999999999998e-05 2024-09-13T09:12:37.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.175 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.175 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1992 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.175 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.504132+0000", 2024-09-13T09:12:37.175 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.640123478, 2024-09-13T09:12:37.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011378241000000001, 2024-09-13T09:12:37.176 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.176 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.176 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.176 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.176 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.176 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1992 2024-09-13T09:12:37.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.176 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504132+0000", 2024-09-13T09:12:37.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504132+0000", 2024-09-13T09:12:37.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504132+0000", 2024-09-13T09:12:37.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504135+0000", 2024-09-13T09:12:37.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4580000000000002e-06 2024-09-13T09:12:37.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504143+0000", 2024-09-13T09:12:37.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0590000000000003e-06 2024-09-13T09:12:37.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504149+0000", 2024-09-13T09:12:37.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7649999999999999e-06 2024-09-13T09:12:37.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.508003+0000", 2024-09-13T09:12:37.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003854176 2024-09-13T09:12:37.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.508075+0000", 2024-09-13T09:12:37.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2197e-05 2024-09-13T09:12:37.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.508374+0000", 2024-09-13T09:12:37.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00029824099999999998 2024-09-13T09:12:37.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515402+0000", 2024-09-13T09:12:37.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0070279380000000001 2024-09-13T09:12:37.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515437+0000", 2024-09-13T09:12:37.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5274000000000001e-05 2024-09-13T09:12:37.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.181 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.181 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515475+0000", 2024-09-13T09:12:37.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8296000000000001e-05 2024-09-13T09:12:37.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.181 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.181 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515510+0000", 2024-09-13T09:12:37.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5261e-05 2024-09-13T09:12:37.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.181 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.181 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1993 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.181 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.504155+0000", 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.640100449, 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01145854, 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1993 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504155+0000", 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504155+0000", 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504155+0000", 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504158+0000", 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3100000000000001e-06 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504165+0000", 2024-09-13T09:12:37.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7519999999999999e-06 2024-09-13T09:12:37.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.504170+0000", 2024-09-13T09:12:37.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5350000000000004e-06 2024-09-13T09:12:37.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.508797+0000", 2024-09-13T09:12:37.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046267319999999997 2024-09-13T09:12:37.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.508893+0000", 2024-09-13T09:12:37.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5356e-05 2024-09-13T09:12:37.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.509224+0000", 2024-09-13T09:12:37.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033109799999999999 2024-09-13T09:12:37.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515406+0000", 2024-09-13T09:12:37.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061827640000000003 2024-09-13T09:12:37.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515544+0000", 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013761599999999999 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515581+0000", 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6902e-05 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515614+0000", 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2848000000000001e-05 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.186 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2156 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.568613+0000", 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.575642683, 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0087999440000000005, 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2156 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568613+0000", 2024-09-13T09:12:37.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568613+0000", 2024-09-13T09:12:37.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568612+0000", 2024-09-13T09:12:37.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568616+0000", 2024-09-13T09:12:37.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9380000000000002e-06 2024-09-13T09:12:37.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568625+0000", 2024-09-13T09:12:37.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5080000000000005e-06 2024-09-13T09:12:37.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.568630+0000", 2024-09-13T09:12:37.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2920000000000003e-06 2024-09-13T09:12:37.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.571451+0000", 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028206749999999999 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.571530+0000", 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9571000000000001e-05 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.571881+0000", 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035083500000000002 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.575089+0000", 2024-09-13T09:12:37.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003207831 2024-09-13T09:12:37.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.577325+0000", 2024-09-13T09:12:37.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022359139999999999 2024-09-13T09:12:37.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.577374+0000", 2024-09-13T09:12:37.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8667e-05 2024-09-13T09:12:37.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.577413+0000", 2024-09-13T09:12:37.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9211000000000001e-05 2024-09-13T09:12:37.192 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.192 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.192 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.192 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2280 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.192 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.614829+0000", 2024-09-13T09:12:37.192 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.529426917, 2024-09-13T09:12:37.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010493406, 2024-09-13T09:12:37.192 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.193 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.193 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.193 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.193 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.193 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2280 2024-09-13T09:12:37.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.193 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614829+0000", 2024-09-13T09:12:37.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614829+0000", 2024-09-13T09:12:37.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614828+0000", 2024-09-13T09:12:37.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614848+0000", 2024-09-13T09:12:37.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9653e-05 2024-09-13T09:12:37.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614880+0000", 2024-09-13T09:12:37.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2008999999999998e-05 2024-09-13T09:12:37.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614887+0000", 2024-09-13T09:12:37.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7750000000000004e-06 2024-09-13T09:12:37.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617693+0000", 2024-09-13T09:12:37.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028061119999999999 2024-09-13T09:12:37.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617770+0000", 2024-09-13T09:12:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7286999999999994e-05 2024-09-13T09:12:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618140+0000", 2024-09-13T09:12:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000370355 2024-09-13T09:12:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.622954+0000", 2024-09-13T09:12:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048138829999999997 2024-09-13T09:12:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625235+0000", 2024-09-13T09:12:37.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022808160000000002 2024-09-13T09:12:37.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625289+0000", 2024-09-13T09:12:37.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3604999999999999e-05 2024-09-13T09:12:37.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625322+0000", 2024-09-13T09:12:37.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3491999999999997e-05 2024-09-13T09:12:37.197 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.198 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.198 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.198 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2281 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.198 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.614893+0000", 2024-09-13T09:12:37.198 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.529362911, 2024-09-13T09:12:37.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010532901000000001, 2024-09-13T09:12:37.198 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.198 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.198 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.198 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.198 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.198 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2281 2024-09-13T09:12:37.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.199 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614893+0000", 2024-09-13T09:12:37.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614893+0000", 2024-09-13T09:12:37.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614892+0000", 2024-09-13T09:12:37.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614900+0000", 2024-09-13T09:12:37.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2119999999999997e-06 2024-09-13T09:12:37.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614924+0000", 2024-09-13T09:12:37.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4260000000000002e-05 2024-09-13T09:12:37.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614934+0000", 2024-09-13T09:12:37.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7659999999999993e-06 2024-09-13T09:12:37.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619174+0000", 2024-09-13T09:12:37.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042408890000000003 2024-09-13T09:12:37.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619255+0000", 2024-09-13T09:12:37.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0189000000000001e-05 2024-09-13T09:12:37.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619609+0000", 2024-09-13T09:12:37.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035430500000000002 2024-09-13T09:12:37.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.622991+0000", 2024-09-13T09:12:37.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033823220000000001 2024-09-13T09:12:37.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625333+0000", 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023419309999999998 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625385+0000", 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2194999999999997e-05 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625426+0000", 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0278999999999998e-05 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.204 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2282 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.204 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.614942+0000", 2024-09-13T09:12:37.204 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.52931379, 2024-09-13T09:12:37.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010565567999999999, 2024-09-13T09:12:37.204 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.204 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.204 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.204 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.204 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.204 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2282 2024-09-13T09:12:37.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.204 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614942+0000", 2024-09-13T09:12:37.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614942+0000", 2024-09-13T09:12:37.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614941+0000", 2024-09-13T09:12:37.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614947+0000", 2024-09-13T09:12:37.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3789999999999997e-06 2024-09-13T09:12:37.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614956+0000", 2024-09-13T09:12:37.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2969999999999992e-06 2024-09-13T09:12:37.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614961+0000", 2024-09-13T09:12:37.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6110000000000001e-06 2024-09-13T09:12:37.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.207 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620209+0000", 2024-09-13T09:12:37.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052470429999999998 2024-09-13T09:12:37.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.207 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620286+0000", 2024-09-13T09:12:37.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7061e-05 2024-09-13T09:12:37.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.207 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620646+0000", 2024-09-13T09:12:37.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000360164 2024-09-13T09:12:37.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.623024+0000", 2024-09-13T09:12:37.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002378186 2024-09-13T09:12:37.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625434+0000", 2024-09-13T09:12:37.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0024102630000000002 2024-09-13T09:12:37.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625475+0000", 2024-09-13T09:12:37.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0867e-05 2024-09-13T09:12:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625507+0000", 2024-09-13T09:12:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2394000000000002e-05 2024-09-13T09:12:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2283 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.614977+0000", 2024-09-13T09:12:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.529279072, 2024-09-13T09:12:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011508387, 2024-09-13T09:12:37.209 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.210 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.210 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.210 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.210 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.210 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2283 2024-09-13T09:12:37.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.210 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614977+0000", 2024-09-13T09:12:37.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.211 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614977+0000", 2024-09-13T09:12:37.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.211 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.211 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614976+0000", 2024-09-13T09:12:37.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.211 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.211 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614980+0000", 2024-09-13T09:12:37.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8659999999999999e-06 2024-09-13T09:12:37.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.212 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.614997+0000", 2024-09-13T09:12:37.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6946e-05 2024-09-13T09:12:37.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.212 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615003+0000", 2024-09-13T09:12:37.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4829999999999996e-06 2024-09-13T09:12:37.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.212 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.621063+0000", 2024-09-13T09:12:37.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0060601860000000004 2024-09-13T09:12:37.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.213 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.621415+0000", 2024-09-13T09:12:37.213 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035131399999999998 2024-09-13T09:12:37.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.213 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.621769+0000", 2024-09-13T09:12:37.213 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035409500000000002 2024-09-13T09:12:37.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.213 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625219+0000", 2024-09-13T09:12:37.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034497970000000001 2024-09-13T09:12:37.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626400+0000", 2024-09-13T09:12:37.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011811910000000001 2024-09-13T09:12:37.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626448+0000", 2024-09-13T09:12:37.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8615000000000001e-05 2024-09-13T09:12:37.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626485+0000", 2024-09-13T09:12:37.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6491e-05 2024-09-13T09:12:37.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.215 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.215 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2286 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.215 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.615037+0000", 2024-09-13T09:12:37.215 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.529219002, 2024-09-13T09:12:37.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013941546000000001, 2024-09-13T09:12:37.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.215 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.215 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.215 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.216 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.216 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2286 2024-09-13T09:12:37.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.216 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615037+0000", 2024-09-13T09:12:37.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615037+0000", 2024-09-13T09:12:37.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.217 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.217 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615036+0000", 2024-09-13T09:12:37.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.217 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.217 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615039+0000", 2024-09-13T09:12:37.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1829999999999998e-06 2024-09-13T09:12:37.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.217 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.217 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615047+0000", 2024-09-13T09:12:37.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:12:37.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.615052+0000", 2024-09-13T09:12:37.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2780000000000001e-06 2024-09-13T09:12:37.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.623049+0000", 2024-09-13T09:12:37.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0079966499999999992 2024-09-13T09:12:37.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.623128+0000", 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9531999999999997e-05 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.623468+0000", 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033982700000000001 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625551+0000", 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020836140000000001 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628931+0000", 2024-09-13T09:12:37.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033793709999999999 2024-09-13T09:12:37.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628958+0000", 2024-09-13T09:12:37.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6928e-05 2024-09-13T09:12:37.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628978+0000", 2024-09-13T09:12:37.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0426999999999999e-05 2024-09-13T09:12:37.220 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.220 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.221 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.221 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2293 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.221 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.616955+0000", 2024-09-13T09:12:37.221 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.527300409, 2024-09-13T09:12:37.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014655674, 2024-09-13T09:12:37.221 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.221 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.221 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.221 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.221 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.221 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2293 2024-09-13T09:12:37.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.222 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.222 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616955+0000", 2024-09-13T09:12:37.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.222 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616955+0000", 2024-09-13T09:12:37.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.222 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616954+0000", 2024-09-13T09:12:37.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:37.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.223 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616960+0000", 2024-09-13T09:12:37.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1809999999999998e-06 2024-09-13T09:12:37.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.223 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616973+0000", 2024-09-13T09:12:37.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3001e-05 2024-09-13T09:12:37.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.616981+0000", 2024-09-13T09:12:37.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5089999999999996e-06 2024-09-13T09:12:37.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625596+0000", 2024-09-13T09:12:37.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086152260000000001 2024-09-13T09:12:37.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.625680+0000", 2024-09-13T09:12:37.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4098999999999996e-05 2024-09-13T09:12:37.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626054+0000", 2024-09-13T09:12:37.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00037370300000000002 2024-09-13T09:12:37.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.629114+0000", 2024-09-13T09:12:37.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0030600279999999998 2024-09-13T09:12:37.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631561+0000", 2024-09-13T09:12:37.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0024473009999999998 2024-09-13T09:12:37.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631590+0000", 2024-09-13T09:12:37.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9033000000000001e-05 2024-09-13T09:12:37.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631611+0000", 2024-09-13T09:12:37.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0665999999999998e-05 2024-09-13T09:12:37.226 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.226 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.226 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.227 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2297 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.227 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.617105+0000", 2024-09-13T09:12:37.227 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.527151073000001, 2024-09-13T09:12:37.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01465442, 2024-09-13T09:12:37.227 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.227 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.227 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.227 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.227 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.227 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2297 2024-09-13T09:12:37.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.227 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.228 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.228 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617105+0000", 2024-09-13T09:12:37.228 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.228 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.228 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617105+0000", 2024-09-13T09:12:37.228 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.228 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.228 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617104+0000", 2024-09-13T09:12:37.228 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:37.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.229 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.229 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617109+0000", 2024-09-13T09:12:37.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7579999999999998e-06 2024-09-13T09:12:37.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.229 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.229 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617121+0000", 2024-09-13T09:12:37.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1439e-05 2024-09-13T09:12:37.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.229 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.229 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617137+0000", 2024-09-13T09:12:37.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6481999999999999e-05 2024-09-13T09:12:37.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.230 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.230 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626526+0000", 2024-09-13T09:12:37.230 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0093890499999999995 2024-09-13T09:12:37.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.230 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.230 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626617+0000", 2024-09-13T09:12:37.230 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0116000000000003e-05 2024-09-13T09:12:37.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.230 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.230 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.626962+0000", 2024-09-13T09:12:37.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034533300000000002 2024-09-13T09:12:37.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.629177+0000", 2024-09-13T09:12:37.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022147529999999999 2024-09-13T09:12:37.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631706+0000", 2024-09-13T09:12:37.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025289829999999998 2024-09-13T09:12:37.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631735+0000", 2024-09-13T09:12:37.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9309e-05 2024-09-13T09:12:37.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.232 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.232 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631759+0000", 2024-09-13T09:12:37.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4066e-05 2024-09-13T09:12:37.232 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.232 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.232 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.232 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2298 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.232 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.617145+0000", 2024-09-13T09:12:37.232 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.527110712000001, 2024-09-13T09:12:37.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014745108, 2024-09-13T09:12:37.233 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.233 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.233 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.233 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.233 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.233 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2298 2024-09-13T09:12:37.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.233 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.233 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.233 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617145+0000", 2024-09-13T09:12:37.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617145+0000", 2024-09-13T09:12:37.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617144+0000", 2024-09-13T09:12:37.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617148+0000", 2024-09-13T09:12:37.235 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6830000000000001e-06 2024-09-13T09:12:37.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.235 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.235 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617156+0000", 2024-09-13T09:12:37.235 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1610000000000003e-06 2024-09-13T09:12:37.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.235 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.235 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617162+0000", 2024-09-13T09:12:37.235 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1600000000000003e-06 2024-09-13T09:12:37.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.235 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.627419+0000", 2024-09-13T09:12:37.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010256517 2024-09-13T09:12:37.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.236 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.627516+0000", 2024-09-13T09:12:37.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7600000000000001e-05 2024-09-13T09:12:37.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.236 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.627872+0000", 2024-09-13T09:12:37.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035532400000000002 2024-09-13T09:12:37.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631665+0000", 2024-09-13T09:12:37.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037928200000000001 2024-09-13T09:12:37.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631771+0000", 2024-09-13T09:12:37.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000106288 2024-09-13T09:12:37.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631807+0000", 2024-09-13T09:12:37.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5695999999999998e-05 2024-09-13T09:12:37.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631890+0000", 2024-09-13T09:12:37.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3473000000000005e-05 2024-09-13T09:12:37.238 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.238 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.238 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.238 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2308 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.238 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.618672+0000", 2024-09-13T09:12:37.238 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.525583416, 2024-09-13T09:12:37.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013301396, 2024-09-13T09:12:37.238 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.238 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.239 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.239 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.239 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.239 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2308 2024-09-13T09:12:37.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.239 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618672+0000", 2024-09-13T09:12:37.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618672+0000", 2024-09-13T09:12:37.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.240 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618672+0000", 2024-09-13T09:12:37.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.240 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618681+0000", 2024-09-13T09:12:37.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3819999999999993e-06 2024-09-13T09:12:37.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.240 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618690+0000", 2024-09-13T09:12:37.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2420000000000006e-06 2024-09-13T09:12:37.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.241 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618696+0000", 2024-09-13T09:12:37.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2859999999999996e-06 2024-09-13T09:12:37.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.241 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628271+0000", 2024-09-13T09:12:37.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.009574249 2024-09-13T09:12:37.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.241 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628314+0000", 2024-09-13T09:12:37.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3467999999999998e-05 2024-09-13T09:12:37.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.628672+0000", 2024-09-13T09:12:37.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000357726 2024-09-13T09:12:37.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631699+0000", 2024-09-13T09:12:37.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003026759 2024-09-13T09:12:37.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631902+0000", 2024-09-13T09:12:37.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000202862 2024-09-13T09:12:37.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.243 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631943+0000", 2024-09-13T09:12:37.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1084000000000002e-05 2024-09-13T09:12:37.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.243 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631974+0000", 2024-09-13T09:12:37.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1007000000000003e-05 2024-09-13T09:12:37.243 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.243 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.243 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.244 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2309 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.244 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.619333+0000", 2024-09-13T09:12:37.244 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.524922753, 2024-09-13T09:12:37.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012775455999999999, 2024-09-13T09:12:37.244 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.244 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.244 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.244 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.244 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.244 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2309 2024-09-13T09:12:37.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.244 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.245 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619333+0000", 2024-09-13T09:12:37.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.245 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619333+0000", 2024-09-13T09:12:37.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.245 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619332+0000", 2024-09-13T09:12:37.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:37.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619338+0000", 2024-09-13T09:12:37.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.648e-06 2024-09-13T09:12:37.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619349+0000", 2024-09-13T09:12:37.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1545e-05 2024-09-13T09:12:37.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619367+0000", 2024-09-13T09:12:37.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7419e-05 2024-09-13T09:12:37.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.629223+0000", 2024-09-13T09:12:37.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0098561680000000002 2024-09-13T09:12:37.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.629311+0000", 2024-09-13T09:12:37.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8057000000000001e-05 2024-09-13T09:12:37.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.630009+0000", 2024-09-13T09:12:37.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00069777100000000002 2024-09-13T09:12:37.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.248 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631985+0000", 2024-09-13T09:12:37.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019758919999999999 2024-09-13T09:12:37.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.248 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.632022+0000", 2024-09-13T09:12:37.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7410000000000003e-05 2024-09-13T09:12:37.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.632075+0000", 2024-09-13T09:12:37.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3248000000000002e-05 2024-09-13T09:12:37.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.632108+0000", 2024-09-13T09:12:37.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3191000000000001e-05 2024-09-13T09:12:37.249 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.249 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.249 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.249 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2310 3.6 3:671c3922:::rbd_object_map.1123b8b4a28a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:37.249 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.619384+0000", 2024-09-13T09:12:37.250 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.524871731999999, 2024-09-13T09:12:37.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014600315000000001, 2024-09-13T09:12:37.250 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.250 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:37.250 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.250 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:37.250 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:37.250 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2310 2024-09-13T09:12:37.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.250 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.250 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.250 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619384+0000", 2024-09-13T09:12:37.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619384+0000", 2024-09-13T09:12:37.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619383+0000", 2024-09-13T09:12:37.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619387+0000", 2024-09-13T09:12:37.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1200000000000004e-06 2024-09-13T09:12:37.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619398+0000", 2024-09-13T09:12:37.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0841e-05 2024-09-13T09:12:37.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619404+0000", 2024-09-13T09:12:37.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8329999999999999e-06 2024-09-13T09:12:37.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.630863+0000", 2024-09-13T09:12:37.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011459278 2024-09-13T09:12:37.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.630971+0000", 2024-09-13T09:12:37.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010728100000000001 2024-09-13T09:12:37.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:37.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.631296+0000", 2024-09-13T09:12:37.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00032547400000000002 2024-09-13T09:12:37.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.632133+0000", 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00083720599999999995 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.633830+0000", 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001696342 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.633950+0000", 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120444 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.633984+0000", 2024-09-13T09:12:37.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4082000000000003e-05 2024-09-13T09:12:37.255 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.255 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.255 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.255 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4816.0:719 3.1f 3:f983ed45:::rbd_data.1123df4df335.0000000000000008:d [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T09:12:37.255 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:34.218407+0000", 2024-09-13T09:12:37.255 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.925848679, 2024-09-13T09:12:37.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01050302, 2024-09-13T09:12:37.255 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.255 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.255 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.255 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4816", 2024-09-13T09:12:37.256 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2210529289", 2024-09-13T09:12:37.256 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 719 2024-09-13T09:12:37.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.256 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.218407+0000", 2024-09-13T09:12:37.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.218407+0000", 2024-09-13T09:12:37.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.218406+0000", 2024-09-13T09:12:37.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.218409+0000", 2024-09-13T09:12:37.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7219999999999999e-06 2024-09-13T09:12:37.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.218416+0000", 2024-09-13T09:12:37.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:12:37.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.218422+0000", 2024-09-13T09:12:37.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.455e-06 2024-09-13T09:12:37.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.226431+0000", 2024-09-13T09:12:37.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0080093630000000002 2024-09-13T09:12:37.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.226484+0000", 2024-09-13T09:12:37.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2258e-05 2024-09-13T09:12:37.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:34.228910+0000", 2024-09-13T09:12:37.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00242625 2024-09-13T09:12:37.259 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.259 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.259 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.259 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.259 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.259 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:37.262 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:37.262 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:37.258+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:37.262 INFO:teuthology.orchestra.run.smithi157.stdout:1829 write 0x1d44a00 thru 0x1d4f5f6 (0xabf7 bytes) 2024-09-13T09:12:37.262 INFO:teuthology.orchestra.run.smithi157.stdout:1831 trunc from 0x1d4f600 to 0x858e200 2024-09-13T09:12:37.262 INFO:teuthology.orchestra.run.smithi157.stdout:1835 trunc from 0x858e200 to 0x447200 2024-09-13T09:12:37.262 INFO:teuthology.orchestra.run.smithi157.stdout:1836 trunc from 0x447200 to 0x6864000 2024-09-13T09:12:37.262 INFO:teuthology.orchestra.run.smithi157.stdout:1837 trunc from 0x6864000 to 0x1b6e000 2024-09-13T09:12:37.263 INFO:teuthology.orchestra.run.smithi157.stdout:1838 punch from 0x2b5400 to 0x2ba800, (0x5400 bytes) 2024-09-13T09:12:37.263 INFO:teuthology.orchestra.run.smithi157.stdout:1839 trunc from 0x1b6e000 to 0xb406800 2024-09-13T09:12:37.263 INFO:teuthology.orchestra.run.smithi157.stdout:1841 read 0x4467a00 thru 0x4472ec5 (0xb4c6 bytes) 2024-09-13T09:12:37.263 INFO:teuthology.orchestra.run.smithi157.stdout:1842 write 0xe7c4800 thru 0xe7d0d17 (0xc518 bytes) 2024-09-13T09:12:37.263 INFO:teuthology.orchestra.run.smithi157.stdout:1845 read 0x63be000 thru 0x63cd537 (0xf538 bytes) 2024-09-13T09:12:37.264 INFO:teuthology.orchestra.run.smithi157.stdout:1846 read 0x85fa00 thru 0x866e3b (0x743c bytes) 2024-09-13T09:12:37.264 INFO:teuthology.orchestra.run.smithi157.stdout:1847 write 0x48fa00 thru 0x499e1f (0xa420 bytes) 2024-09-13T09:12:37.276 INFO:teuthology.orchestra.run.smithi157.stdout:1849 write 0xea1de00 thru 0xea1f7df (0x19e0 bytes) 2024-09-13T09:12:37.278 INFO:teuthology.orchestra.run.smithi157.stdout:1850 read 0xb3a6e00 thru 0xb3b36f8 (0xc8f9 bytes) 2024-09-13T09:12:37.278 INFO:teuthology.orchestra.run.smithi157.stdout:1852 read 0x56d8a00 thru 0x56dfb03 (0x7104 bytes) 2024-09-13T09:12:37.278 INFO:teuthology.orchestra.run.smithi157.stdout:1853 punch from 0x8175400 to 0x8179a00, (0x4600 bytes) 2024-09-13T09:12:37.279 INFO:teuthology.orchestra.run.smithi157.stdout:1855 read 0xcfe7600 thru 0xcfefd2e (0x872f bytes) 2024-09-13T09:12:37.279 INFO:teuthology.orchestra.run.smithi157.stdout:1856 read 0x3428a00 thru 0x342d904 (0x4f05 bytes) 2024-09-13T09:12:37.279 INFO:teuthology.orchestra.run.smithi157.stdout:1858 read 0x654be00 thru 0x654d6fd (0x18fe bytes) 2024-09-13T09:12:37.279 INFO:teuthology.orchestra.run.smithi157.stdout:1860 read 0x8e62800 thru 0x8e69767 (0x6f68 bytes) 2024-09-13T09:12:37.280 INFO:teuthology.orchestra.run.smithi157.stdout:1863 write 0xa99000 thru 0xa9d341 (0x4342 bytes) 2024-09-13T09:12:37.281 INFO:teuthology.orchestra.run.smithi157.stdout:1864 read 0x1309000 thru 0x1314f4f (0xbf50 bytes) 2024-09-13T09:12:37.281 INFO:teuthology.orchestra.run.smithi157.stdout:1868 read 0x1a43c00 thru 0x1a46891 (0x2c92 bytes) 2024-09-13T09:12:37.281 INFO:teuthology.orchestra.run.smithi157.stdout:1870 read 0x5840e00 thru 0x58466c9 (0x58ca bytes) 2024-09-13T09:12:37.281 INFO:teuthology.orchestra.run.smithi157.stdout:1872 write 0x187800 thru 0x196293 (0xea94 bytes) 2024-09-13T09:12:37.282 INFO:teuthology.orchestra.run.smithi157.stdout:1874 read 0x9ef9200 thru 0x9f0716a (0xdf6b bytes) 2024-09-13T09:12:37.282 INFO:teuthology.orchestra.run.smithi157.stdout:1876 punch from 0x87cbc00 to 0x87d4400, (0x8800 bytes) 2024-09-13T09:12:37.293 INFO:teuthology.orchestra.run.smithi157.stdout:1878 trunc from 0xea1f800 to 0x492a000 2024-09-13T09:12:37.302 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:37.302 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:37.309 INFO:teuthology.orchestra.run.smithi157.stdout:1879 read 0x22e1a00 thru 0x22e310b (0x170c bytes) 2024-09-13T09:12:37.310 INFO:teuthology.orchestra.run.smithi157.stdout:1880 read 0x2c23200 thru 0x2c3059b (0xd39c bytes) 2024-09-13T09:12:37.310 INFO:teuthology.orchestra.run.smithi157.stdout:1881 write 0x32f200 thru 0x33d3c8 (0xe1c9 bytes) 2024-09-13T09:12:37.311 INFO:teuthology.orchestra.run.smithi157.stdout:1882 write 0x1250200 thru 0x125773d (0x753e bytes) 2024-09-13T09:12:37.311 INFO:teuthology.orchestra.run.smithi157.stdout:1883 read 0x26ddc00 thru 0x26e0588 (0x2989 bytes) 2024-09-13T09:12:37.321 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:37.321 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:37.322 INFO:teuthology.orchestra.run.smithi157.stdout:1884 write 0x4b36800 thru 0x4b42d55 (0xc556 bytes) 2024-09-13T09:12:37.339 INFO:teuthology.orchestra.run.smithi157.stdout:1888 write 0x7f0e200 thru 0x7f1b582 (0xd383 bytes) 2024-09-13T09:12:37.339 INFO:teuthology.orchestra.run.smithi157.stdout:1890 punch from 0x304e200 to 0x3059600, (0xb400 bytes) 2024-09-13T09:12:37.340 INFO:teuthology.orchestra.run.smithi157.stdout:1895 write 0xb26000 thru 0xb353b3 (0xf3b4 bytes) 2024-09-13T09:12:37.340 INFO:teuthology.orchestra.run.smithi157.stdout:1896 read 0x1868200 thru 0x186f5b6 (0x73b7 bytes) 2024-09-13T09:12:37.341 INFO:teuthology.orchestra.run.smithi157.stdout:1900 read 0x31d0200 thru 0x31dccec (0xcaed bytes) 2024-09-13T09:12:37.341 INFO:teuthology.orchestra.run.smithi157.stdout:1902 read 0x6ea6800 thru 0x6eabca0 (0x54a1 bytes) 2024-09-13T09:12:37.341 INFO:teuthology.orchestra.run.smithi157.stdout:1906 punch from 0xe09a00 to 0xe0c600, (0x2c00 bytes) 2024-09-13T09:12:37.343 INFO:teuthology.orchestra.run.smithi157.stdout:1908 trunc from 0x7f1b600 to 0x647ca00 2024-09-13T09:12:37.355 INFO:teuthology.orchestra.run.smithi157.stdout:1909 punch from 0x1148c00 to 0x114ea00, (0x5e00 bytes) 2024-09-13T09:12:37.355 INFO:teuthology.orchestra.run.smithi157.stdout:1911 write 0x105ae00 thru 0x1062a15 (0x7c16 bytes) 2024-09-13T09:12:37.357 INFO:teuthology.orchestra.run.smithi157.stdout:1912 read 0x447fe00 thru 0x4486b61 (0x6d62 bytes) 2024-09-13T09:12:37.374 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:37.372+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:37.375 INFO:teuthology.orchestra.run.smithi157.stdout:1916 write 0xc014a00 thru 0xc023d16 (0xf317 bytes) 2024-09-13T09:12:37.379 INFO:teuthology.orchestra.run.smithi157.stdout:1917 trunc from 0xc023e00 to 0x9502c00 2024-09-13T09:12:37.393 INFO:teuthology.orchestra.run.smithi157.stdout:1918 read 0x1dbd400 thru 0x1dc6185 (0x8d86 bytes) 2024-09-13T09:12:37.394 INFO:teuthology.orchestra.run.smithi157.stdout:1919 read 0x4757400 thru 0x4763be5 (0xc7e6 bytes) 2024-09-13T09:12:37.395 INFO:teuthology.orchestra.run.smithi157.stdout:1922 punch from 0x5bcd200 to 0x5bd5a00, (0x8800 bytes) 2024-09-13T09:12:37.395 INFO:teuthology.orchestra.run.smithi157.stdout:1923 read 0x12bd600 thru 0x12c93ad (0xbdae bytes) 2024-09-13T09:12:37.398 INFO:teuthology.orchestra.run.smithi157.stdout:1926 trunc from 0x9502c00 to 0x756aa00 2024-09-13T09:12:37.407 INFO:teuthology.orchestra.run.smithi157.stdout:1930 read 0x2d04600 thru 0x2d0cf6c (0x896d bytes) 2024-09-13T09:12:37.408 INFO:teuthology.orchestra.run.smithi157.stdout:1931 read 0x6c71c00 thru 0x6c73045 (0x1446 bytes) 2024-09-13T09:12:37.408 INFO:teuthology.orchestra.run.smithi157.stdout:1932 write 0x505ca00 thru 0x50643d8 (0x79d9 bytes) 2024-09-13T09:12:37.409 INFO:teuthology.orchestra.run.smithi157.stdout:1933 read 0x5607a00 thru 0x560b527 (0x3b28 bytes) 2024-09-13T09:12:37.409 INFO:teuthology.orchestra.run.smithi157.stdout:1934 read 0x18c3400 thru 0x18d02a6 (0xcea7 bytes) 2024-09-13T09:12:37.419 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:37.419 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:37.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.419 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4706.0:1177 3.7 e42/21 3:e7191d4c:::rbd_header.1123313a4365:head v 42'276, mlcod=42'274)", 2024-09-13T09:12:37.419 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:37.418214+0000", 2024-09-13T09:12:37.419 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.00042529200000000002, 2024-09-13T09:12:37.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00044021700000000002, 2024-09-13T09:12:37.419 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.419 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.419 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:37.418214+0000", 2024-09-13T09:12:37.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:37.418214+0000", 2024-09-13T09:12:37.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:37.418213+0000", 2024-09-13T09:12:37.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:37.418220+0000", 2024-09-13T09:12:37.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.131e-06 2024-09-13T09:12:37.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:37.418233+0000", 2024-09-13T09:12:37.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2655e-05 2024-09-13T09:12:37.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:37.418238+0000", 2024-09-13T09:12:37.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9529999999999998e-06 2024-09-13T09:12:37.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:37.418276+0000", 2024-09-13T09:12:37.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8121000000000003e-05 2024-09-13T09:12:37.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:37.418296+0000", 2024-09-13T09:12:37.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0372000000000001e-05 2024-09-13T09:12:37.423 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.423 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.423 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.423 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.423 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:12:37.423 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 1 2024-09-13T09:12:37.423 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:37.423 INFO:teuthology.orchestra.run.smithi157.stdout:1935 write 0x9e67600 thru 0x9e75cb1 (0xe6b2 bytes) 2024-09-13T09:12:37.424 INFO:teuthology.orchestra.run.smithi157.stdout:1936 write 0x2855000 thru 0x285cf7d (0x7f7e bytes) 2024-09-13T09:12:37.425 INFO:teuthology.orchestra.run.smithi157.stdout:1937 trunc from 0x9e75e00 to 0xb746a00 2024-09-13T09:12:37.428 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:37.443 INFO:teuthology.orchestra.run.smithi157.stdout:1940 trunc from 0xb746a00 to 0x56e1c00 2024-09-13T09:12:37.455 INFO:teuthology.orchestra.run.smithi157.stdout:1941 write 0x4304800 thru 0x430f15f (0xa960 bytes) 2024-09-13T09:12:37.456 INFO:teuthology.orchestra.run.smithi157.stdout:1943 read 0xd7f400 thru 0xd826b1 (0x32b2 bytes) 2024-09-13T09:12:37.457 INFO:teuthology.orchestra.run.smithi157.stdout:1944 punch from 0x523600 to 0x526800, (0x3200 bytes) 2024-09-13T09:12:37.462 INFO:teuthology.orchestra.run.smithi157.stdout:1945 trunc from 0x56e1c00 to 0x99fca00 2024-09-13T09:12:37.462 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:37.461+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:37.470 INFO:teuthology.orchestra.run.smithi157.stdout:1948 trunc from 0x99fca00 to 0x9dd1c00 2024-09-13T09:12:37.484 INFO:teuthology.orchestra.run.smithi157.stdout:1949 trunc from 0x9dd1c00 to 0xe4ce600 2024-09-13T09:12:37.494 INFO:teuthology.orchestra.run.smithi157.stdout:1950 write 0x296600 thru 0x2a59cb (0xf3cc bytes) 2024-09-13T09:12:37.496 INFO:teuthology.orchestra.run.smithi157.stdout:1951 write 0x7b80e00 thru 0x7b81f5c (0x115d bytes) 2024-09-13T09:12:37.496 INFO:teuthology.orchestra.run.smithi157.stdout:1955 read 0x49ece00 thru 0x49fbc7e (0xee7f bytes) 2024-09-13T09:12:37.496 INFO:teuthology.orchestra.run.smithi157.stdout:1958 write 0x34d8400 thru 0x34e65f2 (0xe1f3 bytes) 2024-09-13T09:12:37.497 INFO:teuthology.orchestra.run.smithi157.stdout:1959 write 0x233d400 thru 0x233f785 (0x2386 bytes) 2024-09-13T09:12:37.498 INFO:teuthology.orchestra.run.smithi157.stdout:1960 read 0x3122400 thru 0x312afea (0x8beb bytes) 2024-09-13T09:12:37.498 INFO:teuthology.orchestra.run.smithi157.stdout:1961 write 0x87cb200 thru 0x87cb26c (0x6d bytes) 2024-09-13T09:12:37.499 INFO:teuthology.orchestra.run.smithi157.stdout:1962 write 0x918ca00 thru 0x91964a4 (0x9aa5 bytes) 2024-09-13T09:12:37.499 INFO:teuthology.orchestra.run.smithi157.stdout:1964 read 0x38ad200 thru 0x38b6e01 (0x9c02 bytes) 2024-09-13T09:12:37.500 INFO:teuthology.orchestra.run.smithi157.stdout:1965 read 0x3e9ac00 thru 0x3e9fbf7 (0x4ff8 bytes) 2024-09-13T09:12:37.500 INFO:teuthology.orchestra.run.smithi157.stdout:1968 punch from 0x6ecd200 to 0x6ed1400, (0x4200 bytes) 2024-09-13T09:12:37.507 INFO:teuthology.orchestra.run.smithi157.stdout:1969 trunc from 0xe4ce600 to 0x8885a00 2024-09-13T09:12:37.507 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-09-13T09:12:37.530 INFO:teuthology.orchestra.run.smithi157.stdout:1970 clone 12 order 24 su 1048576 sc 6 2024-09-13T09:12:37.562 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:37.560+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:37.582 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:37.582 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:37.582 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:37.582 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:37.582 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:37.590 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:37.674 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:37.672+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:37.739 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:37.739 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:37.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:37.739 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:37.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.739 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:946 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.739 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.304801+0000", 2024-09-13T09:12:37.739 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.432019818000001, 2024-09-13T09:12:37.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010858095, 2024-09-13T09:12:37.739 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.739 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.740 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.740 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.740 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.740 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 946 2024-09-13T09:12:37.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.740 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.740 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304801+0000", 2024-09-13T09:12:37.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304801+0000", 2024-09-13T09:12:37.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304800+0000", 2024-09-13T09:12:37.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304803+0000", 2024-09-13T09:12:37.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:12:37.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304817+0000", 2024-09-13T09:12:37.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3736e-05 2024-09-13T09:12:37.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304823+0000", 2024-09-13T09:12:37.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1680000000000001e-06 2024-09-13T09:12:37.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315506+0000", 2024-09-13T09:12:37.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010682764000000001 2024-09-13T09:12:37.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315580+0000", 2024-09-13T09:12:37.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4306000000000003e-05 2024-09-13T09:12:37.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315659+0000", 2024-09-13T09:12:37.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8807999999999995e-05 2024-09-13T09:12:37.743 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.743 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.743 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.744 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:947 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.744 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.304829+0000", 2024-09-13T09:12:37.744 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.431991856, 2024-09-13T09:12:37.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011020509, 2024-09-13T09:12:37.744 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.744 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.744 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.744 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.744 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.744 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 947 2024-09-13T09:12:37.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.744 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304829+0000", 2024-09-13T09:12:37.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304829+0000", 2024-09-13T09:12:37.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304828+0000", 2024-09-13T09:12:37.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304831+0000", 2024-09-13T09:12:37.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7819999999999998e-06 2024-09-13T09:12:37.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304838+0000", 2024-09-13T09:12:37.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9909999999999996e-06 2024-09-13T09:12:37.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.304843+0000", 2024-09-13T09:12:37.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2140000000000004e-06 2024-09-13T09:12:37.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315686+0000", 2024-09-13T09:12:37.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010842762000000001 2024-09-13T09:12:37.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315763+0000", 2024-09-13T09:12:37.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6429999999999998e-05 2024-09-13T09:12:37.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.748 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.748 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315849+0000", 2024-09-13T09:12:37.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6891999999999994e-05 2024-09-13T09:12:37.748 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.748 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.748 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.748 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1053 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 13369344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.748 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.309361+0000", 2024-09-13T09:12:37.748 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.427459835000001, 2024-09-13T09:12:37.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011710174, 2024-09-13T09:12:37.748 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.749 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.749 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.749 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.749 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.749 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1053 2024-09-13T09:12:37.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.749 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.749 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.749 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309361+0000", 2024-09-13T09:12:37.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.749 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.749 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309361+0000", 2024-09-13T09:12:37.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309361+0000", 2024-09-13T09:12:37.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309363+0000", 2024-09-13T09:12:37.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1900000000000002e-06 2024-09-13T09:12:37.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309368+0000", 2024-09-13T09:12:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6740000000000002e-06 2024-09-13T09:12:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309373+0000", 2024-09-13T09:12:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5730000000000001e-06 2024-09-13T09:12:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320071+0000", 2024-09-13T09:12:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010697628000000001 2024-09-13T09:12:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320149+0000", 2024-09-13T09:12:37.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8536999999999997e-05 2024-09-13T09:12:37.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.752 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.321071+0000", 2024-09-13T09:12:37.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00092195099999999996 2024-09-13T09:12:37.752 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.752 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.752 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.752 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1054 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.752 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.309377+0000", 2024-09-13T09:12:37.752 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.427443674999999, 2024-09-13T09:12:37.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011868086, 2024-09-13T09:12:37.753 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.753 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.753 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.753 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.753 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.753 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1054 2024-09-13T09:12:37.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.753 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.753 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309377+0000", 2024-09-13T09:12:37.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.754 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.754 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309377+0000", 2024-09-13T09:12:37.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.754 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.754 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309377+0000", 2024-09-13T09:12:37.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.754 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.754 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309379+0000", 2024-09-13T09:12:37.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:12:37.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309385+0000", 2024-09-13T09:12:37.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:12:37.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309391+0000", 2024-09-13T09:12:37.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7440000000000004e-06 2024-09-13T09:12:37.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.321098+0000", 2024-09-13T09:12:37.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011707255999999999 2024-09-13T09:12:37.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.321166+0000", 2024-09-13T09:12:37.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8436999999999995e-05 2024-09-13T09:12:37.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.321245+0000", 2024-09-13T09:12:37.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8988999999999999e-05 2024-09-13T09:12:37.756 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.756 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.756 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.757 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1055 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.757 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.309395+0000", 2024-09-13T09:12:37.757 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.427425596999999, 2024-09-13T09:12:37.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012096084, 2024-09-13T09:12:37.757 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.757 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.757 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.757 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.757 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.757 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1055 2024-09-13T09:12:37.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.757 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309395+0000", 2024-09-13T09:12:37.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309395+0000", 2024-09-13T09:12:37.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309395+0000", 2024-09-13T09:12:37.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309397+0000", 2024-09-13T09:12:37.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.503e-06 2024-09-13T09:12:37.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309405+0000", 2024-09-13T09:12:37.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4379999999999998e-06 2024-09-13T09:12:37.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.309410+0000", 2024-09-13T09:12:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0660000000000003e-06 2024-09-13T09:12:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.321298+0000", 2024-09-13T09:12:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011888156 2024-09-13T09:12:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.321382+0000", 2024-09-13T09:12:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4444000000000007e-05 2024-09-13T09:12:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.760 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.321491+0000", 2024-09-13T09:12:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000108904 2024-09-13T09:12:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1157 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.314790+0000", 2024-09-13T09:12:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.422030993, 2024-09-13T09:12:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010738912, 2024-09-13T09:12:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.761 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1157 2024-09-13T09:12:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314790+0000", 2024-09-13T09:12:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314790+0000", 2024-09-13T09:12:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.763 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314789+0000", 2024-09-13T09:12:37.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.763 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314792+0000", 2024-09-13T09:12:37.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7190000000000001e-06 2024-09-13T09:12:37.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.763 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314798+0000", 2024-09-13T09:12:37.764 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2709999999999998e-06 2024-09-13T09:12:37.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.764 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.764 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314804+0000", 2024-09-13T09:12:37.764 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7819999999999999e-06 2024-09-13T09:12:37.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.764 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.764 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325369+0000", 2024-09-13T09:12:37.764 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010565194 2024-09-13T09:12:37.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.765 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.765 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325437+0000", 2024-09-13T09:12:37.765 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7872000000000003e-05 2024-09-13T09:12:37.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.765 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.765 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325529+0000", 2024-09-13T09:12:37.765 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1476000000000003e-05 2024-09-13T09:12:37.765 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.765 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.765 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.765 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1158 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.766 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.314815+0000", 2024-09-13T09:12:37.766 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.422005566999999, 2024-09-13T09:12:37.766 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010903037000000001, 2024-09-13T09:12:37.766 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.766 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.766 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.766 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.766 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.766 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1158 2024-09-13T09:12:37.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.766 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.766 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314815+0000", 2024-09-13T09:12:37.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.767 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314815+0000", 2024-09-13T09:12:37.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.767 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314815+0000", 2024-09-13T09:12:37.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.768 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.768 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314817+0000", 2024-09-13T09:12:37.768 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.773e-06 2024-09-13T09:12:37.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.768 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.768 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.768 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314824+0000", 2024-09-13T09:12:37.768 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.516e-06 2024-09-13T09:12:37.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.768 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.768 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.768 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314831+0000", 2024-09-13T09:12:37.768 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2130000000000002e-06 2024-09-13T09:12:37.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.769 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.769 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325558+0000", 2024-09-13T09:12:37.769 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010727093 2024-09-13T09:12:37.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.769 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.769 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325624+0000", 2024-09-13T09:12:37.769 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6088999999999997e-05 2024-09-13T09:12:37.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.769 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.770 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325718+0000", 2024-09-13T09:12:37.770 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3861000000000001e-05 2024-09-13T09:12:37.770 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.770 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.770 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.770 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1159 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.770 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.314836+0000", 2024-09-13T09:12:37.770 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.421984454, 2024-09-13T09:12:37.770 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011062855999999999, 2024-09-13T09:12:37.770 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.770 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.770 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.771 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.771 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.771 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1159 2024-09-13T09:12:37.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.771 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314836+0000", 2024-09-13T09:12:37.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314836+0000", 2024-09-13T09:12:37.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.772 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.772 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314836+0000", 2024-09-13T09:12:37.772 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.772 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.772 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314838+0000", 2024-09-13T09:12:37.772 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:12:37.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.772 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.772 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314845+0000", 2024-09-13T09:12:37.773 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3589999999999996e-06 2024-09-13T09:12:37.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.773 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.773 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.773 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314849+0000", 2024-09-13T09:12:37.773 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7790000000000003e-06 2024-09-13T09:12:37.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.773 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.773 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.773 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325747+0000", 2024-09-13T09:12:37.773 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010897558999999999 2024-09-13T09:12:37.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.773 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.774 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.774 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325832+0000", 2024-09-13T09:12:37.774 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5143999999999997e-05 2024-09-13T09:12:37.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.774 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.774 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325899+0000", 2024-09-13T09:12:37.774 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7053999999999999e-05 2024-09-13T09:12:37.774 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.774 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.774 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.775 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1160 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.775 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.314854+0000", 2024-09-13T09:12:37.775 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.421966947, 2024-09-13T09:12:37.775 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011253521000000001, 2024-09-13T09:12:37.775 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.775 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.775 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.775 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.775 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.775 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1160 2024-09-13T09:12:37.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.776 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.776 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.776 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314854+0000", 2024-09-13T09:12:37.776 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.776 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.776 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314854+0000", 2024-09-13T09:12:37.776 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.776 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.777 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314853+0000", 2024-09-13T09:12:37.777 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.777 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.777 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314856+0000", 2024-09-13T09:12:37.777 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6220000000000001e-06 2024-09-13T09:12:37.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.777 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.777 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314862+0000", 2024-09-13T09:12:37.777 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1179999999999997e-06 2024-09-13T09:12:37.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.778 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.778 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314867+0000", 2024-09-13T09:12:37.778 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.848e-06 2024-09-13T09:12:37.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.778 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.778 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.325928+0000", 2024-09-13T09:12:37.778 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011060558 2024-09-13T09:12:37.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.778 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.778 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326014+0000", 2024-09-13T09:12:37.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6793999999999996e-05 2024-09-13T09:12:37.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326107+0000", 2024-09-13T09:12:37.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2998999999999995e-05 2024-09-13T09:12:37.779 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.779 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.779 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.779 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1161 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 14155776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.779 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.314872+0000", 2024-09-13T09:12:37.780 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.421948475000001, 2024-09-13T09:12:37.780 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011426746999999999, 2024-09-13T09:12:37.780 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.780 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.780 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.780 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.780 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.780 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1161 2024-09-13T09:12:37.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.780 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.780 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.780 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314872+0000", 2024-09-13T09:12:37.780 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.781 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.781 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314872+0000", 2024-09-13T09:12:37.781 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.781 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.781 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314872+0000", 2024-09-13T09:12:37.781 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.781 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.781 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314874+0000", 2024-09-13T09:12:37.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3159999999999999e-06 2024-09-13T09:12:37.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.782 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.782 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314881+0000", 2024-09-13T09:12:37.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2700000000000001e-06 2024-09-13T09:12:37.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.782 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.782 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.314885+0000", 2024-09-13T09:12:37.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:12:37.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326137+0000", 2024-09-13T09:12:37.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011251585 2024-09-13T09:12:37.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326210+0000", 2024-09-13T09:12:37.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3453000000000003e-05 2024-09-13T09:12:37.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326299+0000", 2024-09-13T09:12:37.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8651000000000002e-05 2024-09-13T09:12:37.783 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.784 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.784 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.784 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1162 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.784 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.315036+0000", 2024-09-13T09:12:37.784 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.421784347999999, 2024-09-13T09:12:37.784 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011465312, 2024-09-13T09:12:37.784 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.784 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.784 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.784 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.784 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.785 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1162 2024-09-13T09:12:37.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.785 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.785 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.785 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315036+0000", 2024-09-13T09:12:37.785 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.785 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.785 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315036+0000", 2024-09-13T09:12:37.785 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.786 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.786 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315036+0000", 2024-09-13T09:12:37.786 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.786 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.786 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315040+0000", 2024-09-13T09:12:37.786 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0330000000000002e-06 2024-09-13T09:12:37.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.786 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.786 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315050+0000", 2024-09-13T09:12:37.786 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9310000000000003e-06 2024-09-13T09:12:37.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.787 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.787 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.315057+0000", 2024-09-13T09:12:37.787 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.934e-06 2024-09-13T09:12:37.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.787 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.787 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326333+0000", 2024-09-13T09:12:37.787 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01127591 2024-09-13T09:12:37.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.787 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.787 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326415+0000", 2024-09-13T09:12:37.787 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2649000000000001e-05 2024-09-13T09:12:37.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.788 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.788 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.326502+0000", 2024-09-13T09:12:37.788 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6524e-05 2024-09-13T09:12:37.788 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.788 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.788 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.788 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1261 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 4456448~47616] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.788 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320345+0000", 2024-09-13T09:12:37.788 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.416476076, 2024-09-13T09:12:37.789 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011029959000000001, 2024-09-13T09:12:37.789 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.789 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.789 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.789 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.789 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.789 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1261 2024-09-13T09:12:37.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.789 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.789 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.789 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320345+0000", 2024-09-13T09:12:37.789 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.790 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.790 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320345+0000", 2024-09-13T09:12:37.790 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.790 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.790 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320344+0000", 2024-09-13T09:12:37.790 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.790 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.790 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320347+0000", 2024-09-13T09:12:37.790 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3259999999999998e-06 2024-09-13T09:12:37.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.791 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320354+0000", 2024-09-13T09:12:37.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6210000000000004e-06 2024-09-13T09:12:37.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.791 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320359+0000", 2024-09-13T09:12:37.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9320000000000004e-06 2024-09-13T09:12:37.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.792 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.329121+0000", 2024-09-13T09:12:37.792 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0087622510000000004 2024-09-13T09:12:37.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.792 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.792 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.329171+0000", 2024-09-13T09:12:37.792 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9815999999999999e-05 2024-09-13T09:12:37.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.792 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.792 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331375+0000", 2024-09-13T09:12:37.792 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002203408 2024-09-13T09:12:37.792 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.792 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.793 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.793 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1262 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 6160384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.793 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320364+0000", 2024-09-13T09:12:37.793 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.416456927, 2024-09-13T09:12:37.793 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011167203000000001, 2024-09-13T09:12:37.793 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.793 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.793 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.793 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.793 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.793 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1262 2024-09-13T09:12:37.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.794 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.794 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.794 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320364+0000", 2024-09-13T09:12:37.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.794 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.794 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320364+0000", 2024-09-13T09:12:37.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320363+0000", 2024-09-13T09:12:37.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320366+0000", 2024-09-13T09:12:37.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:12:37.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320372+0000", 2024-09-13T09:12:37.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0719999999999996e-06 2024-09-13T09:12:37.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.796 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.796 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320377+0000", 2024-09-13T09:12:37.796 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:12:37.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.796 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.796 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331409+0000", 2024-09-13T09:12:37.796 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011031891 2024-09-13T09:12:37.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.796 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331459+0000", 2024-09-13T09:12:37.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9607999999999997e-05 2024-09-13T09:12:37.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.797 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331531+0000", 2024-09-13T09:12:37.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2192000000000003e-05 2024-09-13T09:12:37.797 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.797 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.797 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.797 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1263 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.798 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320382+0000", 2024-09-13T09:12:37.798 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.416438454, 2024-09-13T09:12:37.798 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011363743000000001, 2024-09-13T09:12:37.798 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.798 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.798 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.798 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.798 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.798 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1263 2024-09-13T09:12:37.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.798 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.798 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.798 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320382+0000", 2024-09-13T09:12:37.799 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.799 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.799 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320382+0000", 2024-09-13T09:12:37.799 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.799 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.799 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320382+0000", 2024-09-13T09:12:37.799 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.800 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.800 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320384+0000", 2024-09-13T09:12:37.800 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.362e-06 2024-09-13T09:12:37.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.800 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.800 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320392+0000", 2024-09-13T09:12:37.800 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.481e-06 2024-09-13T09:12:37.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.800 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.800 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320397+0000", 2024-09-13T09:12:37.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0669999999999999e-06 2024-09-13T09:12:37.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.801 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331566+0000", 2024-09-13T09:12:37.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011168757 2024-09-13T09:12:37.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.801 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331654+0000", 2024-09-13T09:12:37.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8577000000000003e-05 2024-09-13T09:12:37.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.802 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.802 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331746+0000", 2024-09-13T09:12:37.802 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1831999999999998e-05 2024-09-13T09:12:37.802 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.802 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.802 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.802 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.802 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1264 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 8519680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.802 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320401+0000", 2024-09-13T09:12:37.802 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.416419401000001, 2024-09-13T09:12:37.802 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011595169000000001, 2024-09-13T09:12:37.802 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.803 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.803 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.803 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.803 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.803 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1264 2024-09-13T09:12:37.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.803 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320401+0000", 2024-09-13T09:12:37.803 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.804 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320401+0000", 2024-09-13T09:12:37.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.804 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320401+0000", 2024-09-13T09:12:37.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.804 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320404+0000", 2024-09-13T09:12:37.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6299999999999998e-06 2024-09-13T09:12:37.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.805 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.805 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320410+0000", 2024-09-13T09:12:37.805 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9519999999999999e-06 2024-09-13T09:12:37.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.805 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.805 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320414+0000", 2024-09-13T09:12:37.805 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:12:37.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.805 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.806 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331782+0000", 2024-09-13T09:12:37.806 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011367962000000001 2024-09-13T09:12:37.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.806 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.806 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.806 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331878+0000", 2024-09-13T09:12:37.806 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5261999999999997e-05 2024-09-13T09:12:37.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.806 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.806 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.806 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.331997+0000", 2024-09-13T09:12:37.806 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011889 2024-09-13T09:12:37.806 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.806 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.807 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.807 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.807 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.807 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1265 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.807 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320419+0000", 2024-09-13T09:12:37.807 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.416401806, 2024-09-13T09:12:37.807 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011788669, 2024-09-13T09:12:37.807 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.807 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.807 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.807 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.807 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.807 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1265 2024-09-13T09:12:37.808 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.808 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.808 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.808 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.808 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320419+0000", 2024-09-13T09:12:37.808 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.808 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.808 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.808 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.808 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320419+0000", 2024-09-13T09:12:37.808 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.808 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.808 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.809 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.809 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320419+0000", 2024-09-13T09:12:37.809 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.809 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.809 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.809 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320421+0000", 2024-09-13T09:12:37.809 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.024e-06 2024-09-13T09:12:37.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.809 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.809 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.809 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320428+0000", 2024-09-13T09:12:37.809 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2150000000000004e-06 2024-09-13T09:12:37.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.810 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.810 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.810 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320432+0000", 2024-09-13T09:12:37.810 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5719999999999996e-06 2024-09-13T09:12:37.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.810 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.810 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.810 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332032+0000", 2024-09-13T09:12:37.810 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011599412 2024-09-13T09:12:37.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.810 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.810 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.810 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332110+0000", 2024-09-13T09:12:37.810 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8318000000000005e-05 2024-09-13T09:12:37.811 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.811 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.811 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.811 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332208+0000", 2024-09-13T09:12:37.811 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7449999999999997e-05 2024-09-13T09:12:37.811 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.811 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.811 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.811 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.811 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.811 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1266 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.811 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320437+0000", 2024-09-13T09:12:37.811 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.416383318999999, 2024-09-13T09:12:37.812 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012024399, 2024-09-13T09:12:37.812 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.812 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.812 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.812 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.812 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.812 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1266 2024-09-13T09:12:37.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.812 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.812 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.812 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320437+0000", 2024-09-13T09:12:37.812 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.813 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.813 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320437+0000", 2024-09-13T09:12:37.813 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.813 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.813 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320437+0000", 2024-09-13T09:12:37.813 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.813 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.813 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320439+0000", 2024-09-13T09:12:37.814 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5409999999999999e-06 2024-09-13T09:12:37.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.814 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.814 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320447+0000", 2024-09-13T09:12:37.814 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7379999999999994e-06 2024-09-13T09:12:37.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.814 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.814 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320452+0000", 2024-09-13T09:12:37.814 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6909999999999996e-06 2024-09-13T09:12:37.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.814 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.815 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332249+0000", 2024-09-13T09:12:37.815 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011797626 2024-09-13T09:12:37.815 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.815 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.815 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.815 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332339+0000", 2024-09-13T09:12:37.815 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9959999999999999e-05 2024-09-13T09:12:37.815 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.815 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.815 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.815 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332462+0000", 2024-09-13T09:12:37.815 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012244999999999999 2024-09-13T09:12:37.815 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.816 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.816 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.816 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.816 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1267 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.816 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320456+0000", 2024-09-13T09:12:37.816 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.416364633000001, 2024-09-13T09:12:37.816 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012214348, 2024-09-13T09:12:37.816 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.816 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.816 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.816 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.816 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.817 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1267 2024-09-13T09:12:37.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.817 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.817 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.817 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320456+0000", 2024-09-13T09:12:37.817 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.817 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.817 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320456+0000", 2024-09-13T09:12:37.817 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320456+0000", 2024-09-13T09:12:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320458+0000", 2024-09-13T09:12:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:12:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320464+0000", 2024-09-13T09:12:37.818 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1079999999999998e-06 2024-09-13T09:12:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320469+0000", 2024-09-13T09:12:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7600000000000002e-06 2024-09-13T09:12:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332515+0000", 2024-09-13T09:12:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012045907 2024-09-13T09:12:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.819 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332601+0000", 2024-09-13T09:12:37.820 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5568000000000005e-05 2024-09-13T09:12:37.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.820 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.820 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.820 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332670+0000", 2024-09-13T09:12:37.820 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9958999999999999e-05 2024-09-13T09:12:37.820 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.820 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.820 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.820 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.820 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1268 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.820 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320474+0000", 2024-09-13T09:12:37.821 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.416347158000001, 2024-09-13T09:12:37.821 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012370015, 2024-09-13T09:12:37.821 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.821 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.821 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.821 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.821 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.821 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1268 2024-09-13T09:12:37.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.821 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.821 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.821 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320474+0000", 2024-09-13T09:12:37.821 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.822 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.822 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.822 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320474+0000", 2024-09-13T09:12:37.822 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.822 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.822 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.822 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320473+0000", 2024-09-13T09:12:37.822 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.822 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.822 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.823 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320476+0000", 2024-09-13T09:12:37.823 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:12:37.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.823 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.823 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.823 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320483+0000", 2024-09-13T09:12:37.823 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0770000000000002e-06 2024-09-13T09:12:37.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.823 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.823 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.823 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320488+0000", 2024-09-13T09:12:37.823 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2240000000000003e-06 2024-09-13T09:12:37.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.824 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.824 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332713+0000", 2024-09-13T09:12:37.824 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012225592 2024-09-13T09:12:37.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.824 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.824 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332766+0000", 2024-09-13T09:12:37.824 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2203000000000001e-05 2024-09-13T09:12:37.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.824 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.825 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332844+0000", 2024-09-13T09:12:37.825 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8016000000000007e-05 2024-09-13T09:12:37.825 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.825 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.825 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.825 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:1269 3.b 3:d4c010b4:::rbd_data.11238839e366.0000000000000007:6 [sparse-read 15990784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:37.825 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.320492+0000", 2024-09-13T09:12:37.825 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.416328519, 2024-09-13T09:12:37.825 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012500928, 2024-09-13T09:12:37.825 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:37.825 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:37.826 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:37.826 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:37.826 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:37.826 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1269 2024-09-13T09:12:37.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.826 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:37.826 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.826 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:37.826 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320492+0000", 2024-09-13T09:12:37.826 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.826 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.826 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:37.827 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320492+0000", 2024-09-13T09:12:37.827 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:37.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.827 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.827 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:37.827 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320492+0000", 2024-09-13T09:12:37.827 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:37.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.827 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.827 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:37.827 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320494+0000", 2024-09-13T09:12:37.827 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.435e-06 2024-09-13T09:12:37.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.827 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.828 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:37.828 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320502+0000", 2024-09-13T09:12:37.828 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.571e-06 2024-09-13T09:12:37.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.828 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:37.828 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.320507+0000", 2024-09-13T09:12:37.828 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7589999999999997e-06 2024-09-13T09:12:37.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.828 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:37.828 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332874+0000", 2024-09-13T09:12:37.828 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012366902000000001 2024-09-13T09:12:37.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.829 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:37.829 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332921+0000", 2024-09-13T09:12:37.829 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7615999999999999e-05 2024-09-13T09:12:37.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:37.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:37.829 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:37.829 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.332993+0000", 2024-09-13T09:12:37.829 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2000000000000002e-05 2024-09-13T09:12:37.829 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.829 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.829 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.830 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:37.830 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:37.830 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:37.831 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:37.832 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:37.830+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:37.932 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:37.931+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:37.980 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:37.980 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:37.980 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:37.980 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:37.987 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:38.032 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:38.030+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:38.092 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:38.092 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:38.092 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:38.092 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:38.092 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:38.101 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:38.133 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:38.132+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:38.234 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:38.232+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:38.250 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:38.250 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:38.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:38.250 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:38.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.250 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:285 3.7 3:e76cef2e:::rbd_data.1123ffac6bc2.00000000000000c2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:38.250 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.778209+0000", 2024-09-13T09:12:38.250 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.469691982000001, 2024-09-13T09:12:38.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051374489999999997, 2024-09-13T09:12:38.250 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.250 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.250 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.251 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:38.251 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:38.251 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 285 2024-09-13T09:12:38.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.251 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778209+0000", 2024-09-13T09:12:38.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778209+0000", 2024-09-13T09:12:38.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778209+0000", 2024-09-13T09:12:38.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:38.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778213+0000", 2024-09-13T09:12:38.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.301e-06 2024-09-13T09:12:38.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778226+0000", 2024-09-13T09:12:38.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2923e-05 2024-09-13T09:12:38.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778233+0000", 2024-09-13T09:12:38.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5739999999999998e-06 2024-09-13T09:12:38.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778279+0000", 2024-09-13T09:12:38.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5300000000000003e-05 2024-09-13T09:12:38.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778352+0000", 2024-09-13T09:12:38.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2960999999999995e-05 2024-09-13T09:12:38.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:38.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778461+0000", 2024-09-13T09:12:38.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010903700000000001 2024-09-13T09:12:38.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:38.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.780559+0000", 2024-09-13T09:12:38.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020987150000000001 2024-09-13T09:12:38.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783266+0000", 2024-09-13T09:12:38.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002707096 2024-09-13T09:12:38.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783307+0000", 2024-09-13T09:12:38.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0043999999999998e-05 2024-09-13T09:12:38.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783347+0000", 2024-09-13T09:12:38.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0333000000000002e-05 2024-09-13T09:12:38.256 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.256 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.256 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.256 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:288 3.1e 3:7da6a7d6:::rbd_data.1123ffac6bc2.00000000000000cb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:38.256 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.778243+0000", 2024-09-13T09:12:38.256 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.469658634999998, 2024-09-13T09:12:38.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050784439999999997, 2024-09-13T09:12:38.256 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.256 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.257 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.257 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:38.257 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:38.257 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 288 2024-09-13T09:12:38.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.257 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778243+0000", 2024-09-13T09:12:38.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778243+0000", 2024-09-13T09:12:38.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778242+0000", 2024-09-13T09:12:38.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778245+0000", 2024-09-13T09:12:38.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9909999999999998e-06 2024-09-13T09:12:38.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778264+0000", 2024-09-13T09:12:38.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8698999999999998e-05 2024-09-13T09:12:38.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778268+0000", 2024-09-13T09:12:38.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.002e-06 2024-09-13T09:12:38.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778302+0000", 2024-09-13T09:12:38.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3926e-05 2024-09-13T09:12:38.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778348+0000", 2024-09-13T09:12:38.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6458999999999997e-05 2024-09-13T09:12:38.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:38.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778427+0000", 2024-09-13T09:12:38.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8645000000000003e-05 2024-09-13T09:12:38.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:38.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.779006+0000", 2024-09-13T09:12:38.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00057902799999999999 2024-09-13T09:12:38.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783241+0000", 2024-09-13T09:12:38.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042353790000000001 2024-09-13T09:12:38.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783286+0000", 2024-09-13T09:12:38.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4537999999999999e-05 2024-09-13T09:12:38.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783321+0000", 2024-09-13T09:12:38.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5308999999999997e-05 2024-09-13T09:12:38.262 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.262 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.262 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.262 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:290 3.8 3:17d1c290:::rbd_data.1123ffac6bc2.00000000000000e6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:38.262 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.778368+0000", 2024-09-13T09:12:38.262 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.46953366, 2024-09-13T09:12:38.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049790379999999999, 2024-09-13T09:12:38.262 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.262 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.262 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.262 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:38.262 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 290 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778368+0000", 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778368+0000", 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778367+0000", 2024-09-13T09:12:38.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778371+0000", 2024-09-13T09:12:38.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.98e-06 2024-09-13T09:12:38.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778382+0000", 2024-09-13T09:12:38.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:12:38.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778388+0000", 2024-09-13T09:12:38.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3459999999999999e-06 2024-09-13T09:12:38.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778425+0000", 2024-09-13T09:12:38.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6721000000000003e-05 2024-09-13T09:12:38.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778486+0000", 2024-09-13T09:12:38.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1344000000000001e-05 2024-09-13T09:12:38.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:38.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.778585+0000", 2024-09-13T09:12:38.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8061999999999997e-05 2024-09-13T09:12:38.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:38.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.780584+0000", 2024-09-13T09:12:38.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019999649999999998 2024-09-13T09:12:38.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783298+0000", 2024-09-13T09:12:38.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027135779999999999 2024-09-13T09:12:38.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783326+0000", 2024-09-13T09:12:38.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7727999999999999e-05 2024-09-13T09:12:38.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.783347+0000", 2024-09-13T09:12:38.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0971e-05 2024-09-13T09:12:38.267 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.267 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.267 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.268 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:393 3.7 3:e1888e44:::rbd_data.1123ffac6bc2.00000000000000a4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:38.268 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.836837+0000", 2024-09-13T09:12:38.268 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.411064121999999, 2024-09-13T09:12:38.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048048379999999996, 2024-09-13T09:12:38.268 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.268 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.268 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.268 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:38.268 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:38.268 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 393 2024-09-13T09:12:38.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.268 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836837+0000", 2024-09-13T09:12:38.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836837+0000", 2024-09-13T09:12:38.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836835+0000", 2024-09-13T09:12:38.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-09-13T09:12:38.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836844+0000", 2024-09-13T09:12:38.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2930000000000001e-06 2024-09-13T09:12:38.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836860+0000", 2024-09-13T09:12:38.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5772000000000001e-05 2024-09-13T09:12:38.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836868+0000", 2024-09-13T09:12:38.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0090000000000007e-06 2024-09-13T09:12:38.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.836932+0000", 2024-09-13T09:12:38.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4336000000000002e-05 2024-09-13T09:12:38.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837025+0000", 2024-09-13T09:12:38.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3282999999999994e-05 2024-09-13T09:12:38.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:38.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.837149+0000", 2024-09-13T09:12:38.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000123776 2024-09-13T09:12:38.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:38.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.838672+0000", 2024-09-13T09:12:38.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015235579999999999 2024-09-13T09:12:38.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.841574+0000", 2024-09-13T09:12:38.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029013899999999998 2024-09-13T09:12:38.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.841609+0000", 2024-09-13T09:12:38.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5278999999999999e-05 2024-09-13T09:12:38.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.273 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.841642+0000", 2024-09-13T09:12:38.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2947999999999997e-05 2024-09-13T09:12:38.273 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.273 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.273 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.273 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:451 3.18 3:1c88c91a:::rbd_data.1123ffac6bc2.000000000000002d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:38.273 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.865164+0000", 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.382737099, 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051471370000000004, 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 451 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865164+0000", 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865164+0000", 2024-09-13T09:12:38.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865163+0000", 2024-09-13T09:12:38.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:38.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865169+0000", 2024-09-13T09:12:38.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.798e-06 2024-09-13T09:12:38.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865185+0000", 2024-09-13T09:12:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5404e-05 2024-09-13T09:12:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865193+0000", 2024-09-13T09:12:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:12:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865240+0000", 2024-09-13T09:12:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7293000000000003e-05 2024-09-13T09:12:38.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865331+0000", 2024-09-13T09:12:38.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0672000000000002e-05 2024-09-13T09:12:38.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865410+0000", 2024-09-13T09:12:38.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9319000000000004e-05 2024-09-13T09:12:38.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:38.278 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.867480+0000", 2024-09-13T09:12:38.278 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020703430000000001 2024-09-13T09:12:38.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.278 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.278 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870227+0000", 2024-09-13T09:12:38.278 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00274705 2024-09-13T09:12:38.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.278 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.278 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870272+0000", 2024-09-13T09:12:38.278 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4554000000000001e-05 2024-09-13T09:12:38.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.279 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.279 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870311+0000", 2024-09-13T09:12:38.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9536000000000002e-05 2024-09-13T09:12:38.279 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.279 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.279 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.279 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:453 3.7 3:e75bf08e:::rbd_data.1123ffac6bc2.000000000000002e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:38.279 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:19.865211+0000", 2024-09-13T09:12:38.279 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.382690707999998, 2024-09-13T09:12:38.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051785149999999999, 2024-09-13T09:12:38.279 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.279 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.280 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.280 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:38.280 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:38.280 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 453 2024-09-13T09:12:38.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.280 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.280 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.280 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865211+0000", 2024-09-13T09:12:38.280 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865211+0000", 2024-09-13T09:12:38.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865210+0000", 2024-09-13T09:12:38.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865213+0000", 2024-09-13T09:12:38.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:12:38.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.282 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.282 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865222+0000", 2024-09-13T09:12:38.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8380000000000008e-06 2024-09-13T09:12:38.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.282 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.282 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865228+0000", 2024-09-13T09:12:38.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9900000000000002e-06 2024-09-13T09:12:38.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.282 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.282 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865276+0000", 2024-09-13T09:12:38.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7407999999999997e-05 2024-09-13T09:12:38.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865364+0000", 2024-09-13T09:12:38.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8412000000000007e-05 2024-09-13T09:12:38.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:38.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.865471+0000", 2024-09-13T09:12:38.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000106609 2024-09-13T09:12:38.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:38.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.869879+0000", 2024-09-13T09:12:38.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044085640000000002 2024-09-13T09:12:38.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870308+0000", 2024-09-13T09:12:38.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00042895900000000002 2024-09-13T09:12:38.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870353+0000", 2024-09-13T09:12:38.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4749999999999997e-05 2024-09-13T09:12:38.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:19.870389+0000", 2024-09-13T09:12:38.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6269999999999997e-05 2024-09-13T09:12:38.285 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.285 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.285 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.285 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:1052 3.7 3:e7c11647:::rbd_data.1123ffac6bc2.0000000000000090:head [set-alloc-hint object_size 1048576 write_size 1048576,write 503808~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:38.286 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.140204+0000", 2024-09-13T09:12:38.286 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.107697773000002, 2024-09-13T09:12:38.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055287460000000002, 2024-09-13T09:12:38.286 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.286 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.286 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.286 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:38.286 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:38.286 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1052 2024-09-13T09:12:38.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.286 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.286 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.286 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140204+0000", 2024-09-13T09:12:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140204+0000", 2024-09-13T09:12:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140202+0000", 2024-09-13T09:12:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-09-13T09:12:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140242+0000", 2024-09-13T09:12:38.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0512000000000001e-05 2024-09-13T09:12:38.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140263+0000", 2024-09-13T09:12:38.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1268e-05 2024-09-13T09:12:38.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140277+0000", 2024-09-13T09:12:38.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3346000000000001e-05 2024-09-13T09:12:38.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140333+0000", 2024-09-13T09:12:38.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5695000000000001e-05 2024-09-13T09:12:38.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140461+0000", 2024-09-13T09:12:38.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012853800000000001 2024-09-13T09:12:38.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:38.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140645+0000", 2024-09-13T09:12:38.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000183463 2024-09-13T09:12:38.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:38.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145465+0000", 2024-09-13T09:12:38.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048209289999999998 2024-09-13T09:12:38.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145639+0000", 2024-09-13T09:12:38.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017306799999999999 2024-09-13T09:12:38.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145696+0000", 2024-09-13T09:12:38.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7599000000000002e-05 2024-09-13T09:12:38.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.291 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145732+0000", 2024-09-13T09:12:38.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6254000000000002e-05 2024-09-13T09:12:38.291 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.291 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.291 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.291 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4483.0:1055 3.18 3:192ac1d1:::rbd_data.1123ffac6bc2.0000000000000091:head [set-alloc-hint object_size 1048576 write_size 1048576,write 393216~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:12:38.291 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.140290+0000", 2024-09-13T09:12:38.291 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.107610901000001, 2024-09-13T09:12:38.292 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.005447454, 2024-09-13T09:12:38.292 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.292 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4483", 2024-09-13T09:12:38.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3874616254", 2024-09-13T09:12:38.292 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1055 2024-09-13T09:12:38.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.292 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.292 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.292 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140290+0000", 2024-09-13T09:12:38.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140290+0000", 2024-09-13T09:12:38.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140290+0000", 2024-09-13T09:12:38.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140311+0000", 2024-09-13T09:12:38.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1484e-05 2024-09-13T09:12:38.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140320+0000", 2024-09-13T09:12:38.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1500000000000005e-06 2024-09-13T09:12:38.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140324+0000", 2024-09-13T09:12:38.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:12:38.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140375+0000", 2024-09-13T09:12:38.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1082000000000003e-05 2024-09-13T09:12:38.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140504+0000", 2024-09-13T09:12:38.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012903599999999999 2024-09-13T09:12:38.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.140608+0000", 2024-09-13T09:12:38.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000103669 2024-09-13T09:12:38.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:38.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142875+0000", 2024-09-13T09:12:38.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002267388 2024-09-13T09:12:38.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145639+0000", 2024-09-13T09:12:38.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027635369999999999 2024-09-13T09:12:38.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.297 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.297 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145700+0000", 2024-09-13T09:12:38.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1453999999999998e-05 2024-09-13T09:12:38.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.297 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.297 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.145738+0000", 2024-09-13T09:12:38.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7478999999999998e-05 2024-09-13T09:12:38.297 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.297 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.297 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.297 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4483.0:1069 3.5 e27/21 3:a4a542ae:::rbd_data.1123ffac6bc2.0000000000000036:head v 27'123, mlcod=27'121)", 2024-09-13T09:12:38.298 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:20.142984+0000", 2024-09-13T09:12:38.298 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.104917613000001, 2024-09-13T09:12:38.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052211899999999997, 2024-09-13T09:12:38.298 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.298 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.298 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142984+0000", 2024-09-13T09:12:38.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142984+0000", 2024-09-13T09:12:38.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142983+0000", 2024-09-13T09:12:38.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:38.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.142996+0000", 2024-09-13T09:12:38.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3361e-05 2024-09-13T09:12:38.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143012+0000", 2024-09-13T09:12:38.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5574999999999999e-05 2024-09-13T09:12:38.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143019+0000", 2024-09-13T09:12:38.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7290000000000003e-06 2024-09-13T09:12:38.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143066+0000", 2024-09-13T09:12:38.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7250999999999999e-05 2024-09-13T09:12:38.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.143088+0000", 2024-09-13T09:12:38.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2450999999999999e-05 2024-09-13T09:12:38.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148171+0000", 2024-09-13T09:12:38.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050823190000000001 2024-09-13T09:12:38.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.302 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:20.148205+0000", 2024-09-13T09:12:38.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4350000000000001e-05 2024-09-13T09:12:38.302 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.302 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.302 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.302 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:396 3.8 3:1239702a:::rbd_data.11238839e366.0000000000000006:6 [sparse-read 16515072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:38.302 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.282786+0000", 2024-09-13T09:12:38.302 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.96511576, 2024-09-13T09:12:38.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046983729999999996, 2024-09-13T09:12:38.302 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.302 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:38.303 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.303 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:38.303 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:38.303 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 396 2024-09-13T09:12:38.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.303 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.282786+0000", 2024-09-13T09:12:38.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.282786+0000", 2024-09-13T09:12:38.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.282785+0000", 2024-09-13T09:12:38.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.282787+0000", 2024-09-13T09:12:38.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.624e-06 2024-09-13T09:12:38.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.282792+0000", 2024-09-13T09:12:38.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2780000000000001e-06 2024-09-13T09:12:38.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.282796+0000", 2024-09-13T09:12:38.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5949999999999999e-06 2024-09-13T09:12:38.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.287333+0000", 2024-09-13T09:12:38.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045368270000000002 2024-09-13T09:12:38.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.306 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.287406+0000", 2024-09-13T09:12:38.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2854000000000004e-05 2024-09-13T09:12:38.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.306 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.287484+0000", 2024-09-13T09:12:38.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8499999999999997e-05 2024-09-13T09:12:38.306 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.306 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.306 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.307 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4537.0:503 3.8 3:1239702a:::rbd_data.11238839e366.0000000000000006:6 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:12:38.307 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:24.284527+0000", 2024-09-13T09:12:38.307 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.963373967000001, 2024-09-13T09:12:38.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004859057, 2024-09-13T09:12:38.307 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.307 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:38.307 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.307 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4537", 2024-09-13T09:12:38.307 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3280777304", 2024-09-13T09:12:38.307 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 503 2024-09-13T09:12:38.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.307 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.284527+0000", 2024-09-13T09:12:38.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.284527+0000", 2024-09-13T09:12:38.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.284527+0000", 2024-09-13T09:12:38.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.284534+0000", 2024-09-13T09:12:38.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3679999999999996e-06 2024-09-13T09:12:38.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.284541+0000", 2024-09-13T09:12:38.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3470000000000004e-06 2024-09-13T09:12:38.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.284546+0000", 2024-09-13T09:12:38.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1479999999999997e-06 2024-09-13T09:12:38.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.289186+0000", 2024-09-13T09:12:38.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046402539999999999 2024-09-13T09:12:38.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.289278+0000", 2024-09-13T09:12:38.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1766000000000002e-05 2024-09-13T09:12:38.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:24.289386+0000", 2024-09-13T09:12:38.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0001087 2024-09-13T09:12:38.311 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.311 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.311 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.311 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:1997 3.8 3:17d36caf:::rbd_data.1123b8b4a28a.0000000000000012:head [set-alloc-hint object_size 2097152 write_size 2097152,write 524288~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:38.311 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.507177+0000", 2024-09-13T09:12:38.311 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.740724220000001, 2024-09-13T09:12:38.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0085576109999999997, 2024-09-13T09:12:38.311 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.311 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.311 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.311 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:38.312 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:38.312 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1997 2024-09-13T09:12:38.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.312 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507177+0000", 2024-09-13T09:12:38.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507177+0000", 2024-09-13T09:12:38.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507175+0000", 2024-09-13T09:12:38.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-09-13T09:12:38.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507217+0000", 2024-09-13T09:12:38.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2543000000000002e-05 2024-09-13T09:12:38.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507238+0000", 2024-09-13T09:12:38.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0445000000000002e-05 2024-09-13T09:12:38.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507247+0000", 2024-09-13T09:12:38.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5959999999999993e-06 2024-09-13T09:12:38.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507299+0000", 2024-09-13T09:12:38.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1966999999999999e-05 2024-09-13T09:12:38.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507448+0000", 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014851600000000001 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.507598+0000", 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015040100000000001 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.513345+0000", 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057464170000000002 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515644+0000", 2024-09-13T09:12:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022998300000000001 2024-09-13T09:12:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515697+0000", 2024-09-13T09:12:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2210000000000003e-05 2024-09-13T09:12:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.515735+0000", 2024-09-13T09:12:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8126000000000001e-05 2024-09-13T09:12:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2296 3.7 3:e05a98c0:::rbd_data.1123b8b4a28a.000000000000002f:head [write 1933312~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.617024+0000", 2024-09-13T09:12:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.630877025, 2024-09-13T09:12:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0054662820000000003, 2024-09-13T09:12:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2296 2024-09-13T09:12:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617024+0000", 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617024+0000", 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617023+0000", 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617043+0000", 2024-09-13T09:12:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9673e-05 2024-09-13T09:12:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617058+0000", 2024-09-13T09:12:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4966999999999999e-05 2024-09-13T09:12:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617066+0000", 2024-09-13T09:12:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7069999999999992e-06 2024-09-13T09:12:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617138+0000", 2024-09-13T09:12:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2360999999999994e-05 2024-09-13T09:12:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617213+0000", 2024-09-13T09:12:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5009e-05 2024-09-13T09:12:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.617356+0000", 2024-09-13T09:12:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014259 2024-09-13T09:12:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.618868+0000", 2024-09-13T09:12:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015122480000000001 2024-09-13T09:12:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.622378+0000", 2024-09-13T09:12:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0035100069999999999 2024-09-13T09:12:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.622426+0000", 2024-09-13T09:12:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8106999999999999e-05 2024-09-13T09:12:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.622491+0000", 2024-09-13T09:12:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4759999999999997e-05 2024-09-13T09:12:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4567.0:2311 3.9 e33/21 3:93ffaa8c:::rbd_data.1123b8b4a28a.0000000000000041:head v 33'32, mlcod=33'30)", 2024-09-13T09:12:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.619681+0000", 2024-09-13T09:12:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.62822048, 2024-09-13T09:12:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049754830000000002, 2024-09-13T09:12:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619681+0000", 2024-09-13T09:12:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619681+0000", 2024-09-13T09:12:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619680+0000", 2024-09-13T09:12:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619692+0000", 2024-09-13T09:12:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.24e-05 2024-09-13T09:12:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619708+0000", 2024-09-13T09:12:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5475e-05 2024-09-13T09:12:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619715+0000", 2024-09-13T09:12:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1219999999999997e-06 2024-09-13T09:12:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619781+0000", 2024-09-13T09:12:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5957999999999994e-05 2024-09-13T09:12:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.619801+0000", 2024-09-13T09:12:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9658999999999999e-05 2024-09-13T09:12:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.624615+0000", 2024-09-13T09:12:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048141399999999997 2024-09-13T09:12:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.624656+0000", 2024-09-13T09:12:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1619000000000003e-05 2024-09-13T09:12:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4567.0:2316 3.1f e33/22 3:f905e326:::rbd_data.1123b8b4a28a.0000000000000042:head v 33'45, mlcod=32'43)", 2024-09-13T09:12:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.620013+0000", 2024-09-13T09:12:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.62788795, 2024-09-13T09:12:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047162929999999999, 2024-09-13T09:12:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620013+0000", 2024-09-13T09:12:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620013+0000", 2024-09-13T09:12:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620013+0000", 2024-09-13T09:12:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620029+0000", 2024-09-13T09:12:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6342e-05 2024-09-13T09:12:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620044+0000", 2024-09-13T09:12:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4958e-05 2024-09-13T09:12:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620050+0000", 2024-09-13T09:12:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2439999999999998e-06 2024-09-13T09:12:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620115+0000", 2024-09-13T09:12:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4462999999999996e-05 2024-09-13T09:12:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.620134+0000", 2024-09-13T09:12:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9692999999999999e-05 2024-09-13T09:12:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.624698+0000", 2024-09-13T09:12:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045634120000000002 2024-09-13T09:12:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.624730+0000", 2024-09-13T09:12:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2027999999999999e-05 2024-09-13T09:12:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2436 3.18 3:195218e5:::rbd_data.1123b8b4a28a.0000000000000039:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.674604+0000", 2024-09-13T09:12:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.573296912, 2024-09-13T09:12:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055259439999999996, 2024-09-13T09:12:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2436 2024-09-13T09:12:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674604+0000", 2024-09-13T09:12:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674604+0000", 2024-09-13T09:12:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674603+0000", 2024-09-13T09:12:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674609+0000", 2024-09-13T09:12:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0660000000000003e-06 2024-09-13T09:12:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674621+0000", 2024-09-13T09:12:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.202e-05 2024-09-13T09:12:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674628+0000", 2024-09-13T09:12:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5020000000000003e-06 2024-09-13T09:12:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674677+0000", 2024-09-13T09:12:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9211e-05 2024-09-13T09:12:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674774+0000", 2024-09-13T09:12:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6341000000000005e-05 2024-09-13T09:12:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.674899+0000", 2024-09-13T09:12:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000125537 2024-09-13T09:12:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.676820+0000", 2024-09-13T09:12:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019208109999999999 2024-09-13T09:12:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680047+0000", 2024-09-13T09:12:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0032271130000000002 2024-09-13T09:12:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680095+0000", 2024-09-13T09:12:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7691999999999997e-05 2024-09-13T09:12:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680130+0000", 2024-09-13T09:12:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5651999999999997e-05 2024-09-13T09:12:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2440 3.18 3:1b8df09d:::rbd_data.1123b8b4a28a.000000000000003c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.675520+0000", 2024-09-13T09:12:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.57238141, 2024-09-13T09:12:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047183549999999996, 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2440 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675520+0000", 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675520+0000", 2024-09-13T09:12:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675519+0000", 2024-09-13T09:12:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675523+0000", 2024-09-13T09:12:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:12:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675532+0000", 2024-09-13T09:12:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:12:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675537+0000", 2024-09-13T09:12:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.429e-06 2024-09-13T09:12:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675903+0000", 2024-09-13T09:12:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000366051 2024-09-13T09:12:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.675977+0000", 2024-09-13T09:12:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3775000000000004e-05 2024-09-13T09:12:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.676093+0000", 2024-09-13T09:12:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000115912 2024-09-13T09:12:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.676826+0000", 2024-09-13T09:12:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00073320999999999998 2024-09-13T09:12:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680167+0000", 2024-09-13T09:12:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033408650000000002 2024-09-13T09:12:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680207+0000", 2024-09-13T09:12:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9901999999999998e-05 2024-09-13T09:12:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680238+0000", 2024-09-13T09:12:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1587000000000001e-05 2024-09-13T09:12:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(client.4567.0:2454 3.9 e33/21 3:96917efd:::rbd_data.1123b8b4a28a.0000000000000066:head v 33'37, mlcod=33'33)", 2024-09-13T09:12:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.677108+0000", 2024-09-13T09:12:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.570793031999999, 2024-09-13T09:12:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046617639999999997, 2024-09-13T09:12:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677108+0000", 2024-09-13T09:12:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677108+0000", 2024-09-13T09:12:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677108+0000", 2024-09-13T09:12:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677113+0000", 2024-09-13T09:12:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7309999999999998e-06 2024-09-13T09:12:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677122+0000", 2024-09-13T09:12:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9080000000000001e-06 2024-09-13T09:12:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677137+0000", 2024-09-13T09:12:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4358e-05 2024-09-13T09:12:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677175+0000", 2024-09-13T09:12:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8696999999999999e-05 2024-09-13T09:12:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.677192+0000", 2024-09-13T09:12:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6328999999999999e-05 2024-09-13T09:12:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.681720+0000", 2024-09-13T09:12:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00452799 2024-09-13T09:12:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.681770+0000", 2024-09-13T09:12:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0386999999999997e-05 2024-09-13T09:12:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2486 3.7 3:e1179f14:::rbd_header.1123b8b4a28a:head [notify cookie 140249875563536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.680326+0000", 2024-09-13T09:12:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.567575176, 2024-09-13T09:12:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046642200000000002, 2024-09-13T09:12:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2486 2024-09-13T09:12:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680326+0000", 2024-09-13T09:12:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680326+0000", 2024-09-13T09:12:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680325+0000", 2024-09-13T09:12:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680331+0000", 2024-09-13T09:12:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7880000000000003e-06 2024-09-13T09:12:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680343+0000", 2024-09-13T09:12:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2026e-05 2024-09-13T09:12:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680350+0000", 2024-09-13T09:12:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9149999999999999e-06 2024-09-13T09:12:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.684893+0000", 2024-09-13T09:12:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045435379999999997 2024-09-13T09:12:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.684927+0000", 2024-09-13T09:12:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3200000000000001e-05 2024-09-13T09:12:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.684990+0000", 2024-09-13T09:12:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3870999999999999e-05 2024-09-13T09:12:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4567.0:2490 3.7 3:e1179f14:::rbd_header.1123b8b4a28a:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:25.680585+0000", 2024-09-13T09:12:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.567316624, 2024-09-13T09:12:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047373210000000001, 2024-09-13T09:12:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4567", 2024-09-13T09:12:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2490 2024-09-13T09:12:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680585+0000", 2024-09-13T09:12:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680585+0000", 2024-09-13T09:12:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680584+0000", 2024-09-13T09:12:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680615+0000", 2024-09-13T09:12:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0546000000000002e-05 2024-09-13T09:12:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680624+0000", 2024-09-13T09:12:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6320000000000003e-06 2024-09-13T09:12:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.680630+0000", 2024-09-13T09:12:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7130000000000002e-06 2024-09-13T09:12:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.685242+0000", 2024-09-13T09:12:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046121699999999996 2024-09-13T09:12:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.685277+0000", 2024-09-13T09:12:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4307999999999997e-05 2024-09-13T09:12:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:25.685322+0000", 2024-09-13T09:12:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5438000000000001e-05 2024-09-13T09:12:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:38.357 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:38.358 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:38.359 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:38.357+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:38.459 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:38.458+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:38.464 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:38.464 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:38.464 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:38.464 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:38.473 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:38.560 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:38.560+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:38.575 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone11 from 0x8885a00 (overlap 0x447200) to 0x223800 2024-09-13T09:12:38.620 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:38.621 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:38.621 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:38.621 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:38.621 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:38.630 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:38.661 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:38.659+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:38.763 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:38.762+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:38.774 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:38.774 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:38.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:38.774 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:38.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.774 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:38.774 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:11:59.289661+0000", 2024-09-13T09:12:38.774 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.482100514000003, 2024-09-13T09:12:38.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.93386499199999995, 2024-09-13T09:12:38.775 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.775 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.775 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1 2024-09-13T09:12:38.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.775 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289661+0000", 2024-09-13T09:12:38.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289661+0000", 2024-09-13T09:12:38.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289659+0000", 2024-09-13T09:12:38.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:38.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289669+0000", 2024-09-13T09:12:38.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:12:38.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289681+0000", 2024-09-13T09:12:38.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2316e-05 2024-09-13T09:12:38.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289693+0000", 2024-09-13T09:12:38.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2549e-05 2024-09-13T09:12:38.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289744+0000", 2024-09-13T09:12:38.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0365999999999999e-05 2024-09-13T09:12:38.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:38.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:11:59.289752+0000", 2024-09-13T09:12:38.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5590000000000006e-06 2024-09-13T09:12:38.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.218301+0000", 2024-09-13T09:12:38.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.928549085 2024-09-13T09:12:38.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.218447+0000", 2024-09-13T09:12:38.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014537400000000001 2024-09-13T09:12:38.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.218615+0000", 2024-09-13T09:12:38.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000168195 2024-09-13T09:12:38.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.219277+0000", 2024-09-13T09:12:38.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00066202600000000002 2024-09-13T09:12:38.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.223461+0000", 2024-09-13T09:12:38.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004184152 2024-09-13T09:12:38.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.223500+0000", 2024-09-13T09:12:38.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8733000000000003e-05 2024-09-13T09:12:38.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.223526+0000", 2024-09-13T09:12:38.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5612000000000001e-05 2024-09-13T09:12:38.781 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.781 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.781 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.781 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:38.781 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:38.781 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.524384009000002, 2024-09-13T09:12:38.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079358310000000008, 2024-09-13T09:12:38.781 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.781 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.781 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.782 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.782 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.782 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35 2024-09-13T09:12:38.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.782 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:38.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:38.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247377+0000", 2024-09-13T09:12:38.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:38.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247410+0000", 2024-09-13T09:12:38.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3053999999999999e-05 2024-09-13T09:12:38.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247420+0000", 2024-09-13T09:12:38.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7189999999999996e-06 2024-09-13T09:12:38.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247429+0000", 2024-09-13T09:12:38.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5340000000000005e-06 2024-09-13T09:12:38.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.249500+0000", 2024-09-13T09:12:38.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020708990000000002 2024-09-13T09:12:38.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.249551+0000", 2024-09-13T09:12:38.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0762e-05 2024-09-13T09:12:38.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.249649+0000", 2024-09-13T09:12:38.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.768e-05 2024-09-13T09:12:38.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.251260+0000", 2024-09-13T09:12:38.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001611101 2024-09-13T09:12:38.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255178+0000", 2024-09-13T09:12:38.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039180930000000001 2024-09-13T09:12:38.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255256+0000", 2024-09-13T09:12:38.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8572000000000006e-05 2024-09-13T09:12:38.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255313+0000", 2024-09-13T09:12:38.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6625000000000001e-05 2024-09-13T09:12:38.786 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.787 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.787 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.787 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:38.787 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:38.787 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.524326831000003, 2024-09-13T09:12:38.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079428840000000007, 2024-09-13T09:12:38.787 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.787 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.787 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.787 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.787 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.788 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36 2024-09-13T09:12:38.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.788 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:38.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:38.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247434+0000", 2024-09-13T09:12:38.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247438+0000", 2024-09-13T09:12:38.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7409999999999998e-06 2024-09-13T09:12:38.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247442+0000", 2024-09-13T09:12:38.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:12:38.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247444+0000", 2024-09-13T09:12:38.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8880000000000001e-06 2024-09-13T09:12:38.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250381+0000", 2024-09-13T09:12:38.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0029366800000000001 2024-09-13T09:12:38.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250446+0000", 2024-09-13T09:12:38.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5173999999999996e-05 2024-09-13T09:12:38.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250546+0000", 2024-09-13T09:12:38.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100142 2024-09-13T09:12:38.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.251514+0000", 2024-09-13T09:12:38.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00096772000000000004 2024-09-13T09:12:38.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255336+0000", 2024-09-13T09:12:38.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038214719999999998 2024-09-13T09:12:38.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255358+0000", 2024-09-13T09:12:38.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2141e-05 2024-09-13T09:12:38.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255377+0000", 2024-09-13T09:12:38.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9398999999999998e-05 2024-09-13T09:12:38.793 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.793 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.793 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.793 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:38.793 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:38.793 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.524312229000003, 2024-09-13T09:12:38.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079696799999999998, 2024-09-13T09:12:38.793 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.793 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.793 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.793 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.793 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.794 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37 2024-09-13T09:12:38.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.794 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:38.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:38.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247449+0000", 2024-09-13T09:12:38.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:38.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247451+0000", 2024-09-13T09:12:38.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6000000000000001e-06 2024-09-13T09:12:38.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247463+0000", 2024-09-13T09:12:38.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1609e-05 2024-09-13T09:12:38.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.247473+0000", 2024-09-13T09:12:38.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0536999999999999e-05 2024-09-13T09:12:38.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250802+0000", 2024-09-13T09:12:38.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033287270000000001 2024-09-13T09:12:38.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250853+0000", 2024-09-13T09:12:38.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1329000000000003e-05 2024-09-13T09:12:38.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.250935+0000", 2024-09-13T09:12:38.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1396999999999993e-05 2024-09-13T09:12:38.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.251796+0000", 2024-09-13T09:12:38.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00086075099999999999 2024-09-13T09:12:38.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255382+0000", 2024-09-13T09:12:38.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003586334 2024-09-13T09:12:38.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255402+0000", 2024-09-13T09:12:38.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0392000000000001e-05 2024-09-13T09:12:38.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.255419+0000", 2024-09-13T09:12:38.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6229e-05 2024-09-13T09:12:38.798 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.798 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.798 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.799 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:38.799 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:38.799 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.496378135, 2024-09-13T09:12:38.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077389640000000001, 2024-09-13T09:12:38.799 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.799 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.799 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.799 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.799 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.799 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 56 2024-09-13T09:12:38.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.800 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:38.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:38.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275383+0000", 2024-09-13T09:12:38.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275385+0000", 2024-09-13T09:12:38.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:12:38.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275388+0000", 2024-09-13T09:12:38.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3739999999999998e-06 2024-09-13T09:12:38.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.275396+0000", 2024-09-13T09:12:38.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0099999999999995e-06 2024-09-13T09:12:38.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.276666+0000", 2024-09-13T09:12:38.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00126968 2024-09-13T09:12:38.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.276710+0000", 2024-09-13T09:12:38.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4141999999999999e-05 2024-09-13T09:12:38.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.276785+0000", 2024-09-13T09:12:38.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4777000000000006e-05 2024-09-13T09:12:38.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.282050+0000", 2024-09-13T09:12:38.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0052647320000000003 2024-09-13T09:12:38.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283071+0000", 2024-09-13T09:12:38.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001021196 2024-09-13T09:12:38.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283102+0000", 2024-09-13T09:12:38.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0849999999999998e-05 2024-09-13T09:12:38.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283122+0000", 2024-09-13T09:12:38.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0190000000000002e-05 2024-09-13T09:12:38.804 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.804 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.804 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:58 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.488394620000001, 2024-09-13T09:12:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064599870000000004, 2024-09-13T09:12:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 58 2024-09-13T09:12:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:38.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:38.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283366+0000", 2024-09-13T09:12:38.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283398+0000", 2024-09-13T09:12:38.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1646000000000002e-05 2024-09-13T09:12:38.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283403+0000", 2024-09-13T09:12:38.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0540000000000002e-06 2024-09-13T09:12:38.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283406+0000", 2024-09-13T09:12:38.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3110000000000001e-06 2024-09-13T09:12:38.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283841+0000", 2024-09-13T09:12:38.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000434549 2024-09-13T09:12:38.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283900+0000", 2024-09-13T09:12:38.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9153000000000001e-05 2024-09-13T09:12:38.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283975+0000", 2024-09-13T09:12:38.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4967000000000002e-05 2024-09-13T09:12:38.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.285844+0000", 2024-09-13T09:12:38.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001869435 2024-09-13T09:12:38.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289769+0000", 2024-09-13T09:12:38.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039245230000000001 2024-09-13T09:12:38.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289804+0000", 2024-09-13T09:12:38.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4999000000000002e-05 2024-09-13T09:12:38.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289826+0000", 2024-09-13T09:12:38.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2857000000000002e-05 2024-09-13T09:12:38.810 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.810 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.810 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.810 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:59 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:38.811 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.283413+0000", 2024-09-13T09:12:38.811 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.488348275, 2024-09-13T09:12:38.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064987589999999998, 2024-09-13T09:12:38.811 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.811 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.811 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.811 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.811 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.811 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 59 2024-09-13T09:12:38.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.811 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283413+0000", 2024-09-13T09:12:38.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283413+0000", 2024-09-13T09:12:38.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283412+0000", 2024-09-13T09:12:38.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283416+0000", 2024-09-13T09:12:38.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:12:38.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283420+0000", 2024-09-13T09:12:38.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6789999999999996e-06 2024-09-13T09:12:38.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.283423+0000", 2024-09-13T09:12:38.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.672e-06 2024-09-13T09:12:38.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.284235+0000", 2024-09-13T09:12:38.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00081158900000000004 2024-09-13T09:12:38.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.284282+0000", 2024-09-13T09:12:38.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7213999999999999e-05 2024-09-13T09:12:38.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.284359+0000", 2024-09-13T09:12:38.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6814e-05 2024-09-13T09:12:38.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.285868+0000", 2024-09-13T09:12:38.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015097310000000001 2024-09-13T09:12:38.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289836+0000", 2024-09-13T09:12:38.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003967613 2024-09-13T09:12:38.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289877+0000", 2024-09-13T09:12:38.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1519e-05 2024-09-13T09:12:38.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.289912+0000", 2024-09-13T09:12:38.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4088999999999998e-05 2024-09-13T09:12:38.816 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.816 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.816 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.816 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:38.816 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.298116+0000", 2024-09-13T09:12:38.816 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.473645470999998, 2024-09-13T09:12:38.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063667139999999999, 2024-09-13T09:12:38.817 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.817 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.817 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.817 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.817 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.817 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 69 2024-09-13T09:12:38.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.817 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298116+0000", 2024-09-13T09:12:38.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298116+0000", 2024-09-13T09:12:38.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298115+0000", 2024-09-13T09:12:38.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298193+0000", 2024-09-13T09:12:38.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7441999999999995e-05 2024-09-13T09:12:38.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298213+0000", 2024-09-13T09:12:38.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0829999999999999e-05 2024-09-13T09:12:38.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298224+0000", 2024-09-13T09:12:38.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0282000000000001e-05 2024-09-13T09:12:38.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299016+0000", 2024-09-13T09:12:38.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00079248999999999997 2024-09-13T09:12:38.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299098+0000", 2024-09-13T09:12:38.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1607999999999995e-05 2024-09-13T09:12:38.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299223+0000", 2024-09-13T09:12:38.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000124838 2024-09-13T09:12:38.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.300536+0000", 2024-09-13T09:12:38.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001312843 2024-09-13T09:12:38.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304428+0000", 2024-09-13T09:12:38.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0038928000000000001 2024-09-13T09:12:38.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304463+0000", 2024-09-13T09:12:38.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4214999999999997e-05 2024-09-13T09:12:38.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304482+0000", 2024-09-13T09:12:38.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9800999999999999e-05 2024-09-13T09:12:38.822 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.822 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.822 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.822 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:38.822 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.298237+0000", 2024-09-13T09:12:38.822 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.473524503, 2024-09-13T09:12:38.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063589789999999998, 2024-09-13T09:12:38.822 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.822 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.823 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.823 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.823 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.823 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 70 2024-09-13T09:12:38.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.823 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298237+0000", 2024-09-13T09:12:38.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298237+0000", 2024-09-13T09:12:38.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298236+0000", 2024-09-13T09:12:38.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:38.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298242+0000", 2024-09-13T09:12:38.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5889999999999999e-06 2024-09-13T09:12:38.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298252+0000", 2024-09-13T09:12:38.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6539999999999994e-06 2024-09-13T09:12:38.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.298257+0000", 2024-09-13T09:12:38.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4240000000000001e-06 2024-09-13T09:12:38.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299735+0000", 2024-09-13T09:12:38.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0014772120000000001 2024-09-13T09:12:38.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299851+0000", 2024-09-13T09:12:38.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000115998 2024-09-13T09:12:38.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.299975+0000", 2024-09-13T09:12:38.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000124805 2024-09-13T09:12:38.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.300830+0000", 2024-09-13T09:12:38.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0008543 2024-09-13T09:12:38.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304519+0000", 2024-09-13T09:12:38.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036896849999999998 2024-09-13T09:12:38.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304560+0000", 2024-09-13T09:12:38.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1009e-05 2024-09-13T09:12:38.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.304596+0000", 2024-09-13T09:12:38.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5114999999999998e-05 2024-09-13T09:12:38.828 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.828 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.828 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.828 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:85 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:38.828 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:38.828 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.459574103000001, 2024-09-13T09:12:38.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068319399999999999, 2024-09-13T09:12:38.828 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.828 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.828 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.828 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.829 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.829 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 85 2024-09-13T09:12:38.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.829 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:38.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:38.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312187+0000", 2024-09-13T09:12:38.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312190+0000", 2024-09-13T09:12:38.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2389999999999998e-06 2024-09-13T09:12:38.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312198+0000", 2024-09-13T09:12:38.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8599999999999993e-06 2024-09-13T09:12:38.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312201+0000", 2024-09-13T09:12:38.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.399e-06 2024-09-13T09:12:38.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.313614+0000", 2024-09-13T09:12:38.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001413369 2024-09-13T09:12:38.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.313702+0000", 2024-09-13T09:12:38.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7556e-05 2024-09-13T09:12:38.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.313843+0000", 2024-09-13T09:12:38.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000141015 2024-09-13T09:12:38.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318968+0000", 2024-09-13T09:12:38.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0051251680000000003 2024-09-13T09:12:38.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318973+0000", 2024-09-13T09:12:38.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9520000000000002e-06 2024-09-13T09:12:38.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318999+0000", 2024-09-13T09:12:38.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5706000000000001e-05 2024-09-13T09:12:38.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319019+0000", 2024-09-13T09:12:38.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0024e-05 2024-09-13T09:12:38.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.834 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.834 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.834 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:38.834 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:38.834 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.459556083000003, 2024-09-13T09:12:38.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069390709999999998, 2024-09-13T09:12:38.834 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.834 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.834 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 86 2024-09-13T09:12:38.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.835 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:38.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:38.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312205+0000", 2024-09-13T09:12:38.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312207+0000", 2024-09-13T09:12:38.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.243e-06 2024-09-13T09:12:38.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312211+0000", 2024-09-13T09:12:38.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1219999999999997e-06 2024-09-13T09:12:38.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312215+0000", 2024-09-13T09:12:38.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4680000000000001e-06 2024-09-13T09:12:38.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314378+0000", 2024-09-13T09:12:38.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021634309999999999 2024-09-13T09:12:38.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314426+0000", 2024-09-13T09:12:38.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7963000000000001e-05 2024-09-13T09:12:38.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314507+0000", 2024-09-13T09:12:38.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0619999999999997e-05 2024-09-13T09:12:38.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319024+0000", 2024-09-13T09:12:38.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045178140000000002 2024-09-13T09:12:38.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319065+0000", 2024-09-13T09:12:38.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0216999999999998e-05 2024-09-13T09:12:38.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319108+0000", 2024-09-13T09:12:38.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3618000000000001e-05 2024-09-13T09:12:38.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319144+0000", 2024-09-13T09:12:38.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5858999999999997e-05 2024-09-13T09:12:38.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.839 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.840 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:38.840 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:38.840 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.459541704999999, 2024-09-13T09:12:38.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011377922, 2024-09-13T09:12:38.840 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.840 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.840 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 87 2024-09-13T09:12:38.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.840 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:38.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:38.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312219+0000", 2024-09-13T09:12:38.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:38.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312249+0000", 2024-09-13T09:12:38.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9377e-05 2024-09-13T09:12:38.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312255+0000", 2024-09-13T09:12:38.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0220000000000001e-06 2024-09-13T09:12:38.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312259+0000", 2024-09-13T09:12:38.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1110000000000001e-06 2024-09-13T09:12:38.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314845+0000", 2024-09-13T09:12:38.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0025858750000000001 2024-09-13T09:12:38.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314891+0000", 2024-09-13T09:12:38.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6347999999999997e-05 2024-09-13T09:12:38.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.314968+0000", 2024-09-13T09:12:38.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7428999999999993e-05 2024-09-13T09:12:38.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319174+0000", 2024-09-13T09:12:38.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042056999999999997 2024-09-13T09:12:38.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323544+0000", 2024-09-13T09:12:38.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043701119999999998 2024-09-13T09:12:38.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323575+0000", 2024-09-13T09:12:38.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1285000000000003e-05 2024-09-13T09:12:38.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323597+0000", 2024-09-13T09:12:38.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1891000000000001e-05 2024-09-13T09:12:38.845 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.845 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.845 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.846 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-13T09:12:38.846 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:38.846 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.459498947999997, 2024-09-13T09:12:38.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011622512, 2024-09-13T09:12:38.846 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.846 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.846 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.846 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.846 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.846 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 88 2024-09-13T09:12:38.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.846 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:38.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:38.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312262+0000", 2024-09-13T09:12:38.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312265+0000", 2024-09-13T09:12:38.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8380000000000002e-06 2024-09-13T09:12:38.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312268+0000", 2024-09-13T09:12:38.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2310000000000001e-06 2024-09-13T09:12:38.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.312271+0000", 2024-09-13T09:12:38.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.551e-06 2024-09-13T09:12:38.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318458+0000", 2024-09-13T09:12:38.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061866710000000004 2024-09-13T09:12:38.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318540+0000", 2024-09-13T09:12:38.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1845e-05 2024-09-13T09:12:38.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.318668+0000", 2024-09-13T09:12:38.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012830900000000001 2024-09-13T09:12:38.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.319598+0000", 2024-09-13T09:12:38.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000929377 2024-09-13T09:12:38.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323840+0000", 2024-09-13T09:12:38.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0042420180000000002 2024-09-13T09:12:38.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323866+0000", 2024-09-13T09:12:38.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6262999999999999e-05 2024-09-13T09:12:38.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.323885+0000", 2024-09-13T09:12:38.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8675e-05 2024-09-13T09:12:38.851 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.851 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.851 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.851 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:38.851 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:38.852 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.299120954000003, 2024-09-13T09:12:38.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0062525000000000002, 2024-09-13T09:12:38.852 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.852 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.852 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113 2024-09-13T09:12:38.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.852 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:38.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:38.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472640+0000", 2024-09-13T09:12:38.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472644+0000", 2024-09-13T09:12:38.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-13T09:12:38.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472650+0000", 2024-09-13T09:12:38.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:12:38.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472655+0000", 2024-09-13T09:12:38.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:12:38.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477258+0000", 2024-09-13T09:12:38.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004603762 2024-09-13T09:12:38.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477303+0000", 2024-09-13T09:12:38.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5031999999999998e-05 2024-09-13T09:12:38.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477375+0000", 2024-09-13T09:12:38.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1186000000000006e-05 2024-09-13T09:12:38.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478772+0000", 2024-09-13T09:12:38.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00139704 2024-09-13T09:12:38.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478817+0000", 2024-09-13T09:12:38.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5120999999999997e-05 2024-09-13T09:12:38.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478859+0000", 2024-09-13T09:12:38.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2567000000000001e-05 2024-09-13T09:12:38.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478893+0000", 2024-09-13T09:12:38.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3210000000000002e-05 2024-09-13T09:12:38.857 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.857 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.857 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.857 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:38.857 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:38.857 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.299097646, 2024-09-13T09:12:38.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063268899999999999, 2024-09-13T09:12:38.858 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.858 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.858 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 114 2024-09-13T09:12:38.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.858 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:38.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:38.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472663+0000", 2024-09-13T09:12:38.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472667+0000", 2024-09-13T09:12:38.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9219999999999999e-06 2024-09-13T09:12:38.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472673+0000", 2024-09-13T09:12:38.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.63e-06 2024-09-13T09:12:38.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472678+0000", 2024-09-13T09:12:38.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6689999999999997e-06 2024-09-13T09:12:38.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477608+0000", 2024-09-13T09:12:38.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0049302870000000002 2024-09-13T09:12:38.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477685+0000", 2024-09-13T09:12:38.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6922000000000007e-05 2024-09-13T09:12:38.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.477804+0000", 2024-09-13T09:12:38.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011865499999999999 2024-09-13T09:12:38.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478935+0000", 2024-09-13T09:12:38.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0011305099999999999 2024-09-13T09:12:38.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478942+0000", 2024-09-13T09:12:38.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3969999999999997e-06 2024-09-13T09:12:38.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478970+0000", 2024-09-13T09:12:38.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8447000000000001e-05 2024-09-13T09:12:38.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478990+0000", 2024-09-13T09:12:38.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9933e-05 2024-09-13T09:12:38.863 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.863 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.863 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.863 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:38.863 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472870+0000", 2024-09-13T09:12:38.863 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.298891228999999, 2024-09-13T09:12:38.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068457680000000003, 2024-09-13T09:12:38.864 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.864 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.864 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.864 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.864 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.864 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 115 2024-09-13T09:12:38.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.864 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472870+0000", 2024-09-13T09:12:38.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472870+0000", 2024-09-13T09:12:38.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472869+0000", 2024-09-13T09:12:38.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:38.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472933+0000", 2024-09-13T09:12:38.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3448000000000006e-05 2024-09-13T09:12:38.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472943+0000", 2024-09-13T09:12:38.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0234999999999999e-05 2024-09-13T09:12:38.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472948+0000", 2024-09-13T09:12:38.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:12:38.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478311+0000", 2024-09-13T09:12:38.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053631759999999999 2024-09-13T09:12:38.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478364+0000", 2024-09-13T09:12:38.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3174000000000003e-05 2024-09-13T09:12:38.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.478441+0000", 2024-09-13T09:12:38.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6211000000000006e-05 2024-09-13T09:12:38.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479649+0000", 2024-09-13T09:12:38.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001208319 2024-09-13T09:12:38.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479673+0000", 2024-09-13T09:12:38.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4060999999999999e-05 2024-09-13T09:12:38.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479697+0000", 2024-09-13T09:12:38.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3958999999999999e-05 2024-09-13T09:12:38.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479716+0000", 2024-09-13T09:12:38.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8661e-05 2024-09-13T09:12:38.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.869 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.869 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:38.869 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472955+0000", 2024-09-13T09:12:38.869 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.298806173999999, 2024-09-13T09:12:38.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069091580000000003, 2024-09-13T09:12:38.869 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.869 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.870 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 116 2024-09-13T09:12:38.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.870 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472955+0000", 2024-09-13T09:12:38.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472955+0000", 2024-09-13T09:12:38.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472954+0000", 2024-09-13T09:12:38.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472959+0000", 2024-09-13T09:12:38.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1309999999999999e-06 2024-09-13T09:12:38.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472964+0000", 2024-09-13T09:12:38.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6659999999999996e-06 2024-09-13T09:12:38.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472968+0000", 2024-09-13T09:12:38.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.495e-06 2024-09-13T09:12:38.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479021+0000", 2024-09-13T09:12:38.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060531990000000004 2024-09-13T09:12:38.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479062+0000", 2024-09-13T09:12:38.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1436000000000001e-05 2024-09-13T09:12:38.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479131+0000", 2024-09-13T09:12:38.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8533000000000003e-05 2024-09-13T09:12:38.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479727+0000", 2024-09-13T09:12:38.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00059612 2024-09-13T09:12:38.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479827+0000", 2024-09-13T09:12:38.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9940999999999997e-05 2024-09-13T09:12:38.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479848+0000", 2024-09-13T09:12:38.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0877999999999999e-05 2024-09-13T09:12:38.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479864+0000", 2024-09-13T09:12:38.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6175999999999999e-05 2024-09-13T09:12:38.875 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.875 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.875 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.876 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:38.876 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.472991+0000", 2024-09-13T09:12:38.876 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.298770568000002, 2024-09-13T09:12:38.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071846899999999997, 2024-09-13T09:12:38.876 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.876 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.876 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 117 2024-09-13T09:12:38.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.876 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472991+0000", 2024-09-13T09:12:38.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472991+0000", 2024-09-13T09:12:38.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472990+0000", 2024-09-13T09:12:38.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:38.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.472994+0000", 2024-09-13T09:12:38.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4870000000000004e-06 2024-09-13T09:12:38.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.473001+0000", 2024-09-13T09:12:38.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:12:38.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.473005+0000", 2024-09-13T09:12:38.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2579999999999997e-06 2024-09-13T09:12:38.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479356+0000", 2024-09-13T09:12:38.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063509869999999998 2024-09-13T09:12:38.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479407+0000", 2024-09-13T09:12:38.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0349999999999997e-05 2024-09-13T09:12:38.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479488+0000", 2024-09-13T09:12:38.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0922999999999998e-05 2024-09-13T09:12:38.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.479942+0000", 2024-09-13T09:12:38.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00045411799999999999 2024-09-13T09:12:38.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.480136+0000", 2024-09-13T09:12:38.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000193909 2024-09-13T09:12:38.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.480158+0000", 2024-09-13T09:12:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2427999999999999e-05 2024-09-13T09:12:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.480175+0000", 2024-09-13T09:12:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7071e-05 2024-09-13T09:12:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.882 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:38.882 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:38.882 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.919422853999997, 2024-09-13T09:12:38.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066097600000000001, 2024-09-13T09:12:38.882 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.882 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.882 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 140 2024-09-13T09:12:38.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.882 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:38.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:38.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852338+0000", 2024-09-13T09:12:38.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:38.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852387+0000", 2024-09-13T09:12:38.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9191999999999999e-05 2024-09-13T09:12:38.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852392+0000", 2024-09-13T09:12:38.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7299999999999996e-06 2024-09-13T09:12:38.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852395+0000", 2024-09-13T09:12:38.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9979999999999999e-06 2024-09-13T09:12:38.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857443+0000", 2024-09-13T09:12:38.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005048075 2024-09-13T09:12:38.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857503+0000", 2024-09-13T09:12:38.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9577999999999998e-05 2024-09-13T09:12:38.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857571+0000", 2024-09-13T09:12:38.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8320999999999999e-05 2024-09-13T09:12:38.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858400+0000", 2024-09-13T09:12:38.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00082945100000000004 2024-09-13T09:12:38.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858910+0000", 2024-09-13T09:12:38.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050916999999999998 2024-09-13T09:12:38.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858931+0000", 2024-09-13T09:12:38.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1742e-05 2024-09-13T09:12:38.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858948+0000", 2024-09-13T09:12:38.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6714999999999998e-05 2024-09-13T09:12:38.887 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.887 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.887 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.887 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4192.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T09:12:38.887 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:00.852399+0000", 2024-09-13T09:12:38.887 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.919362542999998, 2024-09-13T09:12:38.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068470969999999999, 2024-09-13T09:12:38.888 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:38.888 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:38.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:38.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4192", 2024-09-13T09:12:38.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.129:0/1748302019", 2024-09-13T09:12:38.888 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 141 2024-09-13T09:12:38.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.888 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:38.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:38.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852399+0000", 2024-09-13T09:12:38.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:38.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852399+0000", 2024-09-13T09:12:38.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:38.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:38.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852398+0000", 2024-09-13T09:12:38.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:38.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:38.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852400+0000", 2024-09-13T09:12:38.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:12:38.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:38.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852404+0000", 2024-09-13T09:12:38.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:12:38.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:38.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.852406+0000", 2024-09-13T09:12:38.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4559999999999999e-06 2024-09-13T09:12:38.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:38.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857928+0000", 2024-09-13T09:12:38.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005521593 2024-09-13T09:12:38.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:38.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.857968+0000", 2024-09-13T09:12:38.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0586000000000001e-05 2024-09-13T09:12:38.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:38.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.858029+0000", 2024-09-13T09:12:38.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1140000000000001e-05 2024-09-13T09:12:38.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:38.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859187+0000", 2024-09-13T09:12:38.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001157969 2024-09-13T09:12:38.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:38.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859198+0000", 2024-09-13T09:12:38.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1028e-05 2024-09-13T09:12:38.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:38.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859227+0000", 2024-09-13T09:12:38.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8370000000000001e-05 2024-09-13T09:12:38.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:38.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:38.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:38.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:00.859246+0000", 2024-09-13T09:12:38.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8961e-05 2024-09-13T09:12:38.893 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.893 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.893 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.893 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:38.893 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:38.893 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:38.895 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:38.895 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:38.894+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:38.995 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:38.994+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:39.037 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:39.037 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:39.037 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:39.037 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:39.043 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-13T09:12:39.046 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:39.096 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:39.095+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:39.148 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:39.149 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:39.149 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:39.158 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:39.199 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:39.198+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:39.298 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:39.297+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:39.304 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:39.305 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:39.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:39.305 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:39.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.305 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T09:12:39.305 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:16.378424+0000", 2024-09-13T09:12:39.305 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.924253942, 2024-09-13T09:12:39.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061107690000000003, 2024-09-13T09:12:39.305 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.305 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:12:39.306 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:12:39.306 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9 2024-09-13T09:12:39.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.306 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378424+0000", 2024-09-13T09:12:39.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378424+0000", 2024-09-13T09:12:39.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378422+0000", 2024-09-13T09:12:39.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:39.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378429+0000", 2024-09-13T09:12:39.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6100000000000002e-06 2024-09-13T09:12:39.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378441+0000", 2024-09-13T09:12:39.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1581e-05 2024-09-13T09:12:39.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378449+0000", 2024-09-13T09:12:39.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2679999999999994e-06 2024-09-13T09:12:39.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378496+0000", 2024-09-13T09:12:39.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7286000000000001e-05 2024-09-13T09:12:39.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:12:39.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.378501+0000", 2024-09-13T09:12:39.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0980000000000001e-06 2024-09-13T09:12:39.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.380081+0000", 2024-09-13T09:12:39.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015801879999999999 2024-09-13T09:12:39.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.380177+0000", 2024-09-13T09:12:39.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5311999999999998e-05 2024-09-13T09:12:39.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:39.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.380299+0000", 2024-09-13T09:12:39.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000121921 2024-09-13T09:12:39.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:39.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.381082+0000", 2024-09-13T09:12:39.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00078348100000000004 2024-09-13T09:12:39.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:39.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384464+0000", 2024-09-13T09:12:39.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0033814349999999999 2024-09-13T09:12:39.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384495+0000", 2024-09-13T09:12:39.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1554999999999999e-05 2024-09-13T09:12:39.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:16.384534+0000", 2024-09-13T09:12:39.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9320999999999998e-05 2024-09-13T09:12:39.312 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.312 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.312 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.312 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4400.0:1499 3.11 3:8ebbb892:::rbd_data.11238d3f93be.0000000000000036:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:12:39.312 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:19.136984+0000", 2024-09-13T09:12:39.312 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.165694092999999, 2024-09-13T09:12:39.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0064184419999999999, 2024-09-13T09:12:39.312 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.312 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.312 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.312 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4400", 2024-09-13T09:12:39.312 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/1877038394", 2024-09-13T09:12:39.313 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1499 2024-09-13T09:12:39.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.313 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.136984+0000", 2024-09-13T09:12:39.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.136984+0000", 2024-09-13T09:12:39.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.136983+0000", 2024-09-13T09:12:39.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:39.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.136987+0000", 2024-09-13T09:12:39.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9659999999999998e-06 2024-09-13T09:12:39.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.137001+0000", 2024-09-13T09:12:39.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3832000000000001e-05 2024-09-13T09:12:39.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.137007+0000", 2024-09-13T09:12:39.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8549999999999998e-06 2024-09-13T09:12:39.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.137060+0000", 2024-09-13T09:12:39.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3171999999999998e-05 2024-09-13T09:12:39.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.137141+0000", 2024-09-13T09:12:39.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1116000000000001e-05 2024-09-13T09:12:39.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:39.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.137246+0000", 2024-09-13T09:12:39.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010518099999999999 2024-09-13T09:12:39.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:39.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.138787+0000", 2024-09-13T09:12:39.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015407789999999999 2024-09-13T09:12:39.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:39.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.143355+0000", 2024-09-13T09:12:39.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0045682359999999998 2024-09-13T09:12:39.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.143381+0000", 2024-09-13T09:12:39.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5585999999999999e-05 2024-09-13T09:12:39.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:19.143402+0000", 2024-09-13T09:12:39.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1154000000000001e-05 2024-09-13T09:12:39.317 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.317 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.317 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.318 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:1996 3.11 3:8e1724e1:::rbd_data.1123b8b4a28a.0000000000000072:head [set-alloc-hint object_size 2097152 write_size 2097152,write 532480~24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:39.318 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.507169+0000", 2024-09-13T09:12:39.318 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.795509058, 2024-09-13T09:12:39.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0081220519999999994, 2024-09-13T09:12:39.318 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.318 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.318 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.318 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:39.318 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:39.318 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1996 2024-09-13T09:12:39.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.318 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507169+0000", 2024-09-13T09:12:39.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507169+0000", 2024-09-13T09:12:39.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507168+0000", 2024-09-13T09:12:39.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:39.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507193+0000", 2024-09-13T09:12:39.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5505999999999999e-05 2024-09-13T09:12:39.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507204+0000", 2024-09-13T09:12:39.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1209999999999999e-05 2024-09-13T09:12:39.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507212+0000", 2024-09-13T09:12:39.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:12:39.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507472+0000", 2024-09-13T09:12:39.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025948799999999999 2024-09-13T09:12:39.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507516+0000", 2024-09-13T09:12:39.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4416999999999998e-05 2024-09-13T09:12:39.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:39.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.507598+0000", 2024-09-13T09:12:39.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2081000000000002e-05 2024-09-13T09:12:39.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:39.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.508167+0000", 2024-09-13T09:12:39.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056849899999999998 2024-09-13T09:12:39.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:39.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.515246+0000", 2024-09-13T09:12:39.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0070798809999999997 2024-09-13T09:12:39.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.515272+0000", 2024-09-13T09:12:39.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.529e-05 2024-09-13T09:12:39.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.515291+0000", 2024-09-13T09:12:39.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8887000000000001e-05 2024-09-13T09:12:39.323 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.323 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.323 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.324 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:2110 3.3 3:c083ceea:::rbd_data.1123b8b4a28a.000000000000005d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:39.324 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.548044+0000", 2024-09-13T09:12:39.324 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.754633662, 2024-09-13T09:12:39.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005108502, 2024-09-13T09:12:39.324 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.324 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:39.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:39.324 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2110 2024-09-13T09:12:39.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.324 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548044+0000", 2024-09-13T09:12:39.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548044+0000", 2024-09-13T09:12:39.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548043+0000", 2024-09-13T09:12:39.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:39.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548048+0000", 2024-09-13T09:12:39.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4580000000000003e-06 2024-09-13T09:12:39.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548058+0000", 2024-09-13T09:12:39.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0494e-05 2024-09-13T09:12:39.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548063+0000", 2024-09-13T09:12:39.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:12:39.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548126+0000", 2024-09-13T09:12:39.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2761000000000004e-05 2024-09-13T09:12:39.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548201+0000", 2024-09-13T09:12:39.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5258000000000003e-05 2024-09-13T09:12:39.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:39.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.548297+0000", 2024-09-13T09:12:39.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6409000000000005e-05 2024-09-13T09:12:39.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:39.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.549538+0000", 2024-09-13T09:12:39.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0012407830000000001 2024-09-13T09:12:39.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:39.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.553082+0000", 2024-09-13T09:12:39.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003544045 2024-09-13T09:12:39.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.553130+0000", 2024-09-13T09:12:39.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7664000000000003e-05 2024-09-13T09:12:39.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.553152+0000", 2024-09-13T09:12:39.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2546999999999999e-05 2024-09-13T09:12:39.330 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.330 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.330 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.330 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:2311 3.9 3:93ffaa8c:::rbd_data.1123b8b4a28a.0000000000000041:head [set-alloc-hint object_size 2097152 write_size 2097152,write 684032~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:39.330 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.619294+0000", 2024-09-13T09:12:39.330 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.683383342999999, 2024-09-13T09:12:39.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005528252, 2024-09-13T09:12:39.330 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.330 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.330 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:39.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:39.331 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2311 2024-09-13T09:12:39.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.331 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619294+0000", 2024-09-13T09:12:39.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619294+0000", 2024-09-13T09:12:39.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619293+0000", 2024-09-13T09:12:39.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:39.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619301+0000", 2024-09-13T09:12:39.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5780000000000001e-06 2024-09-13T09:12:39.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619309+0000", 2024-09-13T09:12:39.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9869999999999999e-06 2024-09-13T09:12:39.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619313+0000", 2024-09-13T09:12:39.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.386e-06 2024-09-13T09:12:39.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619346+0000", 2024-09-13T09:12:39.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2567000000000002e-05 2024-09-13T09:12:39.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619412+0000", 2024-09-13T09:12:39.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5654000000000005e-05 2024-09-13T09:12:39.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:39.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.619544+0000", 2024-09-13T09:12:39.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013272699999999999 2024-09-13T09:12:39.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:39.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.620054+0000", 2024-09-13T09:12:39.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00050981500000000005 2024-09-13T09:12:39.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:39.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.624764+0000", 2024-09-13T09:12:39.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0047095640000000003 2024-09-13T09:12:39.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.624802+0000", 2024-09-13T09:12:39.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7805999999999997e-05 2024-09-13T09:12:39.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.624823+0000", 2024-09-13T09:12:39.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0999e-05 2024-09-13T09:12:39.336 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.336 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.336 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.336 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:2454 3.9 3:96917efd:::rbd_data.1123b8b4a28a.0000000000000066:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:39.336 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.676741+0000", 2024-09-13T09:12:39.336 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.625936512999999, 2024-09-13T09:12:39.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053986470000000003, 2024-09-13T09:12:39.336 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.336 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.336 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.336 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:39.337 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:39.337 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2454 2024-09-13T09:12:39.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.337 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676741+0000", 2024-09-13T09:12:39.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676741+0000", 2024-09-13T09:12:39.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676741+0000", 2024-09-13T09:12:39.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:39.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676744+0000", 2024-09-13T09:12:39.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:12:39.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676751+0000", 2024-09-13T09:12:39.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5140000000000004e-06 2024-09-13T09:12:39.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676755+0000", 2024-09-13T09:12:39.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9990000000000002e-06 2024-09-13T09:12:39.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676788+0000", 2024-09-13T09:12:39.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3181e-05 2024-09-13T09:12:39.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676861+0000", 2024-09-13T09:12:39.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2924999999999997e-05 2024-09-13T09:12:39.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:12:39.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.676966+0000", 2024-09-13T09:12:39.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000104535 2024-09-13T09:12:39.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:39.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.677384+0000", 2024-09-13T09:12:39.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00041842600000000001 2024-09-13T09:12:39.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:39.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.681986+0000", 2024-09-13T09:12:39.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0046015250000000004 2024-09-13T09:12:39.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.682054+0000", 2024-09-13T09:12:39.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8235e-05 2024-09-13T09:12:39.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.682140+0000", 2024-09-13T09:12:39.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5840999999999993e-05 2024-09-13T09:12:39.342 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.342 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.342 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.342 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4567.0:2487 3.3 3:c560f27e:::rbd_data.1123b8b4a28a.0000000000000076:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T09:12:39.342 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:25.680361+0000", 2024-09-13T09:12:39.342 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.622316323, 2024-09-13T09:12:39.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057142149999999999, 2024-09-13T09:12:39.342 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.342 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.342 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.342 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4567", 2024-09-13T09:12:39.343 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2061107042", 2024-09-13T09:12:39.343 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2487 2024-09-13T09:12:39.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.343 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680361+0000", 2024-09-13T09:12:39.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680361+0000", 2024-09-13T09:12:39.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680361+0000", 2024-09-13T09:12:39.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:39.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680365+0000", 2024-09-13T09:12:39.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6260000000000003e-06 2024-09-13T09:12:39.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680373+0000", 2024-09-13T09:12:39.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2029999999999992e-06 2024-09-13T09:12:39.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680378+0000", 2024-09-13T09:12:39.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2710000000000003e-06 2024-09-13T09:12:39.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680419+0000", 2024-09-13T09:12:39.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1267000000000003e-05 2024-09-13T09:12:39.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680478+0000", 2024-09-13T09:12:39.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8955000000000001e-05 2024-09-13T09:12:39.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:39.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.680537+0000", 2024-09-13T09:12:39.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9001e-05 2024-09-13T09:12:39.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:39.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.684320+0000", 2024-09-13T09:12:39.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0037828860000000001 2024-09-13T09:12:39.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:39.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.686008+0000", 2024-09-13T09:12:39.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0016885660000000001 2024-09-13T09:12:39.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.686045+0000", 2024-09-13T09:12:39.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6279000000000003e-05 2024-09-13T09:12:39.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:25.686076+0000", 2024-09-13T09:12:39.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0945000000000003e-05 2024-09-13T09:12:39.347 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.347 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.347 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.348 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4816.0:667 3.11 3:8f2ca167:::rbd_data.1123b259dc65.000000000000002b:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1101824~12288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-13T09:12:39.348 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:34.216468+0000", 2024-09-13T09:12:39.348 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0862098429999998, 2024-09-13T09:12:39.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059525569999999998, 2024-09-13T09:12:39.348 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.348 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.348 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.348 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4816", 2024-09-13T09:12:39.348 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2210529289", 2024-09-13T09:12:39.348 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 667 2024-09-13T09:12:39.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.348 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.216468+0000", 2024-09-13T09:12:39.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.216468+0000", 2024-09-13T09:12:39.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.216467+0000", 2024-09-13T09:12:39.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:39.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.216483+0000", 2024-09-13T09:12:39.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6161999999999999e-05 2024-09-13T09:12:39.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.216497+0000", 2024-09-13T09:12:39.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4082e-05 2024-09-13T09:12:39.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.216505+0000", 2024-09-13T09:12:39.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0749999999999998e-06 2024-09-13T09:12:39.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.216552+0000", 2024-09-13T09:12:39.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6634000000000001e-05 2024-09-13T09:12:39.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.216672+0000", 2024-09-13T09:12:39.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120711 2024-09-13T09:12:39.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:12:39.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.216807+0000", 2024-09-13T09:12:39.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013502300000000001 2024-09-13T09:12:39.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:39.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218685+0000", 2024-09-13T09:12:39.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0018771809999999999 2024-09-13T09:12:39.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:39.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.222375+0000", 2024-09-13T09:12:39.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036899160000000001 2024-09-13T09:12:39.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.222400+0000", 2024-09-13T09:12:39.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5065000000000001e-05 2024-09-13T09:12:39.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.222420+0000", 2024-09-13T09:12:39.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0741e-05 2024-09-13T09:12:39.353 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.353 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.354 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4816.0:707 3.11 3:8d5741c9:::rbd_data.1123df4df335.0000000000000001:d [read 442368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T09:12:39.354 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:34.218084+0000", 2024-09-13T09:12:39.354 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0845937689999996, 2024-09-13T09:12:39.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053970479999999998, 2024-09-13T09:12:39.354 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.354 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:39.354 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.354 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4816", 2024-09-13T09:12:39.354 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2210529289", 2024-09-13T09:12:39.354 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 707 2024-09-13T09:12:39.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.355 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218084+0000", 2024-09-13T09:12:39.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218084+0000", 2024-09-13T09:12:39.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218083+0000", 2024-09-13T09:12:39.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:39.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218086+0000", 2024-09-13T09:12:39.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:12:39.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218092+0000", 2024-09-13T09:12:39.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-13T09:12:39.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218097+0000", 2024-09-13T09:12:39.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2680000000000002e-06 2024-09-13T09:12:39.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.223149+0000", 2024-09-13T09:12:39.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0050518999999999998 2024-09-13T09:12:39.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.223231+0000", 2024-09-13T09:12:39.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2460999999999995e-05 2024-09-13T09:12:39.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.223481+0000", 2024-09-13T09:12:39.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024974900000000001 2024-09-13T09:12:39.358 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.358 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.358 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.358 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4816.0:722 3.14 3:2d979bc0:::rbd_data.1123df4df335.0000000000000009:d [read 278528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T09:12:39.358 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:34.218165+0000", 2024-09-13T09:12:39.358 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0845128900000001, 2024-09-13T09:12:39.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0052967700000000001, 2024-09-13T09:12:39.359 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.359 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:39.359 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.359 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4816", 2024-09-13T09:12:39.359 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2210529289", 2024-09-13T09:12:39.359 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 722 2024-09-13T09:12:39.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.359 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218165+0000", 2024-09-13T09:12:39.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218165+0000", 2024-09-13T09:12:39.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218164+0000", 2024-09-13T09:12:39.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:39.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218167+0000", 2024-09-13T09:12:39.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.424e-06 2024-09-13T09:12:39.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218173+0000", 2024-09-13T09:12:39.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9560000000000002e-06 2024-09-13T09:12:39.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218177+0000", 2024-09-13T09:12:39.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:12:39.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.223083+0000", 2024-09-13T09:12:39.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0049059680000000001 2024-09-13T09:12:39.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.223127+0000", 2024-09-13T09:12:39.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3849e-05 2024-09-13T09:12:39.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.223462+0000", 2024-09-13T09:12:39.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00033431999999999998 2024-09-13T09:12:39.363 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.363 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.363 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.363 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4816.0:723 3.14 3:2d979bc0:::rbd_data.1123df4df335.0000000000000009:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T09:12:39.363 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:34.218182+0000", 2024-09-13T09:12:39.363 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.084495649, 2024-09-13T09:12:39.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0056493810000000002, 2024-09-13T09:12:39.363 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.363 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:39.363 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.363 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4816", 2024-09-13T09:12:39.363 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2210529289", 2024-09-13T09:12:39.364 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 723 2024-09-13T09:12:39.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.364 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218182+0000", 2024-09-13T09:12:39.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218182+0000", 2024-09-13T09:12:39.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218182+0000", 2024-09-13T09:12:39.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:39.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218184+0000", 2024-09-13T09:12:39.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0329999999999998e-06 2024-09-13T09:12:39.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218189+0000", 2024-09-13T09:12:39.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4609999999999999e-06 2024-09-13T09:12:39.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218193+0000", 2024-09-13T09:12:39.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2069999999999997e-06 2024-09-13T09:12:39.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.223488+0000", 2024-09-13T09:12:39.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0052948439999999999 2024-09-13T09:12:39.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.223557+0000", 2024-09-13T09:12:39.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8857999999999997e-05 2024-09-13T09:12:39.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.223831+0000", 2024-09-13T09:12:39.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00027432900000000002 2024-09-13T09:12:39.367 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.367 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.367 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.367 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4816.0:724 3.14 3:2d979bc0:::rbd_data.1123df4df335.0000000000000009:d [read 475136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T09:12:39.367 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:34.218197+0000", 2024-09-13T09:12:39.368 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.0844802920000003, 2024-09-13T09:12:39.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059554569999999999, 2024-09-13T09:12:39.368 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.368 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:39.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4816", 2024-09-13T09:12:39.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2210529289", 2024-09-13T09:12:39.368 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 724 2024-09-13T09:12:39.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.368 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218197+0000", 2024-09-13T09:12:39.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218197+0000", 2024-09-13T09:12:39.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218197+0000", 2024-09-13T09:12:39.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:39.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218199+0000", 2024-09-13T09:12:39.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.12e-06 2024-09-13T09:12:39.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218206+0000", 2024-09-13T09:12:39.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4200000000000004e-06 2024-09-13T09:12:39.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.218210+0000", 2024-09-13T09:12:39.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5619999999999997e-06 2024-09-13T09:12:39.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.223848+0000", 2024-09-13T09:12:39.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005638044 2024-09-13T09:12:39.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.223890+0000", 2024-09-13T09:12:39.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2072e-05 2024-09-13T09:12:39.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.224153+0000", 2024-09-13T09:12:39.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00026259900000000002 2024-09-13T09:12:39.372 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.372 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.372 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.373 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4816.0:1505 3.14 3:2a872540:::rbd_data.1123b259dc65.0000000000000032:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1601536~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-13T09:12:39.373 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:34.445688+0000", 2024-09-13T09:12:39.373 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8569897170000003, 2024-09-13T09:12:39.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0062448410000000001, 2024-09-13T09:12:39.373 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.373 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4816", 2024-09-13T09:12:39.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2210529289", 2024-09-13T09:12:39.373 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1505 2024-09-13T09:12:39.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.373 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.445688+0000", 2024-09-13T09:12:39.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.445688+0000", 2024-09-13T09:12:39.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.445687+0000", 2024-09-13T09:12:39.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:39.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.445694+0000", 2024-09-13T09:12:39.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9210000000000003e-06 2024-09-13T09:12:39.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.445703+0000", 2024-09-13T09:12:39.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5769999999999994e-06 2024-09-13T09:12:39.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.445707+0000", 2024-09-13T09:12:39.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.746e-06 2024-09-13T09:12:39.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.445737+0000", 2024-09-13T09:12:39.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9595e-05 2024-09-13T09:12:39.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.445784+0000", 2024-09-13T09:12:39.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6792999999999997e-05 2024-09-13T09:12:39.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:39.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.445867+0000", 2024-09-13T09:12:39.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3331999999999994e-05 2024-09-13T09:12:39.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:39.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.451738+0000", 2024-09-13T09:12:39.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0058708639999999999 2024-09-13T09:12:39.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:39.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.451851+0000", 2024-09-13T09:12:39.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112681 2024-09-13T09:12:39.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.451897+0000", 2024-09-13T09:12:39.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6204e-05 2024-09-13T09:12:39.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.451933+0000", 2024-09-13T09:12:39.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5831000000000003e-05 2024-09-13T09:12:39.378 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.378 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.378 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.379 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4816.0:1508 3.14 3:297d14a0:::rbd_data.1123b259dc65.0000000000000035:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1589248~13824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-13T09:12:39.379 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:34.449477+0000", 2024-09-13T09:12:39.379 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8532002690000002, 2024-09-13T09:12:39.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0050755119999999999, 2024-09-13T09:12:39.379 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.379 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.379 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.379 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4816", 2024-09-13T09:12:39.379 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2210529289", 2024-09-13T09:12:39.379 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1508 2024-09-13T09:12:39.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.379 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.449477+0000", 2024-09-13T09:12:39.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.449477+0000", 2024-09-13T09:12:39.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.449476+0000", 2024-09-13T09:12:39.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:39.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.449490+0000", 2024-09-13T09:12:39.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3101999999999999e-05 2024-09-13T09:12:39.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.449502+0000", 2024-09-13T09:12:39.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2466000000000001e-05 2024-09-13T09:12:39.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.449510+0000", 2024-09-13T09:12:39.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8129999999999996e-06 2024-09-13T09:12:39.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.449553+0000", 2024-09-13T09:12:39.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2818999999999998e-05 2024-09-13T09:12:39.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.449627+0000", 2024-09-13T09:12:39.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3992000000000006e-05 2024-09-13T09:12:39.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:39.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.449755+0000", 2024-09-13T09:12:39.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000128291 2024-09-13T09:12:39.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:39.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.452022+0000", 2024-09-13T09:12:39.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002266896 2024-09-13T09:12:39.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:39.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.454487+0000", 2024-09-13T09:12:39.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.002464787 2024-09-13T09:12:39.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.454524+0000", 2024-09-13T09:12:39.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7256999999999999e-05 2024-09-13T09:12:39.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.454553+0000", 2024-09-13T09:12:39.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9034999999999999e-05 2024-09-13T09:12:39.384 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.384 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.384 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.385 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4816.0:1614 3.1d e41/21 3:b811d8ba:::rbd_object_map.1123b259dc65:head v 41'173, mlcod=40'168)", 2024-09-13T09:12:39.385 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:34.507631+0000", 2024-09-13T09:12:39.385 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7950462629999997, 2024-09-13T09:12:39.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0053774519999999996, 2024-09-13T09:12:39.385 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.385 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.385 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.507631+0000", 2024-09-13T09:12:39.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.507631+0000", 2024-09-13T09:12:39.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.507631+0000", 2024-09-13T09:12:39.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:39.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.507635+0000", 2024-09-13T09:12:39.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9689999999999996e-06 2024-09-13T09:12:39.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.507643+0000", 2024-09-13T09:12:39.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9629999999999998e-06 2024-09-13T09:12:39.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.507648+0000", 2024-09-13T09:12:39.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5679999999999999e-06 2024-09-13T09:12:39.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.507680+0000", 2024-09-13T09:12:39.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1198000000000002e-05 2024-09-13T09:12:39.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.507692+0000", 2024-09-13T09:12:39.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2033e-05 2024-09-13T09:12:39.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.512974+0000", 2024-09-13T09:12:39.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005282368 2024-09-13T09:12:39.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.513009+0000", 2024-09-13T09:12:39.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4820999999999997e-05 2024-09-13T09:12:39.389 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.389 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.389 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.389 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4816.0:1620 3.1d e41/21 3:b811d8ba:::rbd_object_map.1123b259dc65:head v 41'178, mlcod=40'169)", 2024-09-13T09:12:39.389 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:34.510474+0000", 2024-09-13T09:12:39.389 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7922032889999997, 2024-09-13T09:12:39.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0051666079999999996, 2024-09-13T09:12:39.389 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.389 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.390 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.510474+0000", 2024-09-13T09:12:39.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.510474+0000", 2024-09-13T09:12:39.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.510474+0000", 2024-09-13T09:12:39.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:39.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.510479+0000", 2024-09-13T09:12:39.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:12:39.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.510491+0000", 2024-09-13T09:12:39.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1387e-05 2024-09-13T09:12:39.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.510497+0000", 2024-09-13T09:12:39.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:12:39.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.510537+0000", 2024-09-13T09:12:39.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9926999999999999e-05 2024-09-13T09:12:39.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.510555+0000", 2024-09-13T09:12:39.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7465e-05 2024-09-13T09:12:39.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.515617+0000", 2024-09-13T09:12:39.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0050623059999999999 2024-09-13T09:12:39.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.515641+0000", 2024-09-13T09:12:39.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4015999999999998e-05 2024-09-13T09:12:39.393 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.393 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.393 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.394 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4816.0:1667 3.14 3:2a872540:::rbd_data.1123b259dc65.0000000000000032:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3260416~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T09:12:39.394 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:34.511497+0000", 2024-09-13T09:12:39.394 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7911806700000001, 2024-09-13T09:12:39.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0055110749999999998, 2024-09-13T09:12:39.394 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.394 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.394 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.394 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4816", 2024-09-13T09:12:39.394 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2210529289", 2024-09-13T09:12:39.394 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1667 2024-09-13T09:12:39.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.394 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.511497+0000", 2024-09-13T09:12:39.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.511497+0000", 2024-09-13T09:12:39.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.511496+0000", 2024-09-13T09:12:39.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:39.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.511510+0000", 2024-09-13T09:12:39.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3757999999999999e-05 2024-09-13T09:12:39.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.511519+0000", 2024-09-13T09:12:39.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8040000000000007e-06 2024-09-13T09:12:39.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.511525+0000", 2024-09-13T09:12:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6089999999999997e-06 2024-09-13T09:12:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.511559+0000", 2024-09-13T09:12:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4119999999999999e-05 2024-09-13T09:12:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.511606+0000", 2024-09-13T09:12:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6678999999999998e-05 2024-09-13T09:12:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.511691+0000", 2024-09-13T09:12:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5168999999999997e-05 2024-09-13T09:12:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.515712+0000", 2024-09-13T09:12:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0040213419999999998 2024-09-13T09:12:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.516922+0000", 2024-09-13T09:12:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001209335 2024-09-13T09:12:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.516970+0000", 2024-09-13T09:12:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8060999999999999e-05 2024-09-13T09:12:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.517008+0000", 2024-09-13T09:12:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8186999999999999e-05 2024-09-13T09:12:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4816.0:1668 3.14 3:2a872540:::rbd_data.1123b259dc65.0000000000000032:head [set-alloc-hint object_size 4194304 write_size 4194304,write 4096000~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T09:12:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:34.511534+0000", 2024-09-13T09:12:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7911432019999998, 2024-09-13T09:12:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007990601, 2024-09-13T09:12:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4816", 2024-09-13T09:12:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/2210529289", 2024-09-13T09:12:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1668 2024-09-13T09:12:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.511534+0000", 2024-09-13T09:12:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.511534+0000", 2024-09-13T09:12:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.511534+0000", 2024-09-13T09:12:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.511551+0000", 2024-09-13T09:12:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6844999999999999e-05 2024-09-13T09:12:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.511558+0000", 2024-09-13T09:12:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4030000000000001e-06 2024-09-13T09:12:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.511564+0000", 2024-09-13T09:12:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.699e-06 2024-09-13T09:12:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.512959+0000", 2024-09-13T09:12:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001395149 2024-09-13T09:12:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.513044+0000", 2024-09-13T09:12:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4608e-05 2024-09-13T09:12:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 5", 2024-09-13T09:12:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.513174+0000", 2024-09-13T09:12:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013053199999999999 2024-09-13T09:12:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:12:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.518116+0000", 2024-09-13T09:12:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0049416429999999999 2024-09-13T09:12:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.519425+0000", 2024-09-13T09:12:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0013088419999999999 2024-09-13T09:12:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.519475+0000", 2024-09-13T09:12:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9932999999999998e-05 2024-09-13T09:12:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.519525+0000", 2024-09-13T09:12:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0451000000000002e-05 2024-09-13T09:12:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4816.0:2315 3.1b e41/21 3:d9bbe908:::rbd_data.1123b259dc65.000000000000001f:head v 41'120, mlcod=41'118)", 2024-09-13T09:12:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:34.685186+0000", 2024-09-13T09:12:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6174915920000004, 2024-09-13T09:12:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057074099999999996, 2024-09-13T09:12:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685186+0000", 2024-09-13T09:12:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685186+0000", 2024-09-13T09:12:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685185+0000", 2024-09-13T09:12:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685200+0000", 2024-09-13T09:12:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4657e-05 2024-09-13T09:12:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685210+0000", 2024-09-13T09:12:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0104e-05 2024-09-13T09:12:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685217+0000", 2024-09-13T09:12:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4160000000000001e-06 2024-09-13T09:12:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685280+0000", 2024-09-13T09:12:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3211000000000002e-05 2024-09-13T09:12:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685318+0000", 2024-09-13T09:12:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8189000000000003e-05 2024-09-13T09:12:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.690841+0000", 2024-09-13T09:12:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005523366 2024-09-13T09:12:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.690893+0000", 2024-09-13T09:12:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2126000000000002e-05 2024-09-13T09:12:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4816.0:2307 3.1d e41/21 3:b811d8ba:::rbd_object_map.1123b259dc65:head v 41'221, mlcod=41'217)", 2024-09-13T09:12:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:34.685200+0000", 2024-09-13T09:12:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6174780039999996, 2024-09-13T09:12:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0056541930000000001, 2024-09-13T09:12:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685200+0000", 2024-09-13T09:12:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:39.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685200+0000", 2024-09-13T09:12:39.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:39.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:39.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685199+0000", 2024-09-13T09:12:39.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:39.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:39.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685203+0000", 2024-09-13T09:12:39.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:12:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685212+0000", 2024-09-13T09:12:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0219999999999993e-06 2024-09-13T09:12:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685218+0000", 2024-09-13T09:12:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5169999999999999e-06 2024-09-13T09:12:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685281+0000", 2024-09-13T09:12:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2884000000000003e-05 2024-09-13T09:12:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.685298+0000", 2024-09-13T09:12:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6908000000000001e-05 2024-09-13T09:12:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:12:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.690818+0000", 2024-09-13T09:12:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0055200340000000001 2024-09-13T09:12:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:34.690854+0000", 2024-09-13T09:12:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6069000000000003e-05 2024-09-13T09:12:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:39.414 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:39.414 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:12:39.416 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:39.417 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:39.415+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:39.516 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:39.514+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:39.522 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:12:39.573 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:39.573 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:39.573 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:39.573 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:39.579 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:39.579 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:39.582 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:39.604 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:39.605 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:39.625 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:39.625+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:39.629 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:39.629 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:39.648 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:39.649 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:39.669 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:39.669 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:39.685 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:39.685 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:39.694 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:39.694 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:39.703 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:39.717 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:39.715+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:39.782 INFO:teuthology.orchestra.run.smithi157.stdout:1972 punch from 0x51a3600 to 0x51b2e00, (0xf800 bytes) 2024-09-13T09:12:39.792 INFO:teuthology.orchestra.run.smithi157.stdout:1976 read 0x3365c00 thru 0x336cd98 (0x7199 bytes) 2024-09-13T09:12:39.793 INFO:teuthology.orchestra.run.smithi157.stdout:1978 read 0x8296e00 thru 0x82a488c (0xda8d bytes) 2024-09-13T09:12:39.808 INFO:teuthology.orchestra.run.smithi157.stdout:1980 write 0xe52b000 thru 0xe531e14 (0x6e15 bytes) 2024-09-13T09:12:39.809 INFO:teuthology.orchestra.run.smithi157.stdout:1982 punch from 0x7134e00 to 0x7135e00, (0x1000 bytes) 2024-09-13T09:12:39.818 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:39.816+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:39.819 INFO:teuthology.orchestra.run.smithi157.stdout:1986 trunc from 0xe532000 to 0x6845600 2024-09-13T09:12:39.851 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:39.851 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:39.859 INFO:teuthology.orchestra.run.smithi157.stdout:1987 write 0x5cbe200 thru 0x5cc6907 (0x8708 bytes) 2024-09-13T09:12:39.860 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:39.877 INFO:teuthology.orchestra.run.smithi157.stdout:1988 write 0x83cf400 thru 0x83d08fa (0x14fb bytes) 2024-09-13T09:12:39.892 INFO:teuthology.orchestra.run.smithi157.stdout:1989 write 0xbf82800 thru 0xbf8641d (0x3c1e bytes) 2024-09-13T09:12:39.893 INFO:teuthology.orchestra.run.smithi157.stdout:1990 write 0xa7d9400 thru 0xa7e1cde (0x88df bytes) 2024-09-13T09:12:39.894 INFO:teuthology.orchestra.run.smithi157.stdout:1991 write 0x69d3400 thru 0x69d7ce3 (0x48e4 bytes) 2024-09-13T09:12:39.895 INFO:teuthology.orchestra.run.smithi157.stdout:1992 read 0xcef600 thru 0xcf4f82 (0x5983 bytes) 2024-09-13T09:12:39.895 INFO:teuthology.orchestra.run.smithi157.stdout:1994 read 0xb659a00 thru 0xb65e0f3 (0x46f4 bytes) 2024-09-13T09:12:39.895 INFO:teuthology.orchestra.run.smithi157.stdout:1997 read 0x4dd1000 thru 0x4dd80a2 (0x70a3 bytes) 2024-09-13T09:12:39.896 INFO:teuthology.orchestra.run.smithi157.stdout:2000 read 0xb20a00 thru 0xb302f8 (0xf8f9 bytes) 2024-09-13T09:12:39.896 INFO:teuthology.orchestra.run.smithi157.stdout:2001 clone 13 order 18 su 2048 sc 11 2024-09-13T09:12:39.920 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:39.919+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:39.971 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:39.971 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:39.980 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:40.023 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:40.021+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:40.086 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:40.086 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:40.094 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:40.121 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:40.120+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:40.221 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:40.220+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:40.240 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:40.240 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:40.249 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:40.333 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:40.332+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:40.396 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:40.396 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:40.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:40.446 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:40.445+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:40.510 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-09-13T09:12:40.524 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:40.523+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:40.556 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:40.556 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:40.566 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:40.590 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone12 from 0xbf86600 (overlap 0x6845600) to 0x0 2024-09-13T09:12:40.624 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:40.624+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:40.720 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:40.721 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:40.725 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:40.724+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:40.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:40.826 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:40.825+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:40.880 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:40.880 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:40.889 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:40.931 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:40.930+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:40.971 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-13T09:12:41.028 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:41.028+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:41.045 INFO:teuthology.orchestra.run.smithi157.stdout:2002 trunc from 0xbf86600 to 0x6de4400 2024-09-13T09:12:41.056 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:41.056 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:41.067 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:41.129 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:41.128+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:41.134 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-13T09:12:41.135 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:12:41.135 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-13T09:12:41.135 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-13T09:12:41.138 INFO:teuthology.orchestra.run.smithi157.stdout:2003 write 0x6282800 thru 0x6290a70 (0xe271 bytes) 2024-09-13T09:12:41.140 INFO:teuthology.orchestra.run.smithi157.stdout:2004 write 0xda7c00 thru 0xdaab57 (0x2f58 bytes) 2024-09-13T09:12:41.141 INFO:teuthology.orchestra.run.smithi157.stdout:2005 read 0x63bac00 thru 0x63bc285 (0x1686 bytes) 2024-09-13T09:12:41.142 INFO:teuthology.orchestra.run.smithi157.stdout:2006 read 0x4a57600 thru 0x4a5e349 (0x6d4a bytes) 2024-09-13T09:12:41.145 INFO:teuthology.orchestra.run.smithi157.stdout:2007 write 0x1ad8200 thru 0x1ae2151 (0x9f52 bytes) 2024-09-13T09:12:41.146 INFO:teuthology.orchestra.run.smithi157.stdout:skipping zero length punch hole 2024-09-13T09:12:41.217 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:41.217 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:41.229 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:41.229+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:41.231 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:41.331 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:41.330+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:41.341 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:41.341 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:41.351 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:41.395 INFO:teuthology.orchestra.run.smithi157.stdout:2009 write 0x75c5e00 thru 0x75d5df1 (0xfff2 bytes) 2024-09-13T09:12:41.405 INFO:teuthology.orchestra.run.smithi157.stdout:2010 trunc from 0x75d5e00 to 0xb1e4a00 2024-09-13T09:12:41.435 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:41.435+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:41.439 INFO:teuthology.orchestra.run.smithi157.stdout:2011 trunc from 0xb1e4a00 to 0x750d600 2024-09-13T09:12:41.469 INFO:teuthology.orchestra.run.smithi157.stdout:2012 read 0x449ec00 thru 0x44a2bb7 (0x3fb8 bytes) 2024-09-13T09:12:41.472 INFO:teuthology.orchestra.run.smithi157.stdout:2015 write 0x3d19600 thru 0x3d200b6 (0x6ab7 bytes) 2024-09-13T09:12:41.504 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:41.504 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:41.513 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:41.533 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:41.532+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:41.602 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-09-13T09:12:41.601+0000 7f251c3a0640 -1 mon.a@0(leader).osd e45 definitely_dead 0 2024-09-13T09:12:41.632 INFO:teuthology.orchestra.run.smithi157.stdout:2016 write 0xe796e00 thru 0xe7987b4 (0x19b5 bytes) 2024-09-13T09:12:41.632 INFO:teuthology.orchestra.run.smithi157.stdout:2018 read 0xcf8b400 thru 0xcf96abd (0xb6be bytes) 2024-09-13T09:12:41.632 INFO:teuthology.orchestra.run.smithi157.stdout:2022 punch from 0x365da00 to 0x3663c00, (0x6200 bytes) 2024-09-13T09:12:41.634 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:41.633+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:41.659 INFO:teuthology.orchestra.run.smithi157.stdout:2023 trunc from 0xe798800 to 0xac08400 2024-09-13T09:12:41.674 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:41.674 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:41.686 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:41.697 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:12:41.735 INFO:teuthology.orchestra.run.smithi157.stdout:2025 read 0x2475200 thru 0x24821d4 (0xcfd5 bytes) 2024-09-13T09:12:41.741 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:41.740+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:41.742 INFO:teuthology.orchestra.run.smithi157.stdout:2027 read 0x593c800 thru 0x593f8b4 (0x30b5 bytes) 2024-09-13T09:12:41.747 INFO:teuthology.orchestra.run.smithi157.stdout:2028 trunc from 0xac08400 to 0xda16000 2024-09-13T09:12:41.758 INFO:teuthology.orchestra.run.smithi157.stdout:2030 read 0x46dbe00 thru 0x46deaf8 (0x2cf9 bytes) 2024-09-13T09:12:41.760 INFO:teuthology.orchestra.run.smithi157.stdout:2031 read 0xa9dea00 thru 0xa9e21ed (0x37ee bytes) 2024-09-13T09:12:41.760 INFO:teuthology.orchestra.run.smithi157.stdout:2033 read 0x67e0600 thru 0x67eadbc (0xa7bd bytes) 2024-09-13T09:12:41.764 INFO:teuthology.orchestra.run.smithi157.stdout:2034 write 0x97c2400 thru 0x97c68c3 (0x44c4 bytes) 2024-09-13T09:12:41.775 INFO:teuthology.orchestra.run.smithi157.stdout:2035 trunc from 0xda16000 to 0x42ed200 2024-09-13T09:12:41.795 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:41.795 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:41.805 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:41.836 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:41.835+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:41.862 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:41.862 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:41.894 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:41.895 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:41.909 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:41.909 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:41.918 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:41.925 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:41.925 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:41.936 INFO:teuthology.orchestra.run.smithi157.stdout:2036 read 0x31a2200 thru 0x31a4300 (0x2101 bytes) 2024-09-13T09:12:41.943 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:41.943 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:41.943 INFO:teuthology.orchestra.run.smithi157.stdout:2037 read 0x321ee00 thru 0x3224a60 (0x5c61 bytes) 2024-09-13T09:12:41.946 INFO:teuthology.orchestra.run.smithi157.stdout:2039 write 0x267a400 thru 0x2688f9b (0xeb9c bytes) 2024-09-13T09:12:41.947 INFO:teuthology.orchestra.run.smithi157.stdout:2040 punch from 0x2c0cc00 to 0x2c11c00, (0x5000 bytes) 2024-09-13T09:12:41.961 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:41.961+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:41.971 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:41.972 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:42.000 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:42.001 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:42.036 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:42.036+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:42.065 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:42.065 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:42.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:42.065 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:42.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.065 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:15915 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 7100416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.065 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.015840+0000", 2024-09-13T09:12:42.065 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.046187343999999998, 2024-09-13T09:12:42.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039912374, 2024-09-13T09:12:42.065 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.065 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.066 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.066 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.066 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.066 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 15915 2024-09-13T09:12:42.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.066 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015840+0000", 2024-09-13T09:12:42.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015840+0000", 2024-09-13T09:12:42.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015840+0000", 2024-09-13T09:12:42.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015843+0000", 2024-09-13T09:12:42.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:12:42.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015852+0000", 2024-09-13T09:12:42.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4549999999999995e-06 2024-09-13T09:12:42.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015858+0000", 2024-09-13T09:12:42.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.818e-06 2024-09-13T09:12:42.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.055582+0000", 2024-09-13T09:12:42.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039724857000000002 2024-09-13T09:12:42.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.055667+0000", 2024-09-13T09:12:42.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4682e-05 2024-09-13T09:12:42.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.055753+0000", 2024-09-13T09:12:42.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5585000000000001e-05 2024-09-13T09:12:42.070 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.070 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.070 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.070 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:15917 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 7122944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.070 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.015892+0000", 2024-09-13T09:12:42.070 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.046136040000000003, 2024-09-13T09:12:42.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039982729000000002, 2024-09-13T09:12:42.071 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.071 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.071 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.071 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.071 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.071 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 15917 2024-09-13T09:12:42.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.071 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015892+0000", 2024-09-13T09:12:42.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015892+0000", 2024-09-13T09:12:42.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015891+0000", 2024-09-13T09:12:42.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015894+0000", 2024-09-13T09:12:42.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3830000000000001e-06 2024-09-13T09:12:42.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015904+0000", 2024-09-13T09:12:42.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2350000000000005e-06 2024-09-13T09:12:42.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015910+0000", 2024-09-13T09:12:42.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1059999999999996e-06 2024-09-13T09:12:42.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.055778+0000", 2024-09-13T09:12:42.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039868708000000003 2024-09-13T09:12:42.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.055826+0000", 2024-09-13T09:12:42.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7138999999999997e-05 2024-09-13T09:12:42.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.055874+0000", 2024-09-13T09:12:42.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8711999999999997e-05 2024-09-13T09:12:42.075 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.075 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.075 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.076 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:15918 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 7145472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.076 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.015920+0000", 2024-09-13T09:12:42.076 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.046107975000000002, 2024-09-13T09:12:42.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040080187000000003, 2024-09-13T09:12:42.076 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.076 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.076 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 15918 2024-09-13T09:12:42.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.076 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015920+0000", 2024-09-13T09:12:42.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015920+0000", 2024-09-13T09:12:42.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015919+0000", 2024-09-13T09:12:42.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015922+0000", 2024-09-13T09:12:42.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6769999999999999e-06 2024-09-13T09:12:42.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015929+0000", 2024-09-13T09:12:42.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9829999999999999e-06 2024-09-13T09:12:42.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015934+0000", 2024-09-13T09:12:42.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9779999999999996e-06 2024-09-13T09:12:42.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.055904+0000", 2024-09-13T09:12:42.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039970143999999999 2024-09-13T09:12:42.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.055949+0000", 2024-09-13T09:12:42.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5255e-05 2024-09-13T09:12:42.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056000+0000", 2024-09-13T09:12:42.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0596000000000001e-05 2024-09-13T09:12:42.080 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.080 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.080 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.080 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:15919 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 7168000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.080 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.015938+0000", 2024-09-13T09:12:42.080 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.046089170999999998, 2024-09-13T09:12:42.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040173795999999998, 2024-09-13T09:12:42.081 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.081 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.081 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.081 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.081 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.081 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 15919 2024-09-13T09:12:42.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.081 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015938+0000", 2024-09-13T09:12:42.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015938+0000", 2024-09-13T09:12:42.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015938+0000", 2024-09-13T09:12:42.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015940+0000", 2024-09-13T09:12:42.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2539999999999999e-06 2024-09-13T09:12:42.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015946+0000", 2024-09-13T09:12:42.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1639999999999998e-06 2024-09-13T09:12:42.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015951+0000", 2024-09-13T09:12:42.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:12:42.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056025+0000", 2024-09-13T09:12:42.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040073915000000002 2024-09-13T09:12:42.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056065+0000", 2024-09-13T09:12:42.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0052000000000002e-05 2024-09-13T09:12:42.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056112+0000", 2024-09-13T09:12:42.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7247999999999999e-05 2024-09-13T09:12:42.086 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.086 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.086 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.086 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:15920 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 7190528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.086 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.015956+0000", 2024-09-13T09:12:42.086 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.046072022999999997, 2024-09-13T09:12:42.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040290590000000001, 2024-09-13T09:12:42.087 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.087 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.087 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.087 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.087 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.087 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 15920 2024-09-13T09:12:42.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.087 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015956+0000", 2024-09-13T09:12:42.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015956+0000", 2024-09-13T09:12:42.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015955+0000", 2024-09-13T09:12:42.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015957+0000", 2024-09-13T09:12:42.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0779999999999998e-06 2024-09-13T09:12:42.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015963+0000", 2024-09-13T09:12:42.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.913e-06 2024-09-13T09:12:42.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.015972+0000", 2024-09-13T09:12:42.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2560000000000002e-06 2024-09-13T09:12:42.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056145+0000", 2024-09-13T09:12:42.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040173149999999998 2024-09-13T09:12:42.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056191+0000", 2024-09-13T09:12:42.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6349e-05 2024-09-13T09:12:42.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056246+0000", 2024-09-13T09:12:42.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5172e-05 2024-09-13T09:12:42.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.091 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.091 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:15921 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 7213056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.091 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.016067+0000", 2024-09-13T09:12:42.092 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.045960692999999997, 2024-09-13T09:12:42.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040430480999999997, 2024-09-13T09:12:42.092 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.092 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.092 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 15921 2024-09-13T09:12:42.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.092 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016067+0000", 2024-09-13T09:12:42.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016067+0000", 2024-09-13T09:12:42.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016066+0000", 2024-09-13T09:12:42.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:42.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016071+0000", 2024-09-13T09:12:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8420000000000004e-06 2024-09-13T09:12:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016079+0000", 2024-09-13T09:12:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:12:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016085+0000", 2024-09-13T09:12:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:12:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056289+0000", 2024-09-13T09:12:42.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040203307000000001 2024-09-13T09:12:42.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056384+0000", 2024-09-13T09:12:42.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5508999999999996e-05 2024-09-13T09:12:42.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056497+0000", 2024-09-13T09:12:42.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011312 2024-09-13T09:12:42.096 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.096 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.096 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.096 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:15923 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 7235584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.096 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.016098+0000", 2024-09-13T09:12:42.096 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.045929968000000002, 2024-09-13T09:12:42.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040602749, 2024-09-13T09:12:42.096 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.096 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.096 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.096 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.096 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.097 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 15923 2024-09-13T09:12:42.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.097 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016098+0000", 2024-09-13T09:12:42.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016098+0000", 2024-09-13T09:12:42.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016097+0000", 2024-09-13T09:12:42.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016100+0000", 2024-09-13T09:12:42.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:12:42.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016106+0000", 2024-09-13T09:12:42.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9399999999999999e-06 2024-09-13T09:12:42.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016110+0000", 2024-09-13T09:12:42.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5310000000000003e-06 2024-09-13T09:12:42.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056536+0000", 2024-09-13T09:12:42.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040425857000000003 2024-09-13T09:12:42.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056613+0000", 2024-09-13T09:12:42.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7057000000000004e-05 2024-09-13T09:12:42.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056700+0000", 2024-09-13T09:12:42.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7156000000000004e-05 2024-09-13T09:12:42.100 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.100 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.100 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.100 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:15924 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 7258112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.100 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.016192+0000", 2024-09-13T09:12:42.101 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.045835500000000001, 2024-09-13T09:12:42.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040722179999999997, 2024-09-13T09:12:42.101 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.101 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.101 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.101 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.101 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.101 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 15924 2024-09-13T09:12:42.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.101 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016192+0000", 2024-09-13T09:12:42.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016192+0000", 2024-09-13T09:12:42.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016191+0000", 2024-09-13T09:12:42.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:42.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016195+0000", 2024-09-13T09:12:42.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9049999999999999e-06 2024-09-13T09:12:42.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016205+0000", 2024-09-13T09:12:42.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2709999999999998e-06 2024-09-13T09:12:42.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016211+0000", 2024-09-13T09:12:42.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3169999999999998e-06 2024-09-13T09:12:42.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056744+0000", 2024-09-13T09:12:42.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040533365000000002 2024-09-13T09:12:42.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056826+0000", 2024-09-13T09:12:42.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2090999999999997e-05 2024-09-13T09:12:42.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056914+0000", 2024-09-13T09:12:42.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8002000000000002e-05 2024-09-13T09:12:42.110 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.110 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.110 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.111 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:15925 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 7280640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.111 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.016222+0000", 2024-09-13T09:12:42.111 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.045805905000000001, 2024-09-13T09:12:42.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040927008000000001, 2024-09-13T09:12:42.111 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.112 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.112 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.112 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.112 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.112 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 15925 2024-09-13T09:12:42.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.113 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016222+0000", 2024-09-13T09:12:42.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016222+0000", 2024-09-13T09:12:42.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016221+0000", 2024-09-13T09:12:42.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016224+0000", 2024-09-13T09:12:42.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7889999999999999e-06 2024-09-13T09:12:42.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016231+0000", 2024-09-13T09:12:42.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0520000000000004e-06 2024-09-13T09:12:42.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016239+0000", 2024-09-13T09:12:42.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2379999999999997e-06 2024-09-13T09:12:42.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.056959+0000", 2024-09-13T09:12:42.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040719784000000002 2024-09-13T09:12:42.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.057049+0000", 2024-09-13T09:12:42.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0415999999999996e-05 2024-09-13T09:12:42.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.057149+0000", 2024-09-13T09:12:42.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9252000000000004e-05 2024-09-13T09:12:42.120 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.120 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.120 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.120 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:15926 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 7303168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.120 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.016297+0000", 2024-09-13T09:12:42.120 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.045730347999999997, 2024-09-13T09:12:42.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041049137999999999, 2024-09-13T09:12:42.120 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.120 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.121 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 15926 2024-09-13T09:12:42.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.121 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016297+0000", 2024-09-13T09:12:42.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016297+0000", 2024-09-13T09:12:42.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016296+0000", 2024-09-13T09:12:42.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:42.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016301+0000", 2024-09-13T09:12:42.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1010000000000002e-06 2024-09-13T09:12:42.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016309+0000", 2024-09-13T09:12:42.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9199999999999993e-06 2024-09-13T09:12:42.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016315+0000", 2024-09-13T09:12:42.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:12:42.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.057187+0000", 2024-09-13T09:12:42.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040871223999999998 2024-09-13T09:12:42.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.057264+0000", 2024-09-13T09:12:42.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7002000000000006e-05 2024-09-13T09:12:42.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.057346+0000", 2024-09-13T09:12:42.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2717000000000001e-05 2024-09-13T09:12:42.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.125 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.125 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:15927 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 7325696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.125 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.016341+0000", 2024-09-13T09:12:42.125 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.045686579999999997, 2024-09-13T09:12:42.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041217589999999998, 2024-09-13T09:12:42.126 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.126 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.126 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 15927 2024-09-13T09:12:42.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.127 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016341+0000", 2024-09-13T09:12:42.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016341+0000", 2024-09-13T09:12:42.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016340+0000", 2024-09-13T09:12:42.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016344+0000", 2024-09-13T09:12:42.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3699999999999999e-06 2024-09-13T09:12:42.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016352+0000", 2024-09-13T09:12:42.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1689999999999992e-06 2024-09-13T09:12:42.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.016357+0000", 2024-09-13T09:12:42.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.322e-06 2024-09-13T09:12:42.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.057393+0000", 2024-09-13T09:12:42.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041035400999999999 2024-09-13T09:12:42.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.057478+0000", 2024-09-13T09:12:42.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5494000000000005e-05 2024-09-13T09:12:42.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.057559+0000", 2024-09-13T09:12:42.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0452999999999997e-05 2024-09-13T09:12:42.130 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.130 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.130 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.130 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16016 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 6449152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.131 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.019170+0000", 2024-09-13T09:12:42.131 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.042857933000000001, 2024-09-13T09:12:42.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040293389999999998, 2024-09-13T09:12:42.131 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.131 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.131 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.131 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.131 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.131 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16016 2024-09-13T09:12:42.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.131 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019170+0000", 2024-09-13T09:12:42.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019170+0000", 2024-09-13T09:12:42.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019169+0000", 2024-09-13T09:12:42.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019172+0000", 2024-09-13T09:12:42.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7350000000000001e-06 2024-09-13T09:12:42.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019183+0000", 2024-09-13T09:12:42.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1185999999999999e-05 2024-09-13T09:12:42.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019189+0000", 2024-09-13T09:12:42.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7749999999999998e-06 2024-09-13T09:12:42.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.058983+0000", 2024-09-13T09:12:42.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039794474000000003 2024-09-13T09:12:42.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.059069+0000", 2024-09-13T09:12:42.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5172000000000004e-05 2024-09-13T09:12:42.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.059463+0000", 2024-09-13T09:12:42.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039452200000000002 2024-09-13T09:12:42.136 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.136 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.136 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.137 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16017 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 6471680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.137 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.019194+0000", 2024-09-13T09:12:42.137 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.042833286999999998, 2024-09-13T09:12:42.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040490726999999997, 2024-09-13T09:12:42.137 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.137 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.137 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.137 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.137 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.137 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16017 2024-09-13T09:12:42.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.138 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019194+0000", 2024-09-13T09:12:42.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019194+0000", 2024-09-13T09:12:42.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019194+0000", 2024-09-13T09:12:42.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019196+0000", 2024-09-13T09:12:42.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-13T09:12:42.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019199+0000", 2024-09-13T09:12:42.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5750000000000001e-06 2024-09-13T09:12:42.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019202+0000", 2024-09-13T09:12:42.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5459999999999998e-06 2024-09-13T09:12:42.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.059510+0000", 2024-09-13T09:12:42.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040308297 2024-09-13T09:12:42.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.059594+0000", 2024-09-13T09:12:42.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3410000000000003e-05 2024-09-13T09:12:42.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.059685+0000", 2024-09-13T09:12:42.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1365000000000004e-05 2024-09-13T09:12:42.141 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.141 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.141 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.141 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16018 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 6494208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.141 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.019237+0000", 2024-09-13T09:12:42.142 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.042790439999999999, 2024-09-13T09:12:42.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041120029000000002, 2024-09-13T09:12:42.142 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.142 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.142 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.142 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.142 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.142 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16018 2024-09-13T09:12:42.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.142 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019237+0000", 2024-09-13T09:12:42.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019237+0000", 2024-09-13T09:12:42.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019237+0000", 2024-09-13T09:12:42.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019239+0000", 2024-09-13T09:12:42.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7460000000000001e-06 2024-09-13T09:12:42.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019242+0000", 2024-09-13T09:12:42.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6849999999999999e-06 2024-09-13T09:12:42.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019245+0000", 2024-09-13T09:12:42.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6960000000000001e-06 2024-09-13T09:12:42.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.059730+0000", 2024-09-13T09:12:42.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.040484461999999999 2024-09-13T09:12:42.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.059826+0000", 2024-09-13T09:12:42.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6947000000000005e-05 2024-09-13T09:12:42.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.060357+0000", 2024-09-13T09:12:42.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053074100000000005 2024-09-13T09:12:42.146 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.146 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.146 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.146 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16019 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 6516736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.146 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.019247+0000", 2024-09-13T09:12:42.146 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.042780363000000002, 2024-09-13T09:12:42.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041314044000000001, 2024-09-13T09:12:42.146 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.146 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.147 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16019 2024-09-13T09:12:42.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.147 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019247+0000", 2024-09-13T09:12:42.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019247+0000", 2024-09-13T09:12:42.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019247+0000", 2024-09-13T09:12:42.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019248+0000", 2024-09-13T09:12:42.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2270000000000001e-06 2024-09-13T09:12:42.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019251+0000", 2024-09-13T09:12:42.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8279999999999999e-06 2024-09-13T09:12:42.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019255+0000", 2024-09-13T09:12:42.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9020000000000001e-06 2024-09-13T09:12:42.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.060397+0000", 2024-09-13T09:12:42.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041142047000000001 2024-09-13T09:12:42.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.060476+0000", 2024-09-13T09:12:42.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8589000000000003e-05 2024-09-13T09:12:42.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.060561+0000", 2024-09-13T09:12:42.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5696000000000001e-05 2024-09-13T09:12:42.150 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.151 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.151 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.151 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16020 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 6539264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.151 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.019280+0000", 2024-09-13T09:12:42.151 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.042747815000000002, 2024-09-13T09:12:42.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041493084, 2024-09-13T09:12:42.151 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.151 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.151 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.151 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.151 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.152 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16020 2024-09-13T09:12:42.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.152 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019280+0000", 2024-09-13T09:12:42.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019280+0000", 2024-09-13T09:12:42.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019279+0000", 2024-09-13T09:12:42.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019282+0000", 2024-09-13T09:12:42.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5050000000000002e-06 2024-09-13T09:12:42.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019290+0000", 2024-09-13T09:12:42.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2460000000000003e-06 2024-09-13T09:12:42.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019294+0000", 2024-09-13T09:12:42.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3510000000000004e-06 2024-09-13T09:12:42.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.060588+0000", 2024-09-13T09:12:42.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041293241000000001 2024-09-13T09:12:42.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.060659+0000", 2024-09-13T09:12:42.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1518000000000002e-05 2024-09-13T09:12:42.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.060773+0000", 2024-09-13T09:12:42.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000113792 2024-09-13T09:12:42.155 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.155 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.155 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.155 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16021 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 6561792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.156 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.019299+0000", 2024-09-13T09:12:42.156 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.042729037999999997, 2024-09-13T09:12:42.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041663842999999999, 2024-09-13T09:12:42.156 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.156 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.156 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.156 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.156 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.156 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16021 2024-09-13T09:12:42.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.156 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019299+0000", 2024-09-13T09:12:42.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019299+0000", 2024-09-13T09:12:42.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019298+0000", 2024-09-13T09:12:42.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019300+0000", 2024-09-13T09:12:42.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.199e-06 2024-09-13T09:12:42.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019306+0000", 2024-09-13T09:12:42.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:12:42.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019310+0000", 2024-09-13T09:12:42.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.07e-06 2024-09-13T09:12:42.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.060798+0000", 2024-09-13T09:12:42.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041487745999999999 2024-09-13T09:12:42.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.060879+0000", 2024-09-13T09:12:42.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1130000000000004e-05 2024-09-13T09:12:42.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.060962+0000", 2024-09-13T09:12:42.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3288000000000006e-05 2024-09-13T09:12:42.160 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.160 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.160 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.160 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16022 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 6584320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.160 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.019319+0000", 2024-09-13T09:12:42.160 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.042709101999999999, 2024-09-13T09:12:42.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041874969999999997, 2024-09-13T09:12:42.160 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.160 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.160 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.161 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.161 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.161 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16022 2024-09-13T09:12:42.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.161 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019319+0000", 2024-09-13T09:12:42.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019319+0000", 2024-09-13T09:12:42.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019318+0000", 2024-09-13T09:12:42.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019320+0000", 2024-09-13T09:12:42.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.131e-06 2024-09-13T09:12:42.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019330+0000", 2024-09-13T09:12:42.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0087e-05 2024-09-13T09:12:42.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019337+0000", 2024-09-13T09:12:42.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8279999999999997e-06 2024-09-13T09:12:42.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.061001+0000", 2024-09-13T09:12:42.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041663736999999999 2024-09-13T09:12:42.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.061093+0000", 2024-09-13T09:12:42.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1667000000000002e-05 2024-09-13T09:12:42.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.061193+0000", 2024-09-13T09:12:42.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010087899999999999 2024-09-13T09:12:42.164 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.165 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.165 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.165 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16023 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 6606848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.165 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.019393+0000", 2024-09-13T09:12:42.165 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.042634901000000003, 2024-09-13T09:12:42.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042004993999999997, 2024-09-13T09:12:42.165 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.165 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.165 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.165 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.165 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.165 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16023 2024-09-13T09:12:42.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.166 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019393+0000", 2024-09-13T09:12:42.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019393+0000", 2024-09-13T09:12:42.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019392+0000", 2024-09-13T09:12:42.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019396+0000", 2024-09-13T09:12:42.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.653e-06 2024-09-13T09:12:42.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019403+0000", 2024-09-13T09:12:42.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7100000000000007e-06 2024-09-13T09:12:42.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019409+0000", 2024-09-13T09:12:42.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4689999999999996e-06 2024-09-13T09:12:42.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.061233+0000", 2024-09-13T09:12:42.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041824400999999997 2024-09-13T09:12:42.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.061313+0000", 2024-09-13T09:12:42.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0056999999999996e-05 2024-09-13T09:12:42.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.061398+0000", 2024-09-13T09:12:42.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4417000000000002e-05 2024-09-13T09:12:42.169 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.169 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.170 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.170 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16024 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 6629376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.170 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.019419+0000", 2024-09-13T09:12:42.170 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.042609088000000003, 2024-09-13T09:12:42.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042178126000000003, 2024-09-13T09:12:42.170 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.170 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.170 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.170 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.170 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.170 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16024 2024-09-13T09:12:42.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.171 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019419+0000", 2024-09-13T09:12:42.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019419+0000", 2024-09-13T09:12:42.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019418+0000", 2024-09-13T09:12:42.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019421+0000", 2024-09-13T09:12:42.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.492e-06 2024-09-13T09:12:42.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019426+0000", 2024-09-13T09:12:42.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7559999999999996e-06 2024-09-13T09:12:42.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.019431+0000", 2024-09-13T09:12:42.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4479999999999996e-06 2024-09-13T09:12:42.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.061430+0000", 2024-09-13T09:12:42.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041999622 2024-09-13T09:12:42.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.061505+0000", 2024-09-13T09:12:42.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4929e-05 2024-09-13T09:12:42.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.061597+0000", 2024-09-13T09:12:42.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1304000000000005e-05 2024-09-13T09:12:42.174 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.174 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.174 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.174 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.174 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.174 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:42.175 INFO:teuthology.orchestra.run.smithi157.stdout:2041 punch from 0xfe5c00 to 0xfee800, (0x8c00 bytes) 2024-09-13T09:12:42.176 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:42.177 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:42.175+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:42.187 INFO:teuthology.orchestra.run.smithi157.stdout:2047 read 0xd96a00 thru 0xd999df (0x2fe0 bytes) 2024-09-13T09:12:42.276 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:42.275+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:42.333 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:42.333 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:42.333 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:42.333 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:42.341 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:42.377 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:42.376+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:42.451 INFO:teuthology.orchestra.run.smithi157.stdout:2049 write 0xd2a0c00 thru 0xd2ae499 (0xd89a bytes) 2024-09-13T09:12:42.456 INFO:teuthology.orchestra.run.smithi157.stdout:2051 read 0xb6d2600 thru 0xb6d9817 (0x7218 bytes) 2024-09-13T09:12:42.456 INFO:teuthology.orchestra.run.smithi157.stdout:2052 trunc from 0xd2ae600 to 0x9c0f800 2024-09-13T09:12:42.478 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:42.477+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:42.495 INFO:teuthology.orchestra.run.smithi157.stdout:2054 write 0x87bba00 thru 0x87c11bc (0x57bd bytes) 2024-09-13T09:12:42.496 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:42.497 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:42.497 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:42.497 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:42.497 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:42.500 INFO:teuthology.orchestra.run.smithi157.stdout:2055 punch from 0x1fec400 to 0x1ff0e00, (0x4a00 bytes) 2024-09-13T09:12:42.506 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:42.526 INFO:teuthology.orchestra.run.smithi044.stderr:marked in osd.3. 2024-09-13T09:12:42.527 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:12:42.532 INFO:teuthology.orchestra.run.smithi157.stdout:2060 trunc from 0x9c0f800 to 0xba4e400 2024-09-13T09:12:42.547 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-13T09:12:42.591 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:42.590+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:42.654 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:42.654 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:42.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16127 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 7094272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.022986+0000", 2024-09-13T09:12:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.62949371200000004, 2024-09-13T09:12:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045533075999999999, 2024-09-13T09:12:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16127 2024-09-13T09:12:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.022986+0000", 2024-09-13T09:12:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.022986+0000", 2024-09-13T09:12:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.022986+0000", 2024-09-13T09:12:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.022989+0000", 2024-09-13T09:12:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8260000000000001e-06 2024-09-13T09:12:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.022994+0000", 2024-09-13T09:12:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.78e-06 2024-09-13T09:12:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.022996+0000", 2024-09-13T09:12:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8449999999999999e-06 2024-09-13T09:12:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068381+0000", 2024-09-13T09:12:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045384437 2024-09-13T09:12:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068461+0000", 2024-09-13T09:12:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0069999999999997e-05 2024-09-13T09:12:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068520+0000", 2024-09-13T09:12:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8656000000000002e-05 2024-09-13T09:12:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16134 3.c 3.c5acf96c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.023285+0000", 2024-09-13T09:12:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.629195481, 2024-09-13T09:12:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045544921000000002, 2024-09-13T09:12:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16134 2024-09-13T09:12:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023285+0000", 2024-09-13T09:12:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023285+0000", 2024-09-13T09:12:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023284+0000", 2024-09-13T09:12:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023286+0000", 2024-09-13T09:12:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.316e-06 2024-09-13T09:12:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023289+0000", 2024-09-13T09:12:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8169999999999999e-06 2024-09-13T09:12:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023291+0000", 2024-09-13T09:12:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.418e-06 2024-09-13T09:12:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068823+0000", 2024-09-13T09:12:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045531644000000003 2024-09-13T09:12:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068830+0000", 2024-09-13T09:12:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9450000000000004e-06 2024-09-13T09:12:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16135 3.c 3.c5acf96c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.023293+0000", 2024-09-13T09:12:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.62918720500000003, 2024-09-13T09:12:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045568511999999999, 2024-09-13T09:12:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16135 2024-09-13T09:12:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023293+0000", 2024-09-13T09:12:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023293+0000", 2024-09-13T09:12:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023293+0000", 2024-09-13T09:12:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023294+0000", 2024-09-13T09:12:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.203e-06 2024-09-13T09:12:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023297+0000", 2024-09-13T09:12:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.768e-06 2024-09-13T09:12:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023300+0000", 2024-09-13T09:12:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2270000000000002e-06 2024-09-13T09:12:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068853+0000", 2024-09-13T09:12:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045553278000000003 2024-09-13T09:12:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068861+0000", 2024-09-13T09:12:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2579999999999995e-06 2024-09-13T09:12:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16136 3.c 3.c5acf96c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.023304+0000", 2024-09-13T09:12:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.62917640100000005, 2024-09-13T09:12:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045591547000000003, 2024-09-13T09:12:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16136 2024-09-13T09:12:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023304+0000", 2024-09-13T09:12:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023304+0000", 2024-09-13T09:12:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023304+0000", 2024-09-13T09:12:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023305+0000", 2024-09-13T09:12:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.314e-06 2024-09-13T09:12:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023308+0000", 2024-09-13T09:12:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9919999999999999e-06 2024-09-13T09:12:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023310+0000", 2024-09-13T09:12:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.57e-06 2024-09-13T09:12:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068887+0000", 2024-09-13T09:12:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045576608999999997 2024-09-13T09:12:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068895+0000", 2024-09-13T09:12:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2959999999999999e-06 2024-09-13T09:12:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16137 3.c 3.c5acf96c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.023331+0000", 2024-09-13T09:12:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.62914931799999996, 2024-09-13T09:12:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045595084000000001, 2024-09-13T09:12:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16137 2024-09-13T09:12:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023331+0000", 2024-09-13T09:12:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023331+0000", 2024-09-13T09:12:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023330+0000", 2024-09-13T09:12:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023333+0000", 2024-09-13T09:12:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4889999999999998e-06 2024-09-13T09:12:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023341+0000", 2024-09-13T09:12:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8779999999999998e-06 2024-09-13T09:12:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023346+0000", 2024-09-13T09:12:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3600000000000004e-06 2024-09-13T09:12:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068919+0000", 2024-09-13T09:12:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045572806 2024-09-13T09:12:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068926+0000", 2024-09-13T09:12:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.002e-06 2024-09-13T09:12:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17711 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2273280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140832+0000", 2024-09-13T09:12:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.51164812000000004, 2024-09-13T09:12:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045543037000000001, 2024-09-13T09:12:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17711 2024-09-13T09:12:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140832+0000", 2024-09-13T09:12:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140832+0000", 2024-09-13T09:12:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140832+0000", 2024-09-13T09:12:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140834+0000", 2024-09-13T09:12:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:12:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140840+0000", 2024-09-13T09:12:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9519999999999999e-06 2024-09-13T09:12:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140844+0000", 2024-09-13T09:12:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3529999999999997e-06 2024-09-13T09:12:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186207+0000", 2024-09-13T09:12:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045362864000000003 2024-09-13T09:12:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186287+0000", 2024-09-13T09:12:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0023000000000003e-05 2024-09-13T09:12:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186375+0000", 2024-09-13T09:12:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7978000000000004e-05 2024-09-13T09:12:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17712 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2295808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140848+0000", 2024-09-13T09:12:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.51163195900000003, 2024-09-13T09:12:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045675362999999997, 2024-09-13T09:12:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17712 2024-09-13T09:12:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140848+0000", 2024-09-13T09:12:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140848+0000", 2024-09-13T09:12:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140848+0000", 2024-09-13T09:12:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140850+0000", 2024-09-13T09:12:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0990000000000001e-06 2024-09-13T09:12:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140856+0000", 2024-09-13T09:12:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7289999999999997e-06 2024-09-13T09:12:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140860+0000", 2024-09-13T09:12:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6260000000000003e-06 2024-09-13T09:12:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186414+0000", 2024-09-13T09:12:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045553693999999999 2024-09-13T09:12:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186470+0000", 2024-09-13T09:12:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5855e-05 2024-09-13T09:12:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186524+0000", 2024-09-13T09:12:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3732e-05 2024-09-13T09:12:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17713 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2318336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140864+0000", 2024-09-13T09:12:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.51161599000000002, 2024-09-13T09:12:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045842250000000001, 2024-09-13T09:12:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17713 2024-09-13T09:12:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140864+0000", 2024-09-13T09:12:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140864+0000", 2024-09-13T09:12:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140864+0000", 2024-09-13T09:12:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140866+0000", 2024-09-13T09:12:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0590000000000001e-06 2024-09-13T09:12:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140871+0000", 2024-09-13T09:12:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4990000000000002e-06 2024-09-13T09:12:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140876+0000", 2024-09-13T09:12:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3810000000000001e-06 2024-09-13T09:12:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186555+0000", 2024-09-13T09:12:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045679059000000001 2024-09-13T09:12:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186631+0000", 2024-09-13T09:12:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6168999999999995e-05 2024-09-13T09:12:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186706+0000", 2024-09-13T09:12:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5420999999999995e-05 2024-09-13T09:12:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17714 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2340864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140880+0000", 2024-09-13T09:12:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.51160052199999995, 2024-09-13T09:12:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045950484999999999, 2024-09-13T09:12:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17714 2024-09-13T09:12:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140880+0000", 2024-09-13T09:12:42.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140880+0000", 2024-09-13T09:12:42.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140879+0000", 2024-09-13T09:12:42.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140881+0000", 2024-09-13T09:12:42.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9810000000000002e-06 2024-09-13T09:12:42.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140887+0000", 2024-09-13T09:12:42.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.874e-06 2024-09-13T09:12:42.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140891+0000", 2024-09-13T09:12:42.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3839999999999999e-06 2024-09-13T09:12:42.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186735+0000", 2024-09-13T09:12:42.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045843119000000002 2024-09-13T09:12:42.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186778+0000", 2024-09-13T09:12:42.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3618000000000001e-05 2024-09-13T09:12:42.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186830+0000", 2024-09-13T09:12:42.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1919000000000002e-05 2024-09-13T09:12:42.697 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.697 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.697 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.697 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17715 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2363392~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.697 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140896+0000", 2024-09-13T09:12:42.697 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.51158457700000004, 2024-09-13T09:12:42.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046059503000000002, 2024-09-13T09:12:42.698 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.698 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.698 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.698 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.698 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.698 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17715 2024-09-13T09:12:42.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.698 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140896+0000", 2024-09-13T09:12:42.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140896+0000", 2024-09-13T09:12:42.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140895+0000", 2024-09-13T09:12:42.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140897+0000", 2024-09-13T09:12:42.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.029e-06 2024-09-13T09:12:42.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140904+0000", 2024-09-13T09:12:42.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9280000000000004e-06 2024-09-13T09:12:42.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140908+0000", 2024-09-13T09:12:42.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.262e-06 2024-09-13T09:12:42.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186856+0000", 2024-09-13T09:12:42.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045947057999999999 2024-09-13T09:12:42.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186899+0000", 2024-09-13T09:12:42.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3658000000000001e-05 2024-09-13T09:12:42.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186955+0000", 2024-09-13T09:12:42.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5899999999999997e-05 2024-09-13T09:12:42.702 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.702 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.703 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.703 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17716 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2385920~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.703 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140959+0000", 2024-09-13T09:12:42.703 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.51152099900000003, 2024-09-13T09:12:42.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046114687000000001, 2024-09-13T09:12:42.703 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.703 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.703 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.703 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.703 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.703 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17716 2024-09-13T09:12:42.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.704 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140959+0000", 2024-09-13T09:12:42.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140959+0000", 2024-09-13T09:12:42.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140912+0000", 2024-09-13T09:12:42.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999533 2024-09-13T09:12:42.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140961+0000", 2024-09-13T09:12:42.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8798999999999998e-05 2024-09-13T09:12:42.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140966+0000", 2024-09-13T09:12:42.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4970000000000003e-06 2024-09-13T09:12:42.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140969+0000", 2024-09-13T09:12:42.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4170000000000001e-06 2024-09-13T09:12:42.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186982+0000", 2024-09-13T09:12:42.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046012982000000001 2024-09-13T09:12:42.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187024+0000", 2024-09-13T09:12:42.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1894999999999998e-05 2024-09-13T09:12:42.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187074+0000", 2024-09-13T09:12:42.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9956000000000001e-05 2024-09-13T09:12:42.707 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.707 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.707 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.708 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17717 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2408448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.708 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.141107+0000", 2024-09-13T09:12:42.708 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.51137285600000004, 2024-09-13T09:12:42.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046093783999999999, 2024-09-13T09:12:42.708 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.708 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.708 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17717 2024-09-13T09:12:42.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.709 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141107+0000", 2024-09-13T09:12:42.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141107+0000", 2024-09-13T09:12:42.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140972+0000", 2024-09-13T09:12:42.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998646 2024-09-13T09:12:42.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141111+0000", 2024-09-13T09:12:42.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000138765 2024-09-13T09:12:42.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141116+0000", 2024-09-13T09:12:42.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:12:42.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141121+0000", 2024-09-13T09:12:42.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4120000000000003e-06 2024-09-13T09:12:42.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187096+0000", 2024-09-13T09:12:42.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045974947000000002 2024-09-13T09:12:42.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187145+0000", 2024-09-13T09:12:42.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9548000000000001e-05 2024-09-13T09:12:42.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187201+0000", 2024-09-13T09:12:42.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5708000000000003e-05 2024-09-13T09:12:42.712 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.712 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.712 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.712 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17718 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2430976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.712 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.141258+0000", 2024-09-13T09:12:42.713 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.51122230700000004, 2024-09-13T09:12:42.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046068460999999998, 2024-09-13T09:12:42.713 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.713 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.713 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.713 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.713 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.713 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17718 2024-09-13T09:12:42.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.713 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141258+0000", 2024-09-13T09:12:42.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141258+0000", 2024-09-13T09:12:42.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141125+0000", 2024-09-13T09:12:42.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999867 2024-09-13T09:12:42.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141261+0000", 2024-09-13T09:12:42.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013606499999999999 2024-09-13T09:12:42.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141268+0000", 2024-09-13T09:12:42.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4610000000000002e-06 2024-09-13T09:12:42.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141275+0000", 2024-09-13T09:12:42.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8519999999999998e-06 2024-09-13T09:12:42.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187237+0000", 2024-09-13T09:12:42.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045961958999999997 2024-09-13T09:12:42.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187279+0000", 2024-09-13T09:12:42.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2324000000000003e-05 2024-09-13T09:12:42.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187326+0000", 2024-09-13T09:12:42.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7009999999999999e-05 2024-09-13T09:12:42.717 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.718 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.718 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.718 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17719 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2453504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.718 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.141428+0000", 2024-09-13T09:12:42.718 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.51105249799999997, 2024-09-13T09:12:42.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046028438999999997, 2024-09-13T09:12:42.718 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.718 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.718 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17719 2024-09-13T09:12:42.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.719 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141428+0000", 2024-09-13T09:12:42.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141428+0000", 2024-09-13T09:12:42.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141280+0000", 2024-09-13T09:12:42.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998522 2024-09-13T09:12:42.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141430+0000", 2024-09-13T09:12:42.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00015054599999999999 2024-09-13T09:12:42.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141435+0000", 2024-09-13T09:12:42.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:12:42.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141439+0000", 2024-09-13T09:12:42.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7699999999999999e-06 2024-09-13T09:12:42.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187350+0000", 2024-09-13T09:12:42.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045910377000000002 2024-09-13T09:12:42.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187398+0000", 2024-09-13T09:12:42.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8795999999999998e-05 2024-09-13T09:12:42.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187456+0000", 2024-09-13T09:12:42.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7667000000000002e-05 2024-09-13T09:12:42.722 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.723 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.723 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.723 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17720 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2476032~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.723 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.141638+0000", 2024-09-13T09:12:42.723 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.510841714, 2024-09-13T09:12:42.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045968368000000003, 2024-09-13T09:12:42.723 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.723 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.723 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.723 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.723 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.723 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17720 2024-09-13T09:12:42.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.724 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141638+0000", 2024-09-13T09:12:42.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141638+0000", 2024-09-13T09:12:42.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141443+0000", 2024-09-13T09:12:42.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999804 2024-09-13T09:12:42.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141641+0000", 2024-09-13T09:12:42.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019882100000000001 2024-09-13T09:12:42.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141648+0000", 2024-09-13T09:12:42.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.041e-06 2024-09-13T09:12:42.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141653+0000", 2024-09-13T09:12:42.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.442e-06 2024-09-13T09:12:42.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187494+0000", 2024-09-13T09:12:42.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045841621999999999 2024-09-13T09:12:42.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187555+0000", 2024-09-13T09:12:42.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0979e-05 2024-09-13T09:12:42.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187607+0000", 2024-09-13T09:12:42.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1390000000000001e-05 2024-09-13T09:12:42.728 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.728 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.728 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.728 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17723 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2543616~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.728 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142510+0000", 2024-09-13T09:12:42.728 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.50997014399999996, 2024-09-13T09:12:42.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045602886000000002, 2024-09-13T09:12:42.729 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.729 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.729 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.729 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.729 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.729 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17723 2024-09-13T09:12:42.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.729 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142510+0000", 2024-09-13T09:12:42.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142510+0000", 2024-09-13T09:12:42.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142510+0000", 2024-09-13T09:12:42.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142512+0000", 2024-09-13T09:12:42.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:12:42.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142518+0000", 2024-09-13T09:12:42.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5679999999999996e-06 2024-09-13T09:12:42.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142523+0000", 2024-09-13T09:12:42.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9409999999999998e-06 2024-09-13T09:12:42.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187960+0000", 2024-09-13T09:12:42.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045436973999999998 2024-09-13T09:12:42.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188032+0000", 2024-09-13T09:12:42.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1633999999999999e-05 2024-09-13T09:12:42.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188113+0000", 2024-09-13T09:12:42.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0866999999999997e-05 2024-09-13T09:12:42.733 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.733 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.733 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.733 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17724 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2566144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.733 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142529+0000", 2024-09-13T09:12:42.733 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.50995067100000002, 2024-09-13T09:12:42.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045741345000000003, 2024-09-13T09:12:42.733 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.733 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.734 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17724 2024-09-13T09:12:42.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.734 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142529+0000", 2024-09-13T09:12:42.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142529+0000", 2024-09-13T09:12:42.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142529+0000", 2024-09-13T09:12:42.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142531+0000", 2024-09-13T09:12:42.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4320000000000002e-06 2024-09-13T09:12:42.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142538+0000", 2024-09-13T09:12:42.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4330000000000001e-06 2024-09-13T09:12:42.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142543+0000", 2024-09-13T09:12:42.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:12:42.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188147+0000", 2024-09-13T09:12:42.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045603871999999997 2024-09-13T09:12:42.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188191+0000", 2024-09-13T09:12:42.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4388000000000003e-05 2024-09-13T09:12:42.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188271+0000", 2024-09-13T09:12:42.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9808999999999994e-05 2024-09-13T09:12:42.738 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.738 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.738 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.738 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17725 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2588672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.738 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142547+0000", 2024-09-13T09:12:42.738 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.50993287399999998, 2024-09-13T09:12:42.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045918237000000001, 2024-09-13T09:12:42.738 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.738 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.738 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.739 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.739 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.739 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17725 2024-09-13T09:12:42.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.739 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142547+0000", 2024-09-13T09:12:42.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142547+0000", 2024-09-13T09:12:42.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142547+0000", 2024-09-13T09:12:42.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142549+0000", 2024-09-13T09:12:42.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1210000000000001e-06 2024-09-13T09:12:42.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142558+0000", 2024-09-13T09:12:42.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8720000000000008e-06 2024-09-13T09:12:42.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142563+0000", 2024-09-13T09:12:42.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:12:42.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188305+0000", 2024-09-13T09:12:42.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045742532000000002 2024-09-13T09:12:42.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188380+0000", 2024-09-13T09:12:42.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4277999999999996e-05 2024-09-13T09:12:42.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188466+0000", 2024-09-13T09:12:42.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5827000000000003e-05 2024-09-13T09:12:42.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.742 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.743 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17735 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2813952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.743 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144629+0000", 2024-09-13T09:12:42.743 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.50785140600000001, 2024-09-13T09:12:42.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045584794999999997, 2024-09-13T09:12:42.743 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.743 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.743 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17735 2024-09-13T09:12:42.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.743 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144629+0000", 2024-09-13T09:12:42.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144629+0000", 2024-09-13T09:12:42.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144628+0000", 2024-09-13T09:12:42.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144631+0000", 2024-09-13T09:12:42.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3889999999999999e-06 2024-09-13T09:12:42.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144636+0000", 2024-09-13T09:12:42.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:12:42.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144641+0000", 2024-09-13T09:12:42.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4000000000000002e-06 2024-09-13T09:12:42.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190062+0000", 2024-09-13T09:12:42.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045421647000000002 2024-09-13T09:12:42.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190135+0000", 2024-09-13T09:12:42.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2191000000000001e-05 2024-09-13T09:12:42.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190214+0000", 2024-09-13T09:12:42.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8889999999999999e-05 2024-09-13T09:12:42.747 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.747 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.747 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.747 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17736 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2836480~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.747 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144676+0000", 2024-09-13T09:12:42.747 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.50780446700000004, 2024-09-13T09:12:42.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045739389999999998, 2024-09-13T09:12:42.747 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:42.747 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:42.748 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:42.748 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:42.748 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.748 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17736 2024-09-13T09:12:42.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.748 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:42.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:42.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144676+0000", 2024-09-13T09:12:42.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:42.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144676+0000", 2024-09-13T09:12:42.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:42.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:42.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144645+0000", 2024-09-13T09:12:42.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999969 2024-09-13T09:12:42.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:42.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144678+0000", 2024-09-13T09:12:42.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3572999999999999e-05 2024-09-13T09:12:42.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:42.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144687+0000", 2024-09-13T09:12:42.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4670000000000004e-06 2024-09-13T09:12:42.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144692+0000", 2024-09-13T09:12:42.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4099999999999999e-06 2024-09-13T09:12:42.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:42.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190247+0000", 2024-09-13T09:12:42.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045555224999999998 2024-09-13T09:12:42.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:42.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190329+0000", 2024-09-13T09:12:42.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1630000000000003e-05 2024-09-13T09:12:42.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:42.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:42.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:42.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190415+0000", 2024-09-13T09:12:42.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6017e-05 2024-09-13T09:12:42.752 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.752 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.752 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.752 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:42.752 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:42.754 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:42.753+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:42.755 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:42.854 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:42.853+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:42.886 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:42.886 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:42.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.886 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19952 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.886 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.525026+0000", 2024-09-13T09:12:42.886 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33976314299999999, 2024-09-13T09:12:42.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33977881500000001, 2024-09-13T09:12:42.887 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.887 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.887 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.887 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.887 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.887 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19952 2024-09-13T09:12:42.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.887 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525026+0000", 2024-09-13T09:12:42.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525026+0000", 2024-09-13T09:12:42.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525025+0000", 2024-09-13T09:12:42.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525030+0000", 2024-09-13T09:12:42.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5820000000000003e-06 2024-09-13T09:12:42.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525038+0000", 2024-09-13T09:12:42.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2199999999999992e-06 2024-09-13T09:12:42.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525044+0000", 2024-09-13T09:12:42.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2310000000000001e-06 2024-09-13T09:12:42.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534691+0000", 2024-09-13T09:12:42.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00964631 2024-09-13T09:12:42.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534697+0000", 2024-09-13T09:12:42.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7170000000000002e-06 2024-09-13T09:12:42.890 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.890 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.890 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.890 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20062 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.890 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528584+0000", 2024-09-13T09:12:42.891 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33620517, 2024-09-13T09:12:42.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33627451400000002, 2024-09-13T09:12:42.891 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.891 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.891 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.891 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.891 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.891 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20062 2024-09-13T09:12:42.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.891 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528584+0000", 2024-09-13T09:12:42.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528584+0000", 2024-09-13T09:12:42.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528584+0000", 2024-09-13T09:12:42.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528586+0000", 2024-09-13T09:12:42.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2239999999999998e-06 2024-09-13T09:12:42.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528594+0000", 2024-09-13T09:12:42.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9489999999999996e-06 2024-09-13T09:12:42.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528598+0000", 2024-09-13T09:12:42.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6399999999999996e-06 2024-09-13T09:12:42.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535255+0000", 2024-09-13T09:12:42.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0066562160000000004 2024-09-13T09:12:42.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535260+0000", 2024-09-13T09:12:42.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7219999999999996e-06 2024-09-13T09:12:42.894 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.894 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.894 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.895 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20204 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.895 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538602+0000", 2024-09-13T09:12:42.895 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.326187273, 2024-09-13T09:12:42.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32629225299999998, 2024-09-13T09:12:42.895 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.895 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.895 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.895 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.895 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.895 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20204 2024-09-13T09:12:42.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.896 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538602+0000", 2024-09-13T09:12:42.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538602+0000", 2024-09-13T09:12:42.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538601+0000", 2024-09-13T09:12:42.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:42.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538604+0000", 2024-09-13T09:12:42.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.615e-06 2024-09-13T09:12:42.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538610+0000", 2024-09-13T09:12:42.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3890000000000002e-06 2024-09-13T09:12:42.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538616+0000", 2024-09-13T09:12:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1569999999999997e-06 2024-09-13T09:12:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539096+0000", 2024-09-13T09:12:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00047983100000000001 2024-09-13T09:12:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539103+0000", 2024-09-13T09:12:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6229999999999999e-06 2024-09-13T09:12:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.899 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.899 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.899 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.899 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:12:42.899 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32418176100000001, 2024-09-13T09:12:42.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32432119100000001, 2024-09-13T09:12:42.899 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.899 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.899 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.899 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.899 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.899 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:12:42.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.900 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:12:42.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:12:42.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:12:42.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:12:42.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:12:42.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:12:42.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:12:42.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:12:42.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:12:42.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:12:42.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:12:42.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:12:42.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:12:42.903 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.904 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.904 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.904 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20458 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.904 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543571+0000", 2024-09-13T09:12:42.904 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32121809699999998, 2024-09-13T09:12:42.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32139291800000003, 2024-09-13T09:12:42.904 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.904 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.904 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.904 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.904 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.905 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20458 2024-09-13T09:12:42.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.905 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543571+0000", 2024-09-13T09:12:42.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543571+0000", 2024-09-13T09:12:42.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543571+0000", 2024-09-13T09:12:42.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543573+0000", 2024-09-13T09:12:42.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0899999999999999e-06 2024-09-13T09:12:42.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543579+0000", 2024-09-13T09:12:42.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:12:42.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543584+0000", 2024-09-13T09:12:42.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:12:42.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544145+0000", 2024-09-13T09:12:42.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056116100000000002 2024-09-13T09:12:42.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544152+0000", 2024-09-13T09:12:42.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2119999999999997e-06 2024-09-13T09:12:42.908 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.908 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.908 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.908 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20569 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.908 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547226+0000", 2024-09-13T09:12:42.908 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.317563451, 2024-09-13T09:12:42.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31777186899999998, 2024-09-13T09:12:42.908 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.908 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.909 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.909 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.909 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.909 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20569 2024-09-13T09:12:42.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.909 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547226+0000", 2024-09-13T09:12:42.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547226+0000", 2024-09-13T09:12:42.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547225+0000", 2024-09-13T09:12:42.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547228+0000", 2024-09-13T09:12:42.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:12:42.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547234+0000", 2024-09-13T09:12:42.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.37e-06 2024-09-13T09:12:42.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547239+0000", 2024-09-13T09:12:42.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:12:42.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547296+0000", 2024-09-13T09:12:42.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6881e-05 2024-09-13T09:12:42.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547304+0000", 2024-09-13T09:12:42.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8909999999999995e-06 2024-09-13T09:12:42.912 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.912 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.912 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.912 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20605 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.912 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.548241+0000", 2024-09-13T09:12:42.912 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.316548365, 2024-09-13T09:12:42.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31679059999999998, 2024-09-13T09:12:42.912 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.913 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.913 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.913 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.913 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.913 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20605 2024-09-13T09:12:42.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.913 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548241+0000", 2024-09-13T09:12:42.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548241+0000", 2024-09-13T09:12:42.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548240+0000", 2024-09-13T09:12:42.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548243+0000", 2024-09-13T09:12:42.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7779999999999999e-06 2024-09-13T09:12:42.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548250+0000", 2024-09-13T09:12:42.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0430000000000002e-06 2024-09-13T09:12:42.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548255+0000", 2024-09-13T09:12:42.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9869999999999999e-06 2024-09-13T09:12:42.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548668+0000", 2024-09-13T09:12:42.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00041251699999999999 2024-09-13T09:12:42.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548675+0000", 2024-09-13T09:12:42.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:12:42.916 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.916 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.916 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.916 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20719 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.917 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549880+0000", 2024-09-13T09:12:42.917 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31490902100000001, 2024-09-13T09:12:42.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31518764999999999, 2024-09-13T09:12:42.917 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.917 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.917 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.917 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.917 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.917 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20719 2024-09-13T09:12:42.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.917 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549880+0000", 2024-09-13T09:12:42.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549880+0000", 2024-09-13T09:12:42.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549880+0000", 2024-09-13T09:12:42.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549882+0000", 2024-09-13T09:12:42.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7800000000000001e-06 2024-09-13T09:12:42.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549889+0000", 2024-09-13T09:12:42.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9589999999999998e-06 2024-09-13T09:12:42.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549894+0000", 2024-09-13T09:12:42.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0089999999999998e-06 2024-09-13T09:12:42.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549987+0000", 2024-09-13T09:12:42.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2841000000000001e-05 2024-09-13T09:12:42.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549995+0000", 2024-09-13T09:12:42.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0390000000000005e-06 2024-09-13T09:12:42.920 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.920 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.920 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.921 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19953 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.921 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.525072+0000", 2024-09-13T09:12:42.921 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33971709500000002, 2024-09-13T09:12:42.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34002960599999998, 2024-09-13T09:12:42.921 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.921 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.921 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.921 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.921 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.921 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19953 2024-09-13T09:12:42.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.921 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525072+0000", 2024-09-13T09:12:42.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525072+0000", 2024-09-13T09:12:42.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525071+0000", 2024-09-13T09:12:42.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:42.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525082+0000", 2024-09-13T09:12:42.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.118e-05 2024-09-13T09:12:42.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525092+0000", 2024-09-13T09:12:42.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0037000000000001e-05 2024-09-13T09:12:42.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525101+0000", 2024-09-13T09:12:42.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5010000000000004e-06 2024-09-13T09:12:42.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534721+0000", 2024-09-13T09:12:42.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.009620099 2024-09-13T09:12:42.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534728+0000", 2024-09-13T09:12:42.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0219999999999999e-06 2024-09-13T09:12:42.924 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.925 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.925 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.925 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20063 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.925 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528604+0000", 2024-09-13T09:12:42.925 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33618526100000001, 2024-09-13T09:12:42.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.336531307, 2024-09-13T09:12:42.925 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.925 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.925 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.925 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.925 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.926 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20063 2024-09-13T09:12:42.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.926 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528604+0000", 2024-09-13T09:12:42.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528604+0000", 2024-09-13T09:12:42.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528604+0000", 2024-09-13T09:12:42.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528606+0000", 2024-09-13T09:12:42.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.661e-06 2024-09-13T09:12:42.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528612+0000", 2024-09-13T09:12:42.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8300000000000001e-06 2024-09-13T09:12:42.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528617+0000", 2024-09-13T09:12:42.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5369999999999999e-06 2024-09-13T09:12:42.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535282+0000", 2024-09-13T09:12:42.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0066651619999999996 2024-09-13T09:12:42.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.929 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535288+0000", 2024-09-13T09:12:42.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1249999999999998e-06 2024-09-13T09:12:42.929 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.929 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.929 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.929 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20205 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.929 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538621+0000", 2024-09-13T09:12:42.929 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32616835999999999, 2024-09-13T09:12:42.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32654768299999998, 2024-09-13T09:12:42.929 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.929 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.929 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.930 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.930 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.930 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20205 2024-09-13T09:12:42.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.930 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538621+0000", 2024-09-13T09:12:42.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538621+0000", 2024-09-13T09:12:42.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538620+0000", 2024-09-13T09:12:42.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538623+0000", 2024-09-13T09:12:42.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:12:42.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538629+0000", 2024-09-13T09:12:42.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0070000000000003e-06 2024-09-13T09:12:42.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538634+0000", 2024-09-13T09:12:42.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8500000000000002e-06 2024-09-13T09:12:42.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539125+0000", 2024-09-13T09:12:42.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049132600000000005 2024-09-13T09:12:42.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539131+0000", 2024-09-13T09:12:42.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4670000000000001e-06 2024-09-13T09:12:42.933 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.933 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.933 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.933 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20316 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.933 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540641+0000", 2024-09-13T09:12:42.933 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.324148361, 2024-09-13T09:12:42.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.324560923, 2024-09-13T09:12:42.934 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.934 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.934 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.934 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.934 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.934 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20316 2024-09-13T09:12:42.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.934 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540641+0000", 2024-09-13T09:12:42.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540641+0000", 2024-09-13T09:12:42.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540640+0000", 2024-09-13T09:12:42.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540643+0000", 2024-09-13T09:12:42.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:12:42.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540648+0000", 2024-09-13T09:12:42.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5489999999999997e-06 2024-09-13T09:12:42.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:12:42.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9580000000000001e-06 2024-09-13T09:12:42.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541046+0000", 2024-09-13T09:12:42.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000393581 2024-09-13T09:12:42.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541054+0000", 2024-09-13T09:12:42.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:12:42.937 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.937 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.937 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.937 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20459 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.937 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543648+0000", 2024-09-13T09:12:42.938 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.321141648, 2024-09-13T09:12:42.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32158703199999999, 2024-09-13T09:12:42.938 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.938 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.938 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.938 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.938 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.938 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20459 2024-09-13T09:12:42.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.938 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543648+0000", 2024-09-13T09:12:42.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543648+0000", 2024-09-13T09:12:42.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543647+0000", 2024-09-13T09:12:42.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543656+0000", 2024-09-13T09:12:42.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7919999999999998e-06 2024-09-13T09:12:42.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543664+0000", 2024-09-13T09:12:42.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8180000000000004e-06 2024-09-13T09:12:42.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543669+0000", 2024-09-13T09:12:42.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.023e-06 2024-09-13T09:12:42.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.941 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544181+0000", 2024-09-13T09:12:42.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051223900000000005 2024-09-13T09:12:42.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.941 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544187+0000", 2024-09-13T09:12:42.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4980000000000003e-06 2024-09-13T09:12:42.941 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.941 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.941 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.942 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20570 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.942 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547255+0000", 2024-09-13T09:12:42.942 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31753367900000001, 2024-09-13T09:12:42.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31801274299999999, 2024-09-13T09:12:42.942 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.942 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.942 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.942 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.942 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.942 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20570 2024-09-13T09:12:42.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.942 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547255+0000", 2024-09-13T09:12:42.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547255+0000", 2024-09-13T09:12:42.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547255+0000", 2024-09-13T09:12:42.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547263+0000", 2024-09-13T09:12:42.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2490000000000001e-06 2024-09-13T09:12:42.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547271+0000", 2024-09-13T09:12:42.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5070000000000003e-06 2024-09-13T09:12:42.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547283+0000", 2024-09-13T09:12:42.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2191999999999999e-05 2024-09-13T09:12:42.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547343+0000", 2024-09-13T09:12:42.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9769999999999999e-05 2024-09-13T09:12:42.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547351+0000", 2024-09-13T09:12:42.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9489999999999996e-06 2024-09-13T09:12:42.945 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.945 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.945 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.946 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20606 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.946 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.548260+0000", 2024-09-13T09:12:42.946 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31652955199999999, 2024-09-13T09:12:42.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31704444399999998, 2024-09-13T09:12:42.946 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.946 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.946 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.946 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.946 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.946 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20606 2024-09-13T09:12:42.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.947 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548260+0000", 2024-09-13T09:12:42.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548260+0000", 2024-09-13T09:12:42.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548259+0000", 2024-09-13T09:12:42.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548262+0000", 2024-09-13T09:12:42.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:12:42.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548268+0000", 2024-09-13T09:12:42.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0209999999999996e-06 2024-09-13T09:12:42.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548272+0000", 2024-09-13T09:12:42.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:12:42.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548703+0000", 2024-09-13T09:12:42.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00043091000000000001 2024-09-13T09:12:42.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548710+0000", 2024-09-13T09:12:42.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2540000000000003e-06 2024-09-13T09:12:42.950 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.950 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.950 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.950 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20721 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.950 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549912+0000", 2024-09-13T09:12:42.950 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31487673100000002, 2024-09-13T09:12:42.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31542468200000001, 2024-09-13T09:12:42.950 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.950 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.950 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.950 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.951 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.951 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20721 2024-09-13T09:12:42.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.951 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549912+0000", 2024-09-13T09:12:42.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549912+0000", 2024-09-13T09:12:42.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549912+0000", 2024-09-13T09:12:42.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549921+0000", 2024-09-13T09:12:42.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8899999999999996e-06 2024-09-13T09:12:42.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549928+0000", 2024-09-13T09:12:42.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3560000000000004e-06 2024-09-13T09:12:42.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549933+0000", 2024-09-13T09:12:42.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.152e-06 2024-09-13T09:12:42.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550025+0000", 2024-09-13T09:12:42.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1559999999999998e-05 2024-09-13T09:12:42.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550033+0000", 2024-09-13T09:12:42.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0090000000000007e-06 2024-09-13T09:12:42.954 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.954 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.954 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.955 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20836 3.12 3.8d2c5fb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:42.955 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.552914+0000", 2024-09-13T09:12:42.955 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31187563000000001, 2024-09-13T09:12:42.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31245711300000001, 2024-09-13T09:12:42.955 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.955 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.955 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.955 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.955 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.955 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20836 2024-09-13T09:12:42.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.956 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552914+0000", 2024-09-13T09:12:42.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552914+0000", 2024-09-13T09:12:42.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552913+0000", 2024-09-13T09:12:42.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552916+0000", 2024-09-13T09:12:42.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1370000000000002e-06 2024-09-13T09:12:42.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552924+0000", 2024-09-13T09:12:42.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4800000000000004e-06 2024-09-13T09:12:42.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552929+0000", 2024-09-13T09:12:42.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5160000000000002e-06 2024-09-13T09:12:42.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552959+0000", 2024-09-13T09:12:42.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9876e-05 2024-09-13T09:12:42.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552964+0000", 2024-09-13T09:12:42.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:12:42.959 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.959 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.959 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.959 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19954 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.959 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.525107+0000", 2024-09-13T09:12:42.959 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33968208599999999, 2024-09-13T09:12:42.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34029680800000001, 2024-09-13T09:12:42.959 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.959 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.959 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.960 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.960 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.960 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19954 2024-09-13T09:12:42.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.960 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525107+0000", 2024-09-13T09:12:42.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525107+0000", 2024-09-13T09:12:42.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525107+0000", 2024-09-13T09:12:42.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525109+0000", 2024-09-13T09:12:42.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8779999999999998e-06 2024-09-13T09:12:42.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525119+0000", 2024-09-13T09:12:42.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1349999999999998e-06 2024-09-13T09:12:42.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525124+0000", 2024-09-13T09:12:42.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9109999999999998e-06 2024-09-13T09:12:42.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534753+0000", 2024-09-13T09:12:42.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096283389999999996 2024-09-13T09:12:42.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534760+0000", 2024-09-13T09:12:42.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7290000000000003e-06 2024-09-13T09:12:42.964 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.964 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.964 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.964 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20064 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.964 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528621+0000", 2024-09-13T09:12:42.964 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33616833499999998, 2024-09-13T09:12:42.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.336816069, 2024-09-13T09:12:42.964 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.964 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.964 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.965 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.965 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.965 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20064 2024-09-13T09:12:42.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.965 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528621+0000", 2024-09-13T09:12:42.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528621+0000", 2024-09-13T09:12:42.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528621+0000", 2024-09-13T09:12:42.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528628+0000", 2024-09-13T09:12:42.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:12:42.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528635+0000", 2024-09-13T09:12:42.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6769999999999998e-06 2024-09-13T09:12:42.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528641+0000", 2024-09-13T09:12:42.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8590000000000001e-06 2024-09-13T09:12:42.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535308+0000", 2024-09-13T09:12:42.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0066669440000000002 2024-09-13T09:12:42.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535313+0000", 2024-09-13T09:12:42.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7529999999999998e-06 2024-09-13T09:12:42.968 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.968 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.968 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.968 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20206 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.968 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538638+0000", 2024-09-13T09:12:42.968 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32615112000000002, 2024-09-13T09:12:42.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32683193399999999, 2024-09-13T09:12:42.969 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.969 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.969 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.969 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.969 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.969 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20206 2024-09-13T09:12:42.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.969 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538638+0000", 2024-09-13T09:12:42.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538638+0000", 2024-09-13T09:12:42.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538638+0000", 2024-09-13T09:12:42.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538640+0000", 2024-09-13T09:12:42.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.244e-06 2024-09-13T09:12:42.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538646+0000", 2024-09-13T09:12:42.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9150000000000001e-06 2024-09-13T09:12:42.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538651+0000", 2024-09-13T09:12:42.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8099999999999997e-06 2024-09-13T09:12:42.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539151+0000", 2024-09-13T09:12:42.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00050013200000000003 2024-09-13T09:12:42.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539157+0000", 2024-09-13T09:12:42.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:12:42.972 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.972 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.972 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.973 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20317 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.973 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540657+0000", 2024-09-13T09:12:42.973 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32413200199999997, 2024-09-13T09:12:42.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32484575399999999, 2024-09-13T09:12:42.973 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.973 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.973 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.973 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.973 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.973 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20317 2024-09-13T09:12:42.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.973 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540657+0000", 2024-09-13T09:12:42.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540657+0000", 2024-09-13T09:12:42.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540657+0000", 2024-09-13T09:12:42.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540659+0000", 2024-09-13T09:12:42.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.193e-06 2024-09-13T09:12:42.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540666+0000", 2024-09-13T09:12:42.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0809999999999996e-06 2024-09-13T09:12:42.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540670+0000", 2024-09-13T09:12:42.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.878e-06 2024-09-13T09:12:42.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541085+0000", 2024-09-13T09:12:42.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00041507300000000002 2024-09-13T09:12:42.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541093+0000", 2024-09-13T09:12:42.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9619999999999993e-06 2024-09-13T09:12:42.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.977 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.977 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20460 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.977 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543676+0000", 2024-09-13T09:12:42.977 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32111308999999999, 2024-09-13T09:12:42.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32185978999999998, 2024-09-13T09:12:42.978 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.978 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.978 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20460 2024-09-13T09:12:42.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.978 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543676+0000", 2024-09-13T09:12:42.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543676+0000", 2024-09-13T09:12:42.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543676+0000", 2024-09-13T09:12:42.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543678+0000", 2024-09-13T09:12:42.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4229999999999999e-06 2024-09-13T09:12:42.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543685+0000", 2024-09-13T09:12:42.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.404e-06 2024-09-13T09:12:42.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543691+0000", 2024-09-13T09:12:42.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5320000000000003e-06 2024-09-13T09:12:42.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544214+0000", 2024-09-13T09:12:42.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052324499999999998 2024-09-13T09:12:42.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544221+0000", 2024-09-13T09:12:42.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5769999999999999e-06 2024-09-13T09:12:42.982 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.982 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.982 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.983 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20571 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.983 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547289+0000", 2024-09-13T09:12:42.983 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31749977899999998, 2024-09-13T09:12:42.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31827998800000001, 2024-09-13T09:12:42.983 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.983 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.983 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.983 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.983 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.983 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20571 2024-09-13T09:12:42.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.983 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547289+0000", 2024-09-13T09:12:42.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547289+0000", 2024-09-13T09:12:42.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547289+0000", 2024-09-13T09:12:42.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547291+0000", 2024-09-13T09:12:42.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:12:42.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547298+0000", 2024-09-13T09:12:42.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4509999999999997e-06 2024-09-13T09:12:42.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547303+0000", 2024-09-13T09:12:42.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6460000000000001e-06 2024-09-13T09:12:42.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547385+0000", 2024-09-13T09:12:42.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2392000000000007e-05 2024-09-13T09:12:42.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547389+0000", 2024-09-13T09:12:42.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9099999999999998e-06 2024-09-13T09:12:42.987 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.987 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.987 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.987 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20722 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.988 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549947+0000", 2024-09-13T09:12:42.988 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31484206999999997, 2024-09-13T09:12:42.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31565586000000001, 2024-09-13T09:12:42.988 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.988 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.988 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.988 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.988 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.988 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20722 2024-09-13T09:12:42.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.989 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549947+0000", 2024-09-13T09:12:42.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549947+0000", 2024-09-13T09:12:42.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549947+0000", 2024-09-13T09:12:42.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549949+0000", 2024-09-13T09:12:42.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:12:42.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549956+0000", 2024-09-13T09:12:42.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9779999999999999e-06 2024-09-13T09:12:42.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549961+0000", 2024-09-13T09:12:42.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5669999999999996e-06 2024-09-13T09:12:42.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550059+0000", 2024-09-13T09:12:42.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8034000000000004e-05 2024-09-13T09:12:42.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550067+0000", 2024-09-13T09:12:42.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:12:42.992 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.992 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.992 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.992 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20837 3.12 3.8d2c5fb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:42.992 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.552939+0000", 2024-09-13T09:12:42.992 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31184983199999999, 2024-09-13T09:12:42.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31269666299999999, 2024-09-13T09:12:42.992 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.992 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.992 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.992 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.993 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:42.993 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20837 2024-09-13T09:12:42.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.993 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:42.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:42.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552939+0000", 2024-09-13T09:12:42.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:42.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552939+0000", 2024-09-13T09:12:42.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:42.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:42.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552939+0000", 2024-09-13T09:12:42.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:42.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:42.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552941+0000", 2024-09-13T09:12:42.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.514e-06 2024-09-13T09:12:42.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:42.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552948+0000", 2024-09-13T09:12:42.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5220000000000004e-06 2024-09-13T09:12:42.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:42.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552952+0000", 2024-09-13T09:12:42.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1980000000000003e-06 2024-09-13T09:12:42.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:42.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552986+0000", 2024-09-13T09:12:42.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4384000000000001e-05 2024-09-13T09:12:42.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:42.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552995+0000", 2024-09-13T09:12:42.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2490000000000001e-06 2024-09-13T09:12:42.998 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.998 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:42.998 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:42.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:42.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:42.998 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19955 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:42.999 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.525129+0000", 2024-09-13T09:12:42.999 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33965972, 2024-09-13T09:12:42.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.340539486, 2024-09-13T09:12:42.999 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:42.999 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:42.999 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:42.999 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:42.999 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.000 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19955 2024-09-13T09:12:43.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.000 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525129+0000", 2024-09-13T09:12:43.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525129+0000", 2024-09-13T09:12:43.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525129+0000", 2024-09-13T09:12:43.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525131+0000", 2024-09-13T09:12:43.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:12:43.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525138+0000", 2024-09-13T09:12:43.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5520000000000001e-06 2024-09-13T09:12:43.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525143+0000", 2024-09-13T09:12:43.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:12:43.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534783+0000", 2024-09-13T09:12:43.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096404669999999998 2024-09-13T09:12:43.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534790+0000", 2024-09-13T09:12:43.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3430000000000001e-06 2024-09-13T09:12:43.005 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.005 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.006 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.006 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20065 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.006 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528645+0000", 2024-09-13T09:12:43.006 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33614382799999998, 2024-09-13T09:12:43.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33705633600000001, 2024-09-13T09:12:43.006 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.006 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.006 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.006 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.006 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.006 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20065 2024-09-13T09:12:43.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.007 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528645+0000", 2024-09-13T09:12:43.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528645+0000", 2024-09-13T09:12:43.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528645+0000", 2024-09-13T09:12:43.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528647+0000", 2024-09-13T09:12:43.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.356e-06 2024-09-13T09:12:43.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528654+0000", 2024-09-13T09:12:43.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4280000000000001e-06 2024-09-13T09:12:43.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528658+0000", 2024-09-13T09:12:43.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6759999999999998e-06 2024-09-13T09:12:43.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535335+0000", 2024-09-13T09:12:43.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0066768319999999997 2024-09-13T09:12:43.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535341+0000", 2024-09-13T09:12:43.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.913e-06 2024-09-13T09:12:43.010 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.010 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.010 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.011 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20207 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.011 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538655+0000", 2024-09-13T09:12:43.011 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32613426099999998, 2024-09-13T09:12:43.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32707941099999999, 2024-09-13T09:12:43.011 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.011 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.011 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.011 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.011 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.011 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20207 2024-09-13T09:12:43.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.011 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538655+0000", 2024-09-13T09:12:43.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538655+0000", 2024-09-13T09:12:43.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538655+0000", 2024-09-13T09:12:43.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538669+0000", 2024-09-13T09:12:43.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4324e-05 2024-09-13T09:12:43.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538681+0000", 2024-09-13T09:12:43.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.166e-05 2024-09-13T09:12:43.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538685+0000", 2024-09-13T09:12:43.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5240000000000002e-06 2024-09-13T09:12:43.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539178+0000", 2024-09-13T09:12:43.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049283200000000001 2024-09-13T09:12:43.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539185+0000", 2024-09-13T09:12:43.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1280000000000002e-06 2024-09-13T09:12:43.014 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.014 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.015 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.015 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20318 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.015 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540674+0000", 2024-09-13T09:12:43.015 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32411501100000001, 2024-09-13T09:12:43.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32509813799999998, 2024-09-13T09:12:43.015 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.015 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.015 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.015 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.015 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.016 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20318 2024-09-13T09:12:43.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.016 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540674+0000", 2024-09-13T09:12:43.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540674+0000", 2024-09-13T09:12:43.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540674+0000", 2024-09-13T09:12:43.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540676+0000", 2024-09-13T09:12:43.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3549999999999999e-06 2024-09-13T09:12:43.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.018 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540681+0000", 2024-09-13T09:12:43.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0420000000000002e-06 2024-09-13T09:12:43.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.018 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.018 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540685+0000", 2024-09-13T09:12:43.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.048e-06 2024-09-13T09:12:43.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.018 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.018 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541119+0000", 2024-09-13T09:12:43.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000434184 2024-09-13T09:12:43.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541126+0000", 2024-09-13T09:12:43.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0430000000000002e-06 2024-09-13T09:12:43.019 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.019 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.019 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.019 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20461 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.019 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543759+0000", 2024-09-13T09:12:43.019 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32102985499999998, 2024-09-13T09:12:43.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32204683899999997, 2024-09-13T09:12:43.020 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.020 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.020 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.020 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.020 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.020 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20461 2024-09-13T09:12:43.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.020 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543759+0000", 2024-09-13T09:12:43.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543759+0000", 2024-09-13T09:12:43.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543759+0000", 2024-09-13T09:12:43.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543762+0000", 2024-09-13T09:12:43.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4910000000000001e-06 2024-09-13T09:12:43.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543770+0000", 2024-09-13T09:12:43.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8979999999999996e-06 2024-09-13T09:12:43.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543776+0000", 2024-09-13T09:12:43.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.648e-06 2024-09-13T09:12:43.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544350+0000", 2024-09-13T09:12:43.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00057366299999999997 2024-09-13T09:12:43.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544358+0000", 2024-09-13T09:12:43.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2570000000000007e-06 2024-09-13T09:12:43.024 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.024 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.024 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.024 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20572 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.024 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547307+0000", 2024-09-13T09:12:43.024 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.317482402, 2024-09-13T09:12:43.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31853991199999998, 2024-09-13T09:12:43.025 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.025 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.025 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.025 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.025 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.025 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20572 2024-09-13T09:12:43.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.025 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547307+0000", 2024-09-13T09:12:43.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547307+0000", 2024-09-13T09:12:43.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547306+0000", 2024-09-13T09:12:43.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547308+0000", 2024-09-13T09:12:43.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0660000000000002e-06 2024-09-13T09:12:43.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547315+0000", 2024-09-13T09:12:43.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0689999999999998e-06 2024-09-13T09:12:43.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547319+0000", 2024-09-13T09:12:43.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5739999999999997e-06 2024-09-13T09:12:43.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547428+0000", 2024-09-13T09:12:43.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000109202 2024-09-13T09:12:43.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547437+0000", 2024-09-13T09:12:43.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8210000000000008e-06 2024-09-13T09:12:43.028 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.028 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.028 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.029 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20723 3.12 3.62ceb1b2 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.029 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549982+0000", 2024-09-13T09:12:43.029 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31480750800000001, 2024-09-13T09:12:43.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.315898815, 2024-09-13T09:12:43.029 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.029 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.029 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.029 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.030 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.030 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20723 2024-09-13T09:12:43.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.030 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549982+0000", 2024-09-13T09:12:43.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549982+0000", 2024-09-13T09:12:43.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549981+0000", 2024-09-13T09:12:43.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549989+0000", 2024-09-13T09:12:43.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0080000000000002e-06 2024-09-13T09:12:43.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549996+0000", 2024-09-13T09:12:43.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9829999999999999e-06 2024-09-13T09:12:43.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550001+0000", 2024-09-13T09:12:43.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0939999999999998e-06 2024-09-13T09:12:43.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550094+0000", 2024-09-13T09:12:43.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2656000000000002e-05 2024-09-13T09:12:43.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550102+0000", 2024-09-13T09:12:43.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8690000000000004e-06 2024-09-13T09:12:43.033 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.033 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.033 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.033 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20838 3.12 3.8d2c5fb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:43.033 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.552957+0000", 2024-09-13T09:12:43.033 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31183229400000001, 2024-09-13T09:12:43.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31296151300000002, 2024-09-13T09:12:43.034 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.034 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.034 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.034 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.034 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.034 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20838 2024-09-13T09:12:43.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.034 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552957+0000", 2024-09-13T09:12:43.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552957+0000", 2024-09-13T09:12:43.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552956+0000", 2024-09-13T09:12:43.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.036 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.036 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552959+0000", 2024-09-13T09:12:43.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:12:43.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.036 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.036 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552966+0000", 2024-09-13T09:12:43.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4390000000000003e-06 2024-09-13T09:12:43.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.036 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.036 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552971+0000", 2024-09-13T09:12:43.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8609999999999997e-06 2024-09-13T09:12:43.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.553019+0000", 2024-09-13T09:12:43.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8028999999999997e-05 2024-09-13T09:12:43.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.553023+0000", 2024-09-13T09:12:43.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.659e-06 2024-09-13T09:12:43.037 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.037 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.038 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.038 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19956 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.038 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.525147+0000", 2024-09-13T09:12:43.038 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.339641888, 2024-09-13T09:12:43.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34080445199999998, 2024-09-13T09:12:43.038 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.038 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.038 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.038 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19956 2024-09-13T09:12:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525147+0000", 2024-09-13T09:12:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525147+0000", 2024-09-13T09:12:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525147+0000", 2024-09-13T09:12:43.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525149+0000", 2024-09-13T09:12:43.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2699999999999999e-06 2024-09-13T09:12:43.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525155+0000", 2024-09-13T09:12:43.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4949999999999999e-06 2024-09-13T09:12:43.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525159+0000", 2024-09-13T09:12:43.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8590000000000004e-06 2024-09-13T09:12:43.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534823+0000", 2024-09-13T09:12:43.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096632009999999997 2024-09-13T09:12:43.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534830+0000", 2024-09-13T09:12:43.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:12:43.042 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.042 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.042 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.042 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20066 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.042 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528675+0000", 2024-09-13T09:12:43.042 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33611407199999999, 2024-09-13T09:12:43.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33731129900000001, 2024-09-13T09:12:43.042 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.042 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.042 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.043 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20066 2024-09-13T09:12:43.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.043 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528675+0000", 2024-09-13T09:12:43.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528675+0000", 2024-09-13T09:12:43.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528675+0000", 2024-09-13T09:12:43.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528677+0000", 2024-09-13T09:12:43.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:12:43.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528685+0000", 2024-09-13T09:12:43.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1590000000000004e-06 2024-09-13T09:12:43.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528691+0000", 2024-09-13T09:12:43.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1659999999999999e-06 2024-09-13T09:12:43.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535363+0000", 2024-09-13T09:12:43.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0066719090000000002 2024-09-13T09:12:43.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535369+0000", 2024-09-13T09:12:43.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1929999999999998e-06 2024-09-13T09:12:43.046 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.046 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.046 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.047 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20208 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.047 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538690+0000", 2024-09-13T09:12:43.047 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32609909599999998, 2024-09-13T09:12:43.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.327330809, 2024-09-13T09:12:43.047 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.047 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.047 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.047 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.047 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.047 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20208 2024-09-13T09:12:43.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.047 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538690+0000", 2024-09-13T09:12:43.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538690+0000", 2024-09-13T09:12:43.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538689+0000", 2024-09-13T09:12:43.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538692+0000", 2024-09-13T09:12:43.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:12:43.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538700+0000", 2024-09-13T09:12:43.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0350000000000001e-06 2024-09-13T09:12:43.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538706+0000", 2024-09-13T09:12:43.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0950000000000001e-06 2024-09-13T09:12:43.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539206+0000", 2024-09-13T09:12:43.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049959099999999997 2024-09-13T09:12:43.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539213+0000", 2024-09-13T09:12:43.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1749999999999999e-06 2024-09-13T09:12:43.050 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.050 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.050 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.051 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20319 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.051 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540689+0000", 2024-09-13T09:12:43.051 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32409992500000001, 2024-09-13T09:12:43.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32536476199999997, 2024-09-13T09:12:43.051 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.051 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.051 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.051 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.051 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.052 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20319 2024-09-13T09:12:43.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.052 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540689+0000", 2024-09-13T09:12:43.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540689+0000", 2024-09-13T09:12:43.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540689+0000", 2024-09-13T09:12:43.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540691+0000", 2024-09-13T09:12:43.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9999999999999999e-06 2024-09-13T09:12:43.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540696+0000", 2024-09-13T09:12:43.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0000000000000004e-06 2024-09-13T09:12:43.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540700+0000", 2024-09-13T09:12:43.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9709999999999998e-06 2024-09-13T09:12:43.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541221+0000", 2024-09-13T09:12:43.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000521325 2024-09-13T09:12:43.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541231+0000", 2024-09-13T09:12:43.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3670000000000003e-06 2024-09-13T09:12:43.055 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.055 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.055 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.055 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20462 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.055 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543910+0000", 2024-09-13T09:12:43.056 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32087895100000002, 2024-09-13T09:12:43.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.322178566, 2024-09-13T09:12:43.056 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.056 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.056 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.056 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.056 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.056 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20462 2024-09-13T09:12:43.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.056 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543910+0000", 2024-09-13T09:12:43.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543910+0000", 2024-09-13T09:12:43.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543910+0000", 2024-09-13T09:12:43.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543913+0000", 2024-09-13T09:12:43.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5279999999999999e-06 2024-09-13T09:12:43.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543921+0000", 2024-09-13T09:12:43.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3520000000000001e-06 2024-09-13T09:12:43.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543926+0000", 2024-09-13T09:12:43.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7699999999999998e-06 2024-09-13T09:12:43.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544436+0000", 2024-09-13T09:12:43.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00050996099999999996 2024-09-13T09:12:43.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544441+0000", 2024-09-13T09:12:43.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0189999999999997e-06 2024-09-13T09:12:43.059 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.059 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.059 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.060 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20573 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.060 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547340+0000", 2024-09-13T09:12:43.060 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31744903200000002, 2024-09-13T09:12:43.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.318781645, 2024-09-13T09:12:43.060 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.060 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.060 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.060 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.060 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.060 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20573 2024-09-13T09:12:43.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.060 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.061 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547340+0000", 2024-09-13T09:12:43.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.061 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547340+0000", 2024-09-13T09:12:43.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547340+0000", 2024-09-13T09:12:43.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547342+0000", 2024-09-13T09:12:43.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.667e-06 2024-09-13T09:12:43.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547349+0000", 2024-09-13T09:12:43.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9879999999999998e-06 2024-09-13T09:12:43.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547354+0000", 2024-09-13T09:12:43.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1830000000000002e-06 2024-09-13T09:12:43.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547473+0000", 2024-09-13T09:12:43.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000118126 2024-09-13T09:12:43.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547480+0000", 2024-09-13T09:12:43.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6229999999999997e-06 2024-09-13T09:12:43.064 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.064 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.064 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.064 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20724 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.064 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550019+0000", 2024-09-13T09:12:43.064 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.314769772, 2024-09-13T09:12:43.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31613968999999997, 2024-09-13T09:12:43.065 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.065 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.065 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.065 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.065 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.065 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20724 2024-09-13T09:12:43.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.065 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550019+0000", 2024-09-13T09:12:43.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550019+0000", 2024-09-13T09:12:43.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550019+0000", 2024-09-13T09:12:43.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550022+0000", 2024-09-13T09:12:43.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5459999999999998e-06 2024-09-13T09:12:43.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550028+0000", 2024-09-13T09:12:43.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4099999999999996e-06 2024-09-13T09:12:43.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550043+0000", 2024-09-13T09:12:43.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5007999999999999e-05 2024-09-13T09:12:43.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550126+0000", 2024-09-13T09:12:43.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3524999999999997e-05 2024-09-13T09:12:43.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550134+0000", 2024-09-13T09:12:43.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9510000000000006e-06 2024-09-13T09:12:43.069 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.069 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.069 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.069 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20839 3.12 3.8d2c5fb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:43.069 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.552976+0000", 2024-09-13T09:12:43.069 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.311813546, 2024-09-13T09:12:43.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31321664399999999, 2024-09-13T09:12:43.070 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.070 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.070 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.070 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.070 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.070 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20839 2024-09-13T09:12:43.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.070 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552976+0000", 2024-09-13T09:12:43.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552976+0000", 2024-09-13T09:12:43.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552975+0000", 2024-09-13T09:12:43.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552984+0000", 2024-09-13T09:12:43.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4290000000000001e-06 2024-09-13T09:12:43.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552990+0000", 2024-09-13T09:12:43.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.579e-06 2024-09-13T09:12:43.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.552995+0000", 2024-09-13T09:12:43.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7520000000000004e-06 2024-09-13T09:12:43.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.073 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.553046+0000", 2024-09-13T09:12:43.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0566999999999999e-05 2024-09-13T09:12:43.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.073 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.553052+0000", 2024-09-13T09:12:43.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8179999999999998e-06 2024-09-13T09:12:43.073 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.073 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.073 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.074 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19957 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.074 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.525164+0000", 2024-09-13T09:12:43.074 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33962562400000001, 2024-09-13T09:12:43.074 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34106372499999998, 2024-09-13T09:12:43.074 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.074 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.074 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.074 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.074 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.074 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19957 2024-09-13T09:12:43.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.075 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.075 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.075 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525164+0000", 2024-09-13T09:12:43.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.075 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.075 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525164+0000", 2024-09-13T09:12:43.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.075 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.076 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525163+0000", 2024-09-13T09:12:43.076 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.076 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.076 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525171+0000", 2024-09-13T09:12:43.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8290000000000008e-06 2024-09-13T09:12:43.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525178+0000", 2024-09-13T09:12:43.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6359999999999997e-06 2024-09-13T09:12:43.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525183+0000", 2024-09-13T09:12:43.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0189999999999997e-06 2024-09-13T09:12:43.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534857+0000", 2024-09-13T09:12:43.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096742459999999992 2024-09-13T09:12:43.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534864+0000", 2024-09-13T09:12:43.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9099999999999999e-06 2024-09-13T09:12:43.078 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.078 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.078 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.078 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20067 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.078 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528695+0000", 2024-09-13T09:12:43.079 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33609417000000003, 2024-09-13T09:12:43.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33756684300000001, 2024-09-13T09:12:43.079 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.079 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.079 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.079 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.079 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.079 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20067 2024-09-13T09:12:43.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.079 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528695+0000", 2024-09-13T09:12:43.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.080 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.080 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528695+0000", 2024-09-13T09:12:43.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.080 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.080 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528695+0000", 2024-09-13T09:12:43.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.081 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.081 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528697+0000", 2024-09-13T09:12:43.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0209999999999998e-06 2024-09-13T09:12:43.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.081 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.081 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528702+0000", 2024-09-13T09:12:43.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3859999999999998e-06 2024-09-13T09:12:43.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.082 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.082 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528706+0000", 2024-09-13T09:12:43.082 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9979999999999997e-06 2024-09-13T09:12:43.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.082 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.082 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535394+0000", 2024-09-13T09:12:43.082 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0066879269999999998 2024-09-13T09:12:43.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.082 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.083 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535401+0000", 2024-09-13T09:12:43.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0469999999999996e-06 2024-09-13T09:12:43.083 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.083 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.083 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.083 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20209 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.083 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538712+0000", 2024-09-13T09:12:43.083 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32607670799999999, 2024-09-13T09:12:43.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32758264399999998, 2024-09-13T09:12:43.083 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.083 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.084 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.084 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.084 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.084 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20209 2024-09-13T09:12:43.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.084 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538712+0000", 2024-09-13T09:12:43.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538712+0000", 2024-09-13T09:12:43.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538712+0000", 2024-09-13T09:12:43.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538714+0000", 2024-09-13T09:12:43.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5959999999999998e-06 2024-09-13T09:12:43.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538720+0000", 2024-09-13T09:12:43.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9649999999999996e-06 2024-09-13T09:12:43.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.087 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538725+0000", 2024-09-13T09:12:43.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4279999999999998e-06 2024-09-13T09:12:43.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.087 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.087 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539234+0000", 2024-09-13T09:12:43.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00050931499999999998 2024-09-13T09:12:43.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.087 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.087 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539241+0000", 2024-09-13T09:12:43.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9650000000000002e-06 2024-09-13T09:12:43.088 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.088 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.088 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.088 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20320 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.088 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540704+0000", 2024-09-13T09:12:43.088 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32408541099999999, 2024-09-13T09:12:43.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32562616300000002, 2024-09-13T09:12:43.089 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.089 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.089 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.089 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.089 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.089 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20320 2024-09-13T09:12:43.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.089 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540704+0000", 2024-09-13T09:12:43.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540704+0000", 2024-09-13T09:12:43.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540703+0000", 2024-09-13T09:12:43.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540711+0000", 2024-09-13T09:12:43.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9910000000000002e-06 2024-09-13T09:12:43.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540717+0000", 2024-09-13T09:12:43.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7699999999999998e-06 2024-09-13T09:12:43.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540721+0000", 2024-09-13T09:12:43.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0709999999999996e-06 2024-09-13T09:12:43.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541482+0000", 2024-09-13T09:12:43.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00076119100000000004 2024-09-13T09:12:43.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.092 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.092 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541491+0000", 2024-09-13T09:12:43.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1770000000000004e-06 2024-09-13T09:12:43.092 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.092 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.092 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.092 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20463 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.092 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543932+0000", 2024-09-13T09:12:43.092 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.320856851, 2024-09-13T09:12:43.093 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.322430736, 2024-09-13T09:12:43.093 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.093 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.093 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.093 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.093 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.093 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20463 2024-09-13T09:12:43.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.093 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.093 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543932+0000", 2024-09-13T09:12:43.093 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543932+0000", 2024-09-13T09:12:43.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543932+0000", 2024-09-13T09:12:43.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543936+0000", 2024-09-13T09:12:43.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.563e-06 2024-09-13T09:12:43.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543942+0000", 2024-09-13T09:12:43.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3099999999999997e-06 2024-09-13T09:12:43.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543947+0000", 2024-09-13T09:12:43.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7870000000000001e-06 2024-09-13T09:12:43.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544474+0000", 2024-09-13T09:12:43.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052717599999999997 2024-09-13T09:12:43.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544478+0000", 2024-09-13T09:12:43.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2699999999999998e-06 2024-09-13T09:12:43.096 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.096 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.096 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.096 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20574 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.096 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547387+0000", 2024-09-13T09:12:43.096 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31740174500000001, 2024-09-13T09:12:43.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31900997399999997, 2024-09-13T09:12:43.097 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.097 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.097 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.097 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.097 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.097 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20574 2024-09-13T09:12:43.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.097 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.098 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547387+0000", 2024-09-13T09:12:43.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.098 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.098 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547387+0000", 2024-09-13T09:12:43.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.098 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.098 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547387+0000", 2024-09-13T09:12:43.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.098 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.099 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547390+0000", 2024-09-13T09:12:43.099 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:12:43.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.099 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.099 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547398+0000", 2024-09-13T09:12:43.099 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5079999999999999e-06 2024-09-13T09:12:43.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.099 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.099 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547403+0000", 2024-09-13T09:12:43.099 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5040000000000002e-06 2024-09-13T09:12:43.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.099 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547540+0000", 2024-09-13T09:12:43.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013657900000000001 2024-09-13T09:12:43.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.100 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547548+0000", 2024-09-13T09:12:43.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3990000000000004e-06 2024-09-13T09:12:43.100 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.100 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.100 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.100 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20726 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.100 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550056+0000", 2024-09-13T09:12:43.100 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31473288399999999, 2024-09-13T09:12:43.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31637422999999998, 2024-09-13T09:12:43.101 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.101 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.101 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.101 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.101 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.101 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20726 2024-09-13T09:12:43.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.101 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550056+0000", 2024-09-13T09:12:43.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550056+0000", 2024-09-13T09:12:43.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550056+0000", 2024-09-13T09:12:43.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550058+0000", 2024-09-13T09:12:43.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4789999999999999e-06 2024-09-13T09:12:43.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.103 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550065+0000", 2024-09-13T09:12:43.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0350000000000004e-06 2024-09-13T09:12:43.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.103 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550070+0000", 2024-09-13T09:12:43.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8749999999999999e-06 2024-09-13T09:12:43.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.103 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550279+0000", 2024-09-13T09:12:43.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00020908400000000001 2024-09-13T09:12:43.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.104 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.104 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550288+0000", 2024-09-13T09:12:43.104 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8380000000000008e-06 2024-09-13T09:12:43.104 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.104 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.104 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.104 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19958 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.104 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.525188+0000", 2024-09-13T09:12:43.104 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33960136400000002, 2024-09-13T09:12:43.104 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34127729499999998, 2024-09-13T09:12:43.105 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.105 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.105 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.105 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.105 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.105 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19958 2024-09-13T09:12:43.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.105 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.105 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.105 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525188+0000", 2024-09-13T09:12:43.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525188+0000", 2024-09-13T09:12:43.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525187+0000", 2024-09-13T09:12:43.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525190+0000", 2024-09-13T09:12:43.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3889999999999999e-06 2024-09-13T09:12:43.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525195+0000", 2024-09-13T09:12:43.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8139999999999997e-06 2024-09-13T09:12:43.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525200+0000", 2024-09-13T09:12:43.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3830000000000002e-06 2024-09-13T09:12:43.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534889+0000", 2024-09-13T09:12:43.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096889009999999998 2024-09-13T09:12:43.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534896+0000", 2024-09-13T09:12:43.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.029e-06 2024-09-13T09:12:43.108 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.108 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.108 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.108 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20068 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.109 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528710+0000", 2024-09-13T09:12:43.109 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33607959199999998, 2024-09-13T09:12:43.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.337790231, 2024-09-13T09:12:43.109 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.109 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.109 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.109 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.109 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.109 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20068 2024-09-13T09:12:43.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.109 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.110 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.110 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528710+0000", 2024-09-13T09:12:43.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.110 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.110 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528710+0000", 2024-09-13T09:12:43.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.110 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.110 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528709+0000", 2024-09-13T09:12:43.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528711+0000", 2024-09-13T09:12:43.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.751e-06 2024-09-13T09:12:43.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528716+0000", 2024-09-13T09:12:43.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:12:43.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528720+0000", 2024-09-13T09:12:43.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:12:43.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.112 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535505+0000", 2024-09-13T09:12:43.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0067845800000000001 2024-09-13T09:12:43.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.112 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535513+0000", 2024-09-13T09:12:43.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1259999999999998e-06 2024-09-13T09:12:43.112 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.112 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.112 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.113 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20210 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.113 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538729+0000", 2024-09-13T09:12:43.113 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32606010800000002, 2024-09-13T09:12:43.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32780379300000001, 2024-09-13T09:12:43.113 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.113 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.113 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.113 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.113 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.113 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20210 2024-09-13T09:12:43.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.113 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538729+0000", 2024-09-13T09:12:43.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538729+0000", 2024-09-13T09:12:43.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538729+0000", 2024-09-13T09:12:43.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538731+0000", 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1050000000000002e-06 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538737+0000", 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8699999999999997e-06 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538741+0000", 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4989999999999996e-06 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.116 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539263+0000", 2024-09-13T09:12:43.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052172999999999998 2024-09-13T09:12:43.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.116 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.116 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539269+0000", 2024-09-13T09:12:43.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:12:43.116 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.116 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.116 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.116 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20321 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.116 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540725+0000", 2024-09-13T09:12:43.117 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32406400299999999, 2024-09-13T09:12:43.117 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32584238199999999, 2024-09-13T09:12:43.117 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.117 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.117 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20321 2024-09-13T09:12:43.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.117 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.117 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.117 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540725+0000", 2024-09-13T09:12:43.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540725+0000", 2024-09-13T09:12:43.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540725+0000", 2024-09-13T09:12:43.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540727+0000", 2024-09-13T09:12:43.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:12:43.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540732+0000", 2024-09-13T09:12:43.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.22e-06 2024-09-13T09:12:43.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540737+0000", 2024-09-13T09:12:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7249999999999997e-06 2024-09-13T09:12:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541536+0000", 2024-09-13T09:12:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00079878299999999996 2024-09-13T09:12:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541545+0000", 2024-09-13T09:12:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7289999999999995e-06 2024-09-13T09:12:43.120 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.121 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.121 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.121 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20464 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.121 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543951+0000", 2024-09-13T09:12:43.121 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32083827100000001, 2024-09-13T09:12:43.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.322650033, 2024-09-13T09:12:43.121 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.121 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.121 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.121 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.121 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.121 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20464 2024-09-13T09:12:43.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.122 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543951+0000", 2024-09-13T09:12:43.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543951+0000", 2024-09-13T09:12:43.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543950+0000", 2024-09-13T09:12:43.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543953+0000", 2024-09-13T09:12:43.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.278e-06 2024-09-13T09:12:43.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543959+0000", 2024-09-13T09:12:43.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:12:43.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.124 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543964+0000", 2024-09-13T09:12:43.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7049999999999998e-06 2024-09-13T09:12:43.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.124 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.124 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544504+0000", 2024-09-13T09:12:43.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053976299999999996 2024-09-13T09:12:43.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.124 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.124 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544511+0000", 2024-09-13T09:12:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0500000000000003e-06 2024-09-13T09:12:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20575 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547421+0000", 2024-09-13T09:12:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31736787399999999, 2024-09-13T09:12:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31921436199999997, 2024-09-13T09:12:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.125 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20575 2024-09-13T09:12:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547421+0000", 2024-09-13T09:12:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547421+0000", 2024-09-13T09:12:43.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547421+0000", 2024-09-13T09:12:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547424+0000", 2024-09-13T09:12:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7599999999999998e-06 2024-09-13T09:12:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547432+0000", 2024-09-13T09:12:43.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2190000000000004e-06 2024-09-13T09:12:43.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547437+0000", 2024-09-13T09:12:43.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9470000000000002e-06 2024-09-13T09:12:43.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547589+0000", 2024-09-13T09:12:43.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015235300000000001 2024-09-13T09:12:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547598+0000", 2024-09-13T09:12:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8789999999999992e-06 2024-09-13T09:12:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20727 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550235+0000", 2024-09-13T09:12:43.129 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31455459200000002, 2024-09-13T09:12:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31643438000000002, 2024-09-13T09:12:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20727 2024-09-13T09:12:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550235+0000", 2024-09-13T09:12:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550235+0000", 2024-09-13T09:12:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550234+0000", 2024-09-13T09:12:43.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550238+0000", 2024-09-13T09:12:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1670000000000001e-06 2024-09-13T09:12:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550252+0000", 2024-09-13T09:12:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3570000000000001e-05 2024-09-13T09:12:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550258+0000", 2024-09-13T09:12:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5570000000000001e-06 2024-09-13T09:12:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550318+0000", 2024-09-13T09:12:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9410999999999998e-05 2024-09-13T09:12:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550326+0000", 2024-09-13T09:12:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1240000000000005e-06 2024-09-13T09:12:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19959 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.525204+0000", 2024-09-13T09:12:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33958529399999998, 2024-09-13T09:12:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34149981400000001, 2024-09-13T09:12:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19959 2024-09-13T09:12:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525204+0000", 2024-09-13T09:12:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525204+0000", 2024-09-13T09:12:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525204+0000", 2024-09-13T09:12:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525206+0000", 2024-09-13T09:12:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1260000000000001e-06 2024-09-13T09:12:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525212+0000", 2024-09-13T09:12:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1090000000000003e-06 2024-09-13T09:12:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525216+0000", 2024-09-13T09:12:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:12:43.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534926+0000", 2024-09-13T09:12:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0097095540000000004 2024-09-13T09:12:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534933+0000", 2024-09-13T09:12:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3339999999999996e-06 2024-09-13T09:12:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20069 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528724+0000", 2024-09-13T09:12:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33606549200000002, 2024-09-13T09:12:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33802547399999999, 2024-09-13T09:12:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.138 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20069 2024-09-13T09:12:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528724+0000", 2024-09-13T09:12:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528724+0000", 2024-09-13T09:12:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528723+0000", 2024-09-13T09:12:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528725+0000", 2024-09-13T09:12:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.728e-06 2024-09-13T09:12:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528730+0000", 2024-09-13T09:12:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8500000000000002e-06 2024-09-13T09:12:43.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528734+0000", 2024-09-13T09:12:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0489999999999997e-06 2024-09-13T09:12:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535543+0000", 2024-09-13T09:12:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068086680000000004 2024-09-13T09:12:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535550+0000", 2024-09-13T09:12:43.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1709999999999996e-06 2024-09-13T09:12:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20211 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538745+0000", 2024-09-13T09:12:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32604372300000001, 2024-09-13T09:12:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32803686500000001, 2024-09-13T09:12:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.142 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20211 2024-09-13T09:12:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538745+0000", 2024-09-13T09:12:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538745+0000", 2024-09-13T09:12:43.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538745+0000", 2024-09-13T09:12:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538747+0000", 2024-09-13T09:12:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:12:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538753+0000", 2024-09-13T09:12:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7359999999999998e-06 2024-09-13T09:12:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538757+0000", 2024-09-13T09:12:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1810000000000003e-06 2024-09-13T09:12:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539290+0000", 2024-09-13T09:12:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000532535 2024-09-13T09:12:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539297+0000", 2024-09-13T09:12:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5510000000000002e-06 2024-09-13T09:12:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20322 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540741+0000", 2024-09-13T09:12:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.324048375, 2024-09-13T09:12:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.326078755, 2024-09-13T09:12:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.146 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20322 2024-09-13T09:12:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540741+0000", 2024-09-13T09:12:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.147 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540741+0000", 2024-09-13T09:12:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540740+0000", 2024-09-13T09:12:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540742+0000", 2024-09-13T09:12:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.951e-06 2024-09-13T09:12:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540752+0000", 2024-09-13T09:12:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6099999999999995e-06 2024-09-13T09:12:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540756+0000", 2024-09-13T09:12:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4429999999999996e-06 2024-09-13T09:12:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541576+0000", 2024-09-13T09:12:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00081988700000000005 2024-09-13T09:12:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541583+0000", 2024-09-13T09:12:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1060000000000002e-06 2024-09-13T09:12:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.150 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20465 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543968+0000", 2024-09-13T09:12:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32082103200000001, 2024-09-13T09:12:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.322884964, 2024-09-13T09:12:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20465 2024-09-13T09:12:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543968+0000", 2024-09-13T09:12:43.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543968+0000", 2024-09-13T09:12:43.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543968+0000", 2024-09-13T09:12:43.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543970+0000", 2024-09-13T09:12:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2400000000000002e-06 2024-09-13T09:12:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543976+0000", 2024-09-13T09:12:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7089999999999999e-06 2024-09-13T09:12:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543980+0000", 2024-09-13T09:12:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2239999999999997e-06 2024-09-13T09:12:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544542+0000", 2024-09-13T09:12:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056191599999999998 2024-09-13T09:12:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544549+0000", 2024-09-13T09:12:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9700000000000002e-06 2024-09-13T09:12:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20576 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547441+0000", 2024-09-13T09:12:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31734772999999999, 2024-09-13T09:12:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31944502000000002, 2024-09-13T09:12:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20576 2024-09-13T09:12:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.155 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547441+0000", 2024-09-13T09:12:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547441+0000", 2024-09-13T09:12:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547441+0000", 2024-09-13T09:12:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547443+0000", 2024-09-13T09:12:43.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4329999999999998e-06 2024-09-13T09:12:43.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547449+0000", 2024-09-13T09:12:43.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8680000000000004e-06 2024-09-13T09:12:43.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547454+0000", 2024-09-13T09:12:43.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4279999999999998e-06 2024-09-13T09:12:43.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547627+0000", 2024-09-13T09:12:43.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017373099999999999 2024-09-13T09:12:43.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547635+0000", 2024-09-13T09:12:43.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5769999999999996e-06 2024-09-13T09:12:43.158 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.158 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.158 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.158 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20728 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.159 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550275+0000", 2024-09-13T09:12:43.159 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31451401600000001, 2024-09-13T09:12:43.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31664439999999999, 2024-09-13T09:12:43.159 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.159 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.159 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.159 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.159 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.159 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20728 2024-09-13T09:12:43.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.159 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550275+0000", 2024-09-13T09:12:43.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550275+0000", 2024-09-13T09:12:43.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550275+0000", 2024-09-13T09:12:43.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550277+0000", 2024-09-13T09:12:43.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7819999999999998e-06 2024-09-13T09:12:43.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550285+0000", 2024-09-13T09:12:43.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5700000000000004e-06 2024-09-13T09:12:43.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550290+0000", 2024-09-13T09:12:43.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9099999999999996e-06 2024-09-13T09:12:43.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550457+0000", 2024-09-13T09:12:43.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000167097 2024-09-13T09:12:43.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550466+0000", 2024-09-13T09:12:43.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6840000000000002e-06 2024-09-13T09:12:43.163 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.163 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.163 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.163 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19928 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.163 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524295+0000", 2024-09-13T09:12:43.163 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34049387199999998, 2024-09-13T09:12:43.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34265801099999998, 2024-09-13T09:12:43.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.163 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.164 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.164 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.164 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.164 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19928 2024-09-13T09:12:43.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.164 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524295+0000", 2024-09-13T09:12:43.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524295+0000", 2024-09-13T09:12:43.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524295+0000", 2024-09-13T09:12:43.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524297+0000", 2024-09-13T09:12:43.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1950000000000002e-06 2024-09-13T09:12:43.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524303+0000", 2024-09-13T09:12:43.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.631e-06 2024-09-13T09:12:43.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524309+0000", 2024-09-13T09:12:43.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4080000000000003e-06 2024-09-13T09:12:43.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533773+0000", 2024-09-13T09:12:43.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0094637290000000006 2024-09-13T09:12:43.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533781+0000", 2024-09-13T09:12:43.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1000000000000004e-06 2024-09-13T09:12:43.167 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.167 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.167 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.167 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19960 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.167 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.525223+0000", 2024-09-13T09:12:43.168 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.339565907, 2024-09-13T09:12:43.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34176300399999998, 2024-09-13T09:12:43.168 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.168 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.168 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.168 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.168 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.168 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19960 2024-09-13T09:12:43.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.168 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525223+0000", 2024-09-13T09:12:43.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525223+0000", 2024-09-13T09:12:43.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525223+0000", 2024-09-13T09:12:43.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525225+0000", 2024-09-13T09:12:43.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:12:43.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525231+0000", 2024-09-13T09:12:43.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1430000000000003e-06 2024-09-13T09:12:43.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525244+0000", 2024-09-13T09:12:43.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3193000000000001e-05 2024-09-13T09:12:43.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534960+0000", 2024-09-13T09:12:43.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0097158699999999997 2024-09-13T09:12:43.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534968+0000", 2024-09-13T09:12:43.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:12:43.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.172 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.172 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20070 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.172 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528738+0000", 2024-09-13T09:12:43.172 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33605158899999998, 2024-09-13T09:12:43.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33828131, 2024-09-13T09:12:43.172 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.172 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.172 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.172 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.173 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.173 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20070 2024-09-13T09:12:43.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.173 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528738+0000", 2024-09-13T09:12:43.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528738+0000", 2024-09-13T09:12:43.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528737+0000", 2024-09-13T09:12:43.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528739+0000", 2024-09-13T09:12:43.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.846e-06 2024-09-13T09:12:43.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528744+0000", 2024-09-13T09:12:43.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0980000000000001e-06 2024-09-13T09:12:43.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528748+0000", 2024-09-13T09:12:43.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.738e-06 2024-09-13T09:12:43.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535576+0000", 2024-09-13T09:12:43.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068284319999999997 2024-09-13T09:12:43.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535586+0000", 2024-09-13T09:12:43.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7389999999999994e-06 2024-09-13T09:12:43.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.176 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.176 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20212 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.176 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538776+0000", 2024-09-13T09:12:43.176 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32601330699999997, 2024-09-13T09:12:43.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32827620800000001, 2024-09-13T09:12:43.176 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.177 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.177 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.177 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.177 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.177 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20212 2024-09-13T09:12:43.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.177 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538776+0000", 2024-09-13T09:12:43.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538776+0000", 2024-09-13T09:12:43.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538775+0000", 2024-09-13T09:12:43.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538778+0000", 2024-09-13T09:12:43.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6469999999999998e-06 2024-09-13T09:12:43.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538784+0000", 2024-09-13T09:12:43.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:12:43.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538789+0000", 2024-09-13T09:12:43.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7129999999999996e-06 2024-09-13T09:12:43.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539321+0000", 2024-09-13T09:12:43.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053185799999999996 2024-09-13T09:12:43.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539328+0000", 2024-09-13T09:12:43.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.691e-06 2024-09-13T09:12:43.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.180 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.181 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20323 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.181 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540760+0000", 2024-09-13T09:12:43.181 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32402878099999999, 2024-09-13T09:12:43.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32632491699999999, 2024-09-13T09:12:43.181 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.181 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.181 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.181 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.181 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.182 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20323 2024-09-13T09:12:43.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.182 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540760+0000", 2024-09-13T09:12:43.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540760+0000", 2024-09-13T09:12:43.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540760+0000", 2024-09-13T09:12:43.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540762+0000", 2024-09-13T09:12:43.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.99e-06 2024-09-13T09:12:43.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540770+0000", 2024-09-13T09:12:43.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7829999999999999e-06 2024-09-13T09:12:43.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540787+0000", 2024-09-13T09:12:43.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6779e-05 2024-09-13T09:12:43.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541608+0000", 2024-09-13T09:12:43.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00082095699999999998 2024-09-13T09:12:43.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541614+0000", 2024-09-13T09:12:43.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8199999999999999e-06 2024-09-13T09:12:43.185 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.185 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.185 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.185 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20466 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.185 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543984+0000", 2024-09-13T09:12:43.186 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32080509099999999, 2024-09-13T09:12:43.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.323134586, 2024-09-13T09:12:43.186 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.186 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.186 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.186 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.186 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.186 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20466 2024-09-13T09:12:43.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.186 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.187 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543984+0000", 2024-09-13T09:12:43.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.187 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543984+0000", 2024-09-13T09:12:43.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543984+0000", 2024-09-13T09:12:43.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543986+0000", 2024-09-13T09:12:43.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2390000000000001e-06 2024-09-13T09:12:43.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543992+0000", 2024-09-13T09:12:43.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9699999999999996e-06 2024-09-13T09:12:43.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543996+0000", 2024-09-13T09:12:43.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5170000000000001e-06 2024-09-13T09:12:43.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544577+0000", 2024-09-13T09:12:43.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00058062399999999996 2024-09-13T09:12:43.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544581+0000", 2024-09-13T09:12:43.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8519999999999997e-06 2024-09-13T09:12:43.190 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.190 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.190 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.191 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20577 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.191 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547469+0000", 2024-09-13T09:12:43.191 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31732052500000002, 2024-09-13T09:12:43.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31968333399999999, 2024-09-13T09:12:43.191 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.191 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.191 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.191 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.191 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.191 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20577 2024-09-13T09:12:43.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.191 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547469+0000", 2024-09-13T09:12:43.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547469+0000", 2024-09-13T09:12:43.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547468+0000", 2024-09-13T09:12:43.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:43.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547471+0000", 2024-09-13T09:12:43.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:12:43.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547478+0000", 2024-09-13T09:12:43.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6480000000000003e-06 2024-09-13T09:12:43.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547483+0000", 2024-09-13T09:12:43.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:12:43.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547665+0000", 2024-09-13T09:12:43.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018202800000000001 2024-09-13T09:12:43.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547673+0000", 2024-09-13T09:12:43.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5000000000000002e-06 2024-09-13T09:12:43.195 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.195 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.195 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.195 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20729 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.195 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550366+0000", 2024-09-13T09:12:43.195 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31442280700000003, 2024-09-13T09:12:43.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.316818867, 2024-09-13T09:12:43.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.195 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.195 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.195 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.196 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.196 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20729 2024-09-13T09:12:43.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.196 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550366+0000", 2024-09-13T09:12:43.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550366+0000", 2024-09-13T09:12:43.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550366+0000", 2024-09-13T09:12:43.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550370+0000", 2024-09-13T09:12:43.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8210000000000003e-06 2024-09-13T09:12:43.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550379+0000", 2024-09-13T09:12:43.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4550000000000001e-06 2024-09-13T09:12:43.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550386+0000", 2024-09-13T09:12:43.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6340000000000003e-06 2024-09-13T09:12:43.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550492+0000", 2024-09-13T09:12:43.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010589700000000001 2024-09-13T09:12:43.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550499+0000", 2024-09-13T09:12:43.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1690000000000003e-06 2024-09-13T09:12:43.199 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.199 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.199 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.199 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19929 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.199 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524314+0000", 2024-09-13T09:12:43.199 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34047550500000001, 2024-09-13T09:12:43.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34290500000000002, 2024-09-13T09:12:43.199 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.199 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.199 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.199 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.200 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.200 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19929 2024-09-13T09:12:43.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.200 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524314+0000", 2024-09-13T09:12:43.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524314+0000", 2024-09-13T09:12:43.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524313+0000", 2024-09-13T09:12:43.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524315+0000", 2024-09-13T09:12:43.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0760000000000001e-06 2024-09-13T09:12:43.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524325+0000", 2024-09-13T09:12:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8300000000000008e-06 2024-09-13T09:12:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524331+0000", 2024-09-13T09:12:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:12:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533807+0000", 2024-09-13T09:12:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0094757820000000003 2024-09-13T09:12:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533821+0000", 2024-09-13T09:12:43.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4844999999999999e-05 2024-09-13T09:12:43.203 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.203 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.203 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.203 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19961 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.203 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.525249+0000", 2024-09-13T09:12:43.203 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33953987099999999, 2024-09-13T09:12:43.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34200225400000001, 2024-09-13T09:12:43.203 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.203 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.203 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.203 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.204 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.204 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19961 2024-09-13T09:12:43.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.204 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525249+0000", 2024-09-13T09:12:43.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525249+0000", 2024-09-13T09:12:43.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525249+0000", 2024-09-13T09:12:43.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525251+0000", 2024-09-13T09:12:43.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5229999999999998e-06 2024-09-13T09:12:43.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525258+0000", 2024-09-13T09:12:43.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8299999999999998e-06 2024-09-13T09:12:43.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525263+0000", 2024-09-13T09:12:43.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3229999999999997e-06 2024-09-13T09:12:43.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534994+0000", 2024-09-13T09:12:43.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0097307829999999998 2024-09-13T09:12:43.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535001+0000", 2024-09-13T09:12:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8850000000000001e-06 2024-09-13T09:12:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20071 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528751+0000", 2024-09-13T09:12:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33603782799999998, 2024-09-13T09:12:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33853323200000002, 2024-09-13T09:12:43.207 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.208 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.208 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.208 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.208 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.208 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20071 2024-09-13T09:12:43.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.208 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528751+0000", 2024-09-13T09:12:43.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528751+0000", 2024-09-13T09:12:43.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528751+0000", 2024-09-13T09:12:43.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528754+0000", 2024-09-13T09:12:43.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.57e-06 2024-09-13T09:12:43.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528759+0000", 2024-09-13T09:12:43.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1669999999999998e-06 2024-09-13T09:12:43.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528762+0000", 2024-09-13T09:12:43.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4920000000000002e-06 2024-09-13T09:12:43.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.211 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535612+0000", 2024-09-13T09:12:43.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068502479999999998 2024-09-13T09:12:43.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.211 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.211 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535619+0000", 2024-09-13T09:12:43.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.99e-06 2024-09-13T09:12:43.211 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.211 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.211 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.211 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.211 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32598217400000001, 2024-09-13T09:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32851107000000002, 2024-09-13T09:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.213 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:12:43.213 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.213 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:12:43.213 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.213 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:12:43.213 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:12:43.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:12:43.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:12:43.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:12:43.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:12:43.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:12:43.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:12:43.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.215 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.215 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:12:43.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:12:43.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.215 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.215 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20324 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.215 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540791+0000", 2024-09-13T09:12:43.215 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.323997904, 2024-09-13T09:12:43.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32655979000000002, 2024-09-13T09:12:43.216 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.216 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.216 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.216 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.216 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.216 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20324 2024-09-13T09:12:43.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.216 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540791+0000", 2024-09-13T09:12:43.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.217 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.217 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540791+0000", 2024-09-13T09:12:43.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.217 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.217 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540791+0000", 2024-09-13T09:12:43.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.217 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.217 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540793+0000", 2024-09-13T09:12:43.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:12:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540816+0000", 2024-09-13T09:12:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2376999999999999e-05 2024-09-13T09:12:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540821+0000", 2024-09-13T09:12:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3650000000000003e-06 2024-09-13T09:12:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541639+0000", 2024-09-13T09:12:43.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00081842500000000001 2024-09-13T09:12:43.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541646+0000", 2024-09-13T09:12:43.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7139999999999996e-06 2024-09-13T09:12:43.219 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.219 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.219 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.220 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20467 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.220 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.544000+0000", 2024-09-13T09:12:43.220 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32078929499999997, 2024-09-13T09:12:43.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32338530100000001, 2024-09-13T09:12:43.220 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.220 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.220 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.220 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.220 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.220 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20467 2024-09-13T09:12:43.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.221 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544000+0000", 2024-09-13T09:12:43.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544000+0000", 2024-09-13T09:12:43.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544000+0000", 2024-09-13T09:12:43.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.222 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544001+0000", 2024-09-13T09:12:43.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8050000000000001e-06 2024-09-13T09:12:43.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.222 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544007+0000", 2024-09-13T09:12:43.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8649999999999998e-06 2024-09-13T09:12:43.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.222 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544012+0000", 2024-09-13T09:12:43.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2490000000000003e-06 2024-09-13T09:12:43.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.223 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544616+0000", 2024-09-13T09:12:43.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00060456300000000002 2024-09-13T09:12:43.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.223 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544622+0000", 2024-09-13T09:12:43.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.519e-06 2024-09-13T09:12:43.223 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.223 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.224 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.224 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20578 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.224 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547501+0000", 2024-09-13T09:12:43.224 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31728829400000003, 2024-09-13T09:12:43.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31991740600000002, 2024-09-13T09:12:43.224 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.224 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.224 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.225 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.225 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.225 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20578 2024-09-13T09:12:43.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.225 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547501+0000", 2024-09-13T09:12:43.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547501+0000", 2024-09-13T09:12:43.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547500+0000", 2024-09-13T09:12:43.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:43.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547503+0000", 2024-09-13T09:12:43.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8739999999999999e-06 2024-09-13T09:12:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547509+0000", 2024-09-13T09:12:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3409999999999999e-06 2024-09-13T09:12:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547514+0000", 2024-09-13T09:12:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5990000000000004e-06 2024-09-13T09:12:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547703+0000", 2024-09-13T09:12:43.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018915699999999999 2024-09-13T09:12:43.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.228 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.228 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:12:43.228 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.515e-06 2024-09-13T09:12:43.228 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.228 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.228 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.228 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20730 3.12 3.8d2c5fb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.228 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550392+0000", 2024-09-13T09:12:43.229 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31439761500000002, 2024-09-13T09:12:43.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31706021699999998, 2024-09-13T09:12:43.229 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.229 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.229 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.229 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.229 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.229 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20730 2024-09-13T09:12:43.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.229 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.229 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.229 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550392+0000", 2024-09-13T09:12:43.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.230 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.230 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550392+0000", 2024-09-13T09:12:43.230 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.230 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550391+0000", 2024-09-13T09:12:43.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550394+0000", 2024-09-13T09:12:43.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4870000000000001e-06 2024-09-13T09:12:43.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550508+0000", 2024-09-13T09:12:43.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000114305 2024-09-13T09:12:43.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.232 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.232 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550514+0000", 2024-09-13T09:12:43.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4710000000000004e-06 2024-09-13T09:12:43.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.232 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.232 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550838+0000", 2024-09-13T09:12:43.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00032345200000000001 2024-09-13T09:12:43.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.232 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.233 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550845+0000", 2024-09-13T09:12:43.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5089999999999996e-06 2024-09-13T09:12:43.233 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.233 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.233 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.233 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19930 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.233 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524335+0000", 2024-09-13T09:12:43.233 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34045373200000001, 2024-09-13T09:12:43.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34314959099999998, 2024-09-13T09:12:43.233 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.233 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.233 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.234 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.234 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.234 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19930 2024-09-13T09:12:43.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.234 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524335+0000", 2024-09-13T09:12:43.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524335+0000", 2024-09-13T09:12:43.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.235 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.235 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524335+0000", 2024-09-13T09:12:43.235 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.235 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.235 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524337+0000", 2024-09-13T09:12:43.235 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:12:43.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.235 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524345+0000", 2024-09-13T09:12:43.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0090000000000007e-06 2024-09-13T09:12:43.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.236 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524351+0000", 2024-09-13T09:12:43.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:12:43.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.236 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533844+0000", 2024-09-13T09:12:43.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0094936680000000002 2024-09-13T09:12:43.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533851+0000", 2024-09-13T09:12:43.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0879999999999997e-06 2024-09-13T09:12:43.237 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.237 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.237 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.237 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19962 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.237 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.525310+0000", 2024-09-13T09:12:43.237 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33947895700000003, 2024-09-13T09:12:43.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34220741999999998, 2024-09-13T09:12:43.238 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.238 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.238 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.238 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.238 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.238 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19962 2024-09-13T09:12:43.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.238 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525310+0000", 2024-09-13T09:12:43.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525310+0000", 2024-09-13T09:12:43.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525310+0000", 2024-09-13T09:12:43.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525313+0000", 2024-09-13T09:12:43.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.399e-06 2024-09-13T09:12:43.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.240 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525322+0000", 2024-09-13T09:12:43.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5019999999999992e-06 2024-09-13T09:12:43.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.240 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525328+0000", 2024-09-13T09:12:43.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0349999999999998e-06 2024-09-13T09:12:43.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535027+0000", 2024-09-13T09:12:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.009699661 2024-09-13T09:12:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535034+0000", 2024-09-13T09:12:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0280000000000003e-06 2024-09-13T09:12:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.241 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20072 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.242 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528766+0000", 2024-09-13T09:12:43.242 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33602341899999999, 2024-09-13T09:12:43.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33878472399999998, 2024-09-13T09:12:43.242 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.242 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.242 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20072 2024-09-13T09:12:43.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.242 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528766+0000", 2024-09-13T09:12:43.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.243 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528766+0000", 2024-09-13T09:12:43.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.243 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528765+0000", 2024-09-13T09:12:43.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528767+0000", 2024-09-13T09:12:43.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7999999999999999e-06 2024-09-13T09:12:43.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528773+0000", 2024-09-13T09:12:43.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3680000000000001e-06 2024-09-13T09:12:43.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528777+0000", 2024-09-13T09:12:43.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8979999999999998e-06 2024-09-13T09:12:43.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.245 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535646+0000", 2024-09-13T09:12:43.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068695589999999999 2024-09-13T09:12:43.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.245 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535654+0000", 2024-09-13T09:12:43.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4719999999999998e-06 2024-09-13T09:12:43.245 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.245 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.245 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.245 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.246 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:12:43.246 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32593059400000002, 2024-09-13T09:12:43.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32872591699999998, 2024-09-13T09:12:43.246 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.246 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.246 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.246 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.246 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.246 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:12:43.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.246 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:12:43.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:12:43.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.248 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:12:43.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:12:43.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.248 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:12:43.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:12:43.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.248 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:12:43.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:12:43.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:12:43.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:12:43.249 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.249 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.249 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.249 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20325 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.250 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540827+0000", 2024-09-13T09:12:43.250 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32396244699999999, 2024-09-13T09:12:43.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32679021699999999, 2024-09-13T09:12:43.250 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.250 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.250 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.250 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.250 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.250 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20325 2024-09-13T09:12:43.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.250 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.250 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.250 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540827+0000", 2024-09-13T09:12:43.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540827+0000", 2024-09-13T09:12:43.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540826+0000", 2024-09-13T09:12:43.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540829+0000", 2024-09-13T09:12:43.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6709999999999999e-06 2024-09-13T09:12:43.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540835+0000", 2024-09-13T09:12:43.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9529999999999996e-06 2024-09-13T09:12:43.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540839+0000", 2024-09-13T09:12:43.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2169999999999996e-06 2024-09-13T09:12:43.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541671+0000", 2024-09-13T09:12:43.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00083167400000000002 2024-09-13T09:12:43.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541678+0000", 2024-09-13T09:12:43.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8449999999999997e-06 2024-09-13T09:12:43.253 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.254 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.254 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.254 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20468 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.254 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.544016+0000", 2024-09-13T09:12:43.254 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32077307900000002, 2024-09-13T09:12:43.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32363351099999998, 2024-09-13T09:12:43.254 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.254 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.254 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.254 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.254 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.254 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20468 2024-09-13T09:12:43.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.255 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544016+0000", 2024-09-13T09:12:43.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544016+0000", 2024-09-13T09:12:43.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544016+0000", 2024-09-13T09:12:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544018+0000", 2024-09-13T09:12:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.588e-06 2024-09-13T09:12:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544024+0000", 2024-09-13T09:12:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1310000000000003e-06 2024-09-13T09:12:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544029+0000", 2024-09-13T09:12:43.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6929999999999998e-06 2024-09-13T09:12:43.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544653+0000", 2024-09-13T09:12:43.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00062436700000000002 2024-09-13T09:12:43.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544658+0000", 2024-09-13T09:12:43.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3409999999999996e-06 2024-09-13T09:12:43.257 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20579 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547534+0000", 2024-09-13T09:12:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31725556999999999, 2024-09-13T09:12:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32014914900000002, 2024-09-13T09:12:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20579 2024-09-13T09:12:43.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.259 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547534+0000", 2024-09-13T09:12:43.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547534+0000", 2024-09-13T09:12:43.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547533+0000", 2024-09-13T09:12:43.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547540+0000", 2024-09-13T09:12:43.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4050000000000003e-06 2024-09-13T09:12:43.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547547+0000", 2024-09-13T09:12:43.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0219999999999999e-06 2024-09-13T09:12:43.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547563+0000", 2024-09-13T09:12:43.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5818999999999999e-05 2024-09-13T09:12:43.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547750+0000", 2024-09-13T09:12:43.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018631999999999999 2024-09-13T09:12:43.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547757+0000", 2024-09-13T09:12:43.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.046e-06 2024-09-13T09:12:43.262 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.262 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.262 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.262 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20731 3.12 3.8d2c5fb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.263 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550531+0000", 2024-09-13T09:12:43.263 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31425839799999999, 2024-09-13T09:12:43.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31718497000000001, 2024-09-13T09:12:43.263 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.263 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.263 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.263 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.263 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.263 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20731 2024-09-13T09:12:43.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.263 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550531+0000", 2024-09-13T09:12:43.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550531+0000", 2024-09-13T09:12:43.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550530+0000", 2024-09-13T09:12:43.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550534+0000", 2024-09-13T09:12:43.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8319999999999999e-06 2024-09-13T09:12:43.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550542+0000", 2024-09-13T09:12:43.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9140000000000008e-06 2024-09-13T09:12:43.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550547+0000", 2024-09-13T09:12:43.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.186e-06 2024-09-13T09:12:43.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550865+0000", 2024-09-13T09:12:43.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00031815800000000003 2024-09-13T09:12:43.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550872+0000", 2024-09-13T09:12:43.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:12:43.266 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.266 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.266 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.266 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19931 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524355+0000", 2024-09-13T09:12:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34043385399999998, 2024-09-13T09:12:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34339335100000001, 2024-09-13T09:12:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19931 2024-09-13T09:12:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524355+0000", 2024-09-13T09:12:43.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.268 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524355+0000", 2024-09-13T09:12:43.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.268 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524355+0000", 2024-09-13T09:12:43.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.268 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524357+0000", 2024-09-13T09:12:43.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3130000000000001e-06 2024-09-13T09:12:43.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524363+0000", 2024-09-13T09:12:43.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8690000000000001e-06 2024-09-13T09:12:43.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524375+0000", 2024-09-13T09:12:43.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1902999999999999e-05 2024-09-13T09:12:43.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534006+0000", 2024-09-13T09:12:43.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096309329999999995 2024-09-13T09:12:43.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534016+0000", 2024-09-13T09:12:43.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:12:43.270 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.270 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.270 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.270 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20073 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.270 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528780+0000", 2024-09-13T09:12:43.271 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33600871199999999, 2024-09-13T09:12:43.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33900122100000002, 2024-09-13T09:12:43.271 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.271 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.271 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.271 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.271 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.271 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20073 2024-09-13T09:12:43.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.271 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528780+0000", 2024-09-13T09:12:43.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528780+0000", 2024-09-13T09:12:43.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.273 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528780+0000", 2024-09-13T09:12:43.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.273 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528782+0000", 2024-09-13T09:12:43.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.238e-06 2024-09-13T09:12:43.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.273 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528788+0000", 2024-09-13T09:12:43.274 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:12:43.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.274 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.274 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528792+0000", 2024-09-13T09:12:43.274 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9140000000000001e-06 2024-09-13T09:12:43.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.274 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.274 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535678+0000", 2024-09-13T09:12:43.274 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068857409999999999 2024-09-13T09:12:43.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.274 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535685+0000", 2024-09-13T09:12:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0319999999999998e-06 2024-09-13T09:12:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:12:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32591431799999998, 2024-09-13T09:12:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32894437500000001, 2024-09-13T09:12:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.275 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.276 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.276 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.276 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.276 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:12:43.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.276 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:12:43.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:12:43.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:12:43.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:12:43.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:12:43.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.278 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.278 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:12:43.278 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:12:43.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.278 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.278 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:12:43.278 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:12:43.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.278 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.278 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:12:43.278 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:12:43.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.279 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.279 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:12:43.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:12:43.279 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.279 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.279 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.279 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20326 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.279 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540843+0000", 2024-09-13T09:12:43.279 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32394652600000001, 2024-09-13T09:12:43.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32701012299999999, 2024-09-13T09:12:43.280 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.280 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.280 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.280 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.280 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.280 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20326 2024-09-13T09:12:43.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.280 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.280 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.280 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540843+0000", 2024-09-13T09:12:43.280 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540843+0000", 2024-09-13T09:12:43.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540842+0000", 2024-09-13T09:12:43.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540844+0000", 2024-09-13T09:12:43.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8369999999999999e-06 2024-09-13T09:12:43.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.282 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.282 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540851+0000", 2024-09-13T09:12:43.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9739999999999996e-06 2024-09-13T09:12:43.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540855+0000", 2024-09-13T09:12:43.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1189999999999999e-06 2024-09-13T09:12:43.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541701+0000", 2024-09-13T09:12:43.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00084557600000000001 2024-09-13T09:12:43.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541708+0000", 2024-09-13T09:12:43.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0360000000000001e-06 2024-09-13T09:12:43.284 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.284 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.284 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.284 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20469 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.284 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.544033+0000", 2024-09-13T09:12:43.284 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32075616099999998, 2024-09-13T09:12:43.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32385271999999998, 2024-09-13T09:12:43.284 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.284 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.284 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.284 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.284 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.285 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20469 2024-09-13T09:12:43.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.285 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544033+0000", 2024-09-13T09:12:43.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544033+0000", 2024-09-13T09:12:43.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.286 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544032+0000", 2024-09-13T09:12:43.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.286 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.286 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544035+0000", 2024-09-13T09:12:43.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:12:43.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.287 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544040+0000", 2024-09-13T09:12:43.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4450000000000004e-06 2024-09-13T09:12:43.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.287 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544046+0000", 2024-09-13T09:12:43.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3279999999999996e-06 2024-09-13T09:12:43.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.287 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544688+0000", 2024-09-13T09:12:43.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00064220200000000003 2024-09-13T09:12:43.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544694+0000", 2024-09-13T09:12:43.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4110000000000001e-06 2024-09-13T09:12:43.288 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.288 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.288 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.288 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20580 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.288 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547569+0000", 2024-09-13T09:12:43.288 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31722055799999999, 2024-09-13T09:12:43.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32035012600000001, 2024-09-13T09:12:43.289 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.289 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.289 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.289 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.289 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.289 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20580 2024-09-13T09:12:43.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.289 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547569+0000", 2024-09-13T09:12:43.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547569+0000", 2024-09-13T09:12:43.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547568+0000", 2024-09-13T09:12:43.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.291 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547571+0000", 2024-09-13T09:12:43.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7e-06 2024-09-13T09:12:43.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.291 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547577+0000", 2024-09-13T09:12:43.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.229e-06 2024-09-13T09:12:43.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.291 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547581+0000", 2024-09-13T09:12:43.292 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.216e-06 2024-09-13T09:12:43.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.292 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.292 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547787+0000", 2024-09-13T09:12:43.292 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00020558100000000001 2024-09-13T09:12:43.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.292 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.292 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547794+0000", 2024-09-13T09:12:43.292 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8070000000000002e-06 2024-09-13T09:12:43.292 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.293 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.293 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.293 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20732 3.12 3.8d2c5fb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.293 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550552+0000", 2024-09-13T09:12:43.293 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31423666900000002, 2024-09-13T09:12:43.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31739929099999997, 2024-09-13T09:12:43.293 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.293 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.293 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.293 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.294 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.294 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20732 2024-09-13T09:12:43.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.294 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550552+0000", 2024-09-13T09:12:43.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550552+0000", 2024-09-13T09:12:43.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550552+0000", 2024-09-13T09:12:43.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550560+0000", 2024-09-13T09:12:43.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.622e-06 2024-09-13T09:12:43.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550566+0000", 2024-09-13T09:12:43.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.579e-06 2024-09-13T09:12:43.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550571+0000", 2024-09-13T09:12:43.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6870000000000002e-06 2024-09-13T09:12:43.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550890+0000", 2024-09-13T09:12:43.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00031888000000000001 2024-09-13T09:12:43.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.297 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.297 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550900+0000", 2024-09-13T09:12:43.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437e-05 2024-09-13T09:12:43.297 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.297 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.297 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.297 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19932 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.297 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524380+0000", 2024-09-13T09:12:43.297 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34040922699999998, 2024-09-13T09:12:43.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34360494400000002, 2024-09-13T09:12:43.297 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.298 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.298 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.298 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.298 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.298 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19932 2024-09-13T09:12:43.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.298 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524380+0000", 2024-09-13T09:12:43.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524380+0000", 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524380+0000", 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524382+0000", 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5550000000000001e-06 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524388+0000", 2024-09-13T09:12:43.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3679999999999998e-06 2024-09-13T09:12:43.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524395+0000", 2024-09-13T09:12:43.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6100000000000002e-06 2024-09-13T09:12:43.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534042+0000", 2024-09-13T09:12:43.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096468760000000004 2024-09-13T09:12:43.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534049+0000", 2024-09-13T09:12:43.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4019999999999996e-06 2024-09-13T09:12:43.301 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.301 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.301 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.301 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20074 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.301 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528796+0000", 2024-09-13T09:12:43.301 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.335993492, 2024-09-13T09:12:43.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33922183500000003, 2024-09-13T09:12:43.301 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.301 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.302 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.302 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.302 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.302 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20074 2024-09-13T09:12:43.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.302 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.302 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.302 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528796+0000", 2024-09-13T09:12:43.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.302 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.302 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528796+0000", 2024-09-13T09:12:43.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528795+0000", 2024-09-13T09:12:43.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528798+0000", 2024-09-13T09:12:43.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3499999999999999e-06 2024-09-13T09:12:43.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528803+0000", 2024-09-13T09:12:43.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:12:43.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528812+0000", 2024-09-13T09:12:43.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1840000000000005e-06 2024-09-13T09:12:43.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535712+0000", 2024-09-13T09:12:43.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068997090000000004 2024-09-13T09:12:43.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535720+0000", 2024-09-13T09:12:43.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:12:43.305 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.305 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.305 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.305 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20184 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.305 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.537970+0000", 2024-09-13T09:12:43.306 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32681899199999997, 2024-09-13T09:12:43.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.330080808, 2024-09-13T09:12:43.306 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.306 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.306 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.306 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.306 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.306 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20184 2024-09-13T09:12:43.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.306 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.306 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.537970+0000", 2024-09-13T09:12:43.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.537970+0000", 2024-09-13T09:12:43.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.537970+0000", 2024-09-13T09:12:43.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.537972+0000", 2024-09-13T09:12:43.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5330000000000001e-06 2024-09-13T09:12:43.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538133+0000", 2024-09-13T09:12:43.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016052699999999999 2024-09-13T09:12:43.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538140+0000", 2024-09-13T09:12:43.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6790000000000005e-06 2024-09-13T09:12:43.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538236+0000", 2024-09-13T09:12:43.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5505000000000001e-05 2024-09-13T09:12:43.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538246+0000", 2024-09-13T09:12:43.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7440000000000002e-06 2024-09-13T09:12:43.309 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.309 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.309 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.310 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.310 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:12:43.310 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32589854400000001, 2024-09-13T09:12:43.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32919363099999999, 2024-09-13T09:12:43.310 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.310 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.310 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.310 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.310 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.310 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:12:43.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.311 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.311 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.311 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:12:43.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.311 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.311 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:12:43.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.311 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:12:43.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:12:43.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:12:43.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:12:43.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:12:43.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:12:43.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:12:43.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:12:43.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:12:43.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:12:43.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:12:43.314 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.314 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.314 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.314 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20327 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.314 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540860+0000", 2024-09-13T09:12:43.314 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32392948399999999, 2024-09-13T09:12:43.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32725828099999998, 2024-09-13T09:12:43.314 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.315 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.315 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.316 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.316 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.316 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20327 2024-09-13T09:12:43.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.316 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540860+0000", 2024-09-13T09:12:43.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540860+0000", 2024-09-13T09:12:43.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540859+0000", 2024-09-13T09:12:43.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540872+0000", 2024-09-13T09:12:43.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2301e-05 2024-09-13T09:12:43.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540881+0000", 2024-09-13T09:12:43.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6260000000000007e-06 2024-09-13T09:12:43.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540886+0000", 2024-09-13T09:12:43.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6179999999999997e-06 2024-09-13T09:12:43.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541737+0000", 2024-09-13T09:12:43.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000851098 2024-09-13T09:12:43.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541744+0000", 2024-09-13T09:12:43.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8950000000000001e-06 2024-09-13T09:12:43.319 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.319 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.319 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.319 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20470 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.319 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.544050+0000", 2024-09-13T09:12:43.319 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32073928099999999, 2024-09-13T09:12:43.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32410132200000003, 2024-09-13T09:12:43.320 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.320 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.320 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.320 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.320 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.320 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20470 2024-09-13T09:12:43.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.320 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544050+0000", 2024-09-13T09:12:43.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544050+0000", 2024-09-13T09:12:43.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544050+0000", 2024-09-13T09:12:43.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544052+0000", 2024-09-13T09:12:43.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.137e-06 2024-09-13T09:12:43.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544057+0000", 2024-09-13T09:12:43.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8239999999999996e-06 2024-09-13T09:12:43.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544062+0000", 2024-09-13T09:12:43.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.481e-06 2024-09-13T09:12:43.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544710+0000", 2024-09-13T09:12:43.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00064758600000000002 2024-09-13T09:12:43.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544716+0000", 2024-09-13T09:12:43.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:12:43.323 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.323 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.324 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.324 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20581 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.324 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547585+0000", 2024-09-13T09:12:43.324 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.317203711, 2024-09-13T09:12:43.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32059896300000001, 2024-09-13T09:12:43.324 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.325 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.325 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.325 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.325 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.325 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20581 2024-09-13T09:12:43.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.325 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547585+0000", 2024-09-13T09:12:43.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547585+0000", 2024-09-13T09:12:43.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547585+0000", 2024-09-13T09:12:43.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547587+0000", 2024-09-13T09:12:43.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2400000000000002e-06 2024-09-13T09:12:43.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547593+0000", 2024-09-13T09:12:43.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4759999999999997e-06 2024-09-13T09:12:43.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547599+0000", 2024-09-13T09:12:43.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3629999999999999e-06 2024-09-13T09:12:43.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547836+0000", 2024-09-13T09:12:43.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000236734 2024-09-13T09:12:43.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547843+0000", 2024-09-13T09:12:43.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:12:43.328 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.328 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.328 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.329 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20733 3.12 3.8d2c5fb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.329 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550576+0000", 2024-09-13T09:12:43.329 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31421323600000001, 2024-09-13T09:12:43.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31764168700000001, 2024-09-13T09:12:43.329 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.329 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.329 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.329 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.329 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.329 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20733 2024-09-13T09:12:43.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.330 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550576+0000", 2024-09-13T09:12:43.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550576+0000", 2024-09-13T09:12:43.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550575+0000", 2024-09-13T09:12:43.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550578+0000", 2024-09-13T09:12:43.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.294e-06 2024-09-13T09:12:43.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550583+0000", 2024-09-13T09:12:43.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1760000000000001e-06 2024-09-13T09:12:43.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550587+0000", 2024-09-13T09:12:43.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4530000000000004e-06 2024-09-13T09:12:43.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550921+0000", 2024-09-13T09:12:43.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033337699999999999 2024-09-13T09:12:43.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550925+0000", 2024-09-13T09:12:43.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9859999999999996e-06 2024-09-13T09:12:43.333 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.333 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.333 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.333 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19933 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.333 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524400+0000", 2024-09-13T09:12:43.333 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34038958499999999, 2024-09-13T09:12:43.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34385087800000003, 2024-09-13T09:12:43.334 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.334 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.334 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.334 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.334 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.334 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19933 2024-09-13T09:12:43.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.334 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524400+0000", 2024-09-13T09:12:43.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524400+0000", 2024-09-13T09:12:43.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524399+0000", 2024-09-13T09:12:43.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524412+0000", 2024-09-13T09:12:43.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3073e-05 2024-09-13T09:12:43.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524419+0000", 2024-09-13T09:12:43.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1180000000000003e-06 2024-09-13T09:12:43.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524432+0000", 2024-09-13T09:12:43.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2163e-05 2024-09-13T09:12:43.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534073+0000", 2024-09-13T09:12:43.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.009641709 2024-09-13T09:12:43.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534080+0000", 2024-09-13T09:12:43.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7859999999999999e-06 2024-09-13T09:12:43.338 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.338 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.338 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.338 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20075 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.338 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528818+0000", 2024-09-13T09:12:43.338 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33597123899999998, 2024-09-13T09:12:43.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33946536199999999, 2024-09-13T09:12:43.338 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.338 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.338 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.338 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.338 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.339 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20075 2024-09-13T09:12:43.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.339 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528818+0000", 2024-09-13T09:12:43.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528818+0000", 2024-09-13T09:12:43.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528818+0000", 2024-09-13T09:12:43.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528820+0000", 2024-09-13T09:12:43.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.481e-06 2024-09-13T09:12:43.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528826+0000", 2024-09-13T09:12:43.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9800000000000003e-06 2024-09-13T09:12:43.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528832+0000", 2024-09-13T09:12:43.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7180000000000001e-06 2024-09-13T09:12:43.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535747+0000", 2024-09-13T09:12:43.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0069152509999999999 2024-09-13T09:12:43.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535754+0000", 2024-09-13T09:12:43.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8759999999999999e-06 2024-09-13T09:12:43.342 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.342 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.342 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.342 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20185 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.343 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538154+0000", 2024-09-13T09:12:43.343 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32663537500000001, 2024-09-13T09:12:43.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33016244500000003, 2024-09-13T09:12:43.343 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.343 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.343 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.343 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.343 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.343 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20185 2024-09-13T09:12:43.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.343 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538154+0000", 2024-09-13T09:12:43.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538154+0000", 2024-09-13T09:12:43.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538153+0000", 2024-09-13T09:12:43.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538157+0000", 2024-09-13T09:12:43.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.014e-06 2024-09-13T09:12:43.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538165+0000", 2024-09-13T09:12:43.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8040000000000002e-06 2024-09-13T09:12:43.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538170+0000", 2024-09-13T09:12:43.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7350000000000004e-06 2024-09-13T09:12:43.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538275+0000", 2024-09-13T09:12:43.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105138 2024-09-13T09:12:43.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538283+0000", 2024-09-13T09:12:43.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1219999999999995e-06 2024-09-13T09:12:43.347 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.347 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.347 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.347 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.347 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:12:43.347 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32588366899999999, 2024-09-13T09:12:43.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32944357600000002, 2024-09-13T09:12:43.347 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.348 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.348 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.348 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.348 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.348 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:12:43.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.348 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:12:43.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:12:43.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:12:43.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:12:43.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:12:43.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:12:43.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:12:43.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:12:43.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:12:43.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:12:43.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:12:43.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:12:43.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:12:43.351 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.351 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.351 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.351 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20328 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.351 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540890+0000", 2024-09-13T09:12:43.351 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32389886000000001, 2024-09-13T09:12:43.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.327491851, 2024-09-13T09:12:43.352 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.352 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.352 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.352 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.352 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.352 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20328 2024-09-13T09:12:43.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.352 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540890+0000", 2024-09-13T09:12:43.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540890+0000", 2024-09-13T09:12:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540890+0000", 2024-09-13T09:12:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540892+0000", 2024-09-13T09:12:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3709999999999998e-06 2024-09-13T09:12:43.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540899+0000", 2024-09-13T09:12:43.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4960000000000001e-06 2024-09-13T09:12:43.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540921+0000", 2024-09-13T09:12:43.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1931999999999999e-05 2024-09-13T09:12:43.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541768+0000", 2024-09-13T09:12:43.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00084786699999999998 2024-09-13T09:12:43.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.355 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541776+0000", 2024-09-13T09:12:43.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.114e-06 2024-09-13T09:12:43.355 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.355 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.355 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.356 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20471 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.356 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.544066+0000", 2024-09-13T09:12:43.356 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32072324200000002, 2024-09-13T09:12:43.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32434941099999998, 2024-09-13T09:12:43.356 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.356 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.356 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.356 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.356 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.356 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20471 2024-09-13T09:12:43.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.356 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.357 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544066+0000", 2024-09-13T09:12:43.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.357 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544066+0000", 2024-09-13T09:12:43.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.358 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544065+0000", 2024-09-13T09:12:43.358 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.358 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.358 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544073+0000", 2024-09-13T09:12:43.358 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0330000000000008e-06 2024-09-13T09:12:43.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.358 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.358 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544081+0000", 2024-09-13T09:12:43.358 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2099999999999996e-06 2024-09-13T09:12:43.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.359 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544085+0000", 2024-09-13T09:12:43.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7180000000000004e-06 2024-09-13T09:12:43.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.359 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544737+0000", 2024-09-13T09:12:43.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00065191000000000001 2024-09-13T09:12:43.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.359 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544745+0000", 2024-09-13T09:12:43.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8350000000000004e-06 2024-09-13T09:12:43.360 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.360 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.360 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.360 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20582 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.360 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547604+0000", 2024-09-13T09:12:43.360 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31718521799999999, 2024-09-13T09:12:43.360 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32084468900000002, 2024-09-13T09:12:43.360 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.360 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.360 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.360 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.361 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.361 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20582 2024-09-13T09:12:43.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.361 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547604+0000", 2024-09-13T09:12:43.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547604+0000", 2024-09-13T09:12:43.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547603+0000", 2024-09-13T09:12:43.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547611+0000", 2024-09-13T09:12:43.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9240000000000007e-06 2024-09-13T09:12:43.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.363 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547618+0000", 2024-09-13T09:12:43.363 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9269999999999999e-06 2024-09-13T09:12:43.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.363 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.363 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547623+0000", 2024-09-13T09:12:43.363 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6419999999999998e-06 2024-09-13T09:12:43.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.363 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.363 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547873+0000", 2024-09-13T09:12:43.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00024962900000000002 2024-09-13T09:12:43.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.364 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.364 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547880+0000", 2024-09-13T09:12:43.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7700000000000001e-06 2024-09-13T09:12:43.364 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.364 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.364 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.364 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20734 3.12 3.8d2c5fb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.364 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550592+0000", 2024-09-13T09:12:43.364 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.314197646, 2024-09-13T09:12:43.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31789028000000003, 2024-09-13T09:12:43.365 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.365 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.365 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.365 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.365 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.365 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20734 2024-09-13T09:12:43.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.365 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.365 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550592+0000", 2024-09-13T09:12:43.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550592+0000", 2024-09-13T09:12:43.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550591+0000", 2024-09-13T09:12:43.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.367 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550603+0000", 2024-09-13T09:12:43.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2194e-05 2024-09-13T09:12:43.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.367 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550612+0000", 2024-09-13T09:12:43.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5050000000000007e-06 2024-09-13T09:12:43.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.367 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550626+0000", 2024-09-13T09:12:43.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.383e-05 2024-09-13T09:12:43.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.368 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.368 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550936+0000", 2024-09-13T09:12:43.368 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00031013200000000002 2024-09-13T09:12:43.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.368 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.368 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550946+0000", 2024-09-13T09:12:43.368 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0314e-05 2024-09-13T09:12:43.368 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.368 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.368 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.369 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19934 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.369 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524437+0000", 2024-09-13T09:12:43.369 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34035184299999999, 2024-09-13T09:12:43.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34407795200000002, 2024-09-13T09:12:43.369 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.369 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.369 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.369 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.369 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.369 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19934 2024-09-13T09:12:43.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.369 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524437+0000", 2024-09-13T09:12:43.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524437+0000", 2024-09-13T09:12:43.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524437+0000", 2024-09-13T09:12:43.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524439+0000", 2024-09-13T09:12:43.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5299999999999999e-06 2024-09-13T09:12:43.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524446+0000", 2024-09-13T09:12:43.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6170000000000003e-06 2024-09-13T09:12:43.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524450+0000", 2024-09-13T09:12:43.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3189999999999996e-06 2024-09-13T09:12:43.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534101+0000", 2024-09-13T09:12:43.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096510640000000009 2024-09-13T09:12:43.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534108+0000", 2024-09-13T09:12:43.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6749999999999996e-06 2024-09-13T09:12:43.373 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.373 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.373 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.373 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20076 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.373 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528836+0000", 2024-09-13T09:12:43.373 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33595301, 2024-09-13T09:12:43.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33971187800000002, 2024-09-13T09:12:43.373 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.373 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.374 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.374 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.374 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.374 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20076 2024-09-13T09:12:43.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.374 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528836+0000", 2024-09-13T09:12:43.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528836+0000", 2024-09-13T09:12:43.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528836+0000", 2024-09-13T09:12:43.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528843+0000", 2024-09-13T09:12:43.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4769999999999997e-06 2024-09-13T09:12:43.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528851+0000", 2024-09-13T09:12:43.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.644e-06 2024-09-13T09:12:43.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528856+0000", 2024-09-13T09:12:43.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:12:43.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535820+0000", 2024-09-13T09:12:43.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0069637170000000003 2024-09-13T09:12:43.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535828+0000", 2024-09-13T09:12:43.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6319999999999997e-06 2024-09-13T09:12:43.377 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.377 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.377 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.378 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20186 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.378 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538174+0000", 2024-09-13T09:12:43.378 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32661485400000001, 2024-09-13T09:12:43.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33040647899999998, 2024-09-13T09:12:43.378 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.378 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.378 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.378 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.378 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.378 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20186 2024-09-13T09:12:43.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.378 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.379 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.379 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538174+0000", 2024-09-13T09:12:43.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.379 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.379 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538174+0000", 2024-09-13T09:12:43.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.379 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.379 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538174+0000", 2024-09-13T09:12:43.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.380 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.380 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538176+0000", 2024-09-13T09:12:43.380 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.419e-06 2024-09-13T09:12:43.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.380 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.380 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538183+0000", 2024-09-13T09:12:43.380 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2960000000000001e-06 2024-09-13T09:12:43.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.380 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.380 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538198+0000", 2024-09-13T09:12:43.380 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4915e-05 2024-09-13T09:12:43.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538311+0000", 2024-09-13T09:12:43.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000112582 2024-09-13T09:12:43.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538319+0000", 2024-09-13T09:12:43.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4260000000000003e-06 2024-09-13T09:12:43.381 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.381 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.382 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.382 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.382 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:12:43.382 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32586620700000002, 2024-09-13T09:12:43.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32971711999999997, 2024-09-13T09:12:43.382 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.382 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.382 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.382 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.382 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.382 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:12:43.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.383 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.383 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:12:43.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.383 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:12:43.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:12:43.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:12:43.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:12:43.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:12:43.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:12:43.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:12:43.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:12:43.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:12:43.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:12:43.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:12:43.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:12:43.386 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.386 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.387 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20329 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.387 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540933+0000", 2024-09-13T09:12:43.387 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32385581800000002, 2024-09-13T09:12:43.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32774036299999998, 2024-09-13T09:12:43.387 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.387 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.387 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.387 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.388 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.388 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20329 2024-09-13T09:12:43.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.388 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.388 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.388 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540933+0000", 2024-09-13T09:12:43.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.388 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.388 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540933+0000", 2024-09-13T09:12:43.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540932+0000", 2024-09-13T09:12:43.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:43.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540937+0000", 2024-09-13T09:12:43.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8359999999999999e-06 2024-09-13T09:12:43.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540948+0000", 2024-09-13T09:12:43.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0356e-05 2024-09-13T09:12:43.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540955+0000", 2024-09-13T09:12:43.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.097e-06 2024-09-13T09:12:43.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541800+0000", 2024-09-13T09:12:43.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00084525300000000002 2024-09-13T09:12:43.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541807+0000", 2024-09-13T09:12:43.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0350000000000004e-06 2024-09-13T09:12:43.391 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.391 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.392 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.392 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20440 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.392 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.542987+0000", 2024-09-13T09:12:43.392 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.321802535, 2024-09-13T09:12:43.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32572047799999998, 2024-09-13T09:12:43.392 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.392 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.392 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.392 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.392 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.392 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20440 2024-09-13T09:12:43.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.393 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.393 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542987+0000", 2024-09-13T09:12:43.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.393 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542987+0000", 2024-09-13T09:12:43.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542986+0000", 2024-09-13T09:12:43.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542988+0000", 2024-09-13T09:12:43.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2639999999999998e-06 2024-09-13T09:12:43.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542997+0000", 2024-09-13T09:12:43.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5599999999999994e-06 2024-09-13T09:12:43.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543001+0000", 2024-09-13T09:12:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:12:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543169+0000", 2024-09-13T09:12:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016782900000000001 2024-09-13T09:12:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543180+0000", 2024-09-13T09:12:43.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0668e-05 2024-09-13T09:12:43.396 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.396 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.396 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.396 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20472 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.396 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.544090+0000", 2024-09-13T09:12:43.396 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32069935399999999, 2024-09-13T09:12:43.396 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32465031100000002, 2024-09-13T09:12:43.396 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.396 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.397 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.397 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.397 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.397 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20472 2024-09-13T09:12:43.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.397 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.397 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544090+0000", 2024-09-13T09:12:43.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.397 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544090+0000", 2024-09-13T09:12:43.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.398 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.398 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544089+0000", 2024-09-13T09:12:43.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.398 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.398 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544092+0000", 2024-09-13T09:12:43.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2529999999999999e-06 2024-09-13T09:12:43.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544098+0000", 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9429999999999997e-06 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544102+0000", 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7060000000000003e-06 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544786+0000", 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00068401499999999995 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544795+0000", 2024-09-13T09:12:43.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6580000000000008e-06 2024-09-13T09:12:43.400 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.400 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.400 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.400 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20583 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.400 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547628+0000", 2024-09-13T09:12:43.400 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31716148700000002, 2024-09-13T09:12:43.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32114556300000002, 2024-09-13T09:12:43.400 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.400 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.400 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.400 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.401 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.401 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20583 2024-09-13T09:12:43.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.401 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.401 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.401 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547628+0000", 2024-09-13T09:12:43.401 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.401 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.401 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547628+0000", 2024-09-13T09:12:43.401 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547627+0000", 2024-09-13T09:12:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547630+0000", 2024-09-13T09:12:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4719999999999998e-06 2024-09-13T09:12:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547636+0000", 2024-09-13T09:12:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1090000000000003e-06 2024-09-13T09:12:43.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547640+0000", 2024-09-13T09:12:43.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3880000000000002e-06 2024-09-13T09:12:43.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547906+0000", 2024-09-13T09:12:43.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000265989 2024-09-13T09:12:43.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547914+0000", 2024-09-13T09:12:43.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4889999999999998e-06 2024-09-13T09:12:43.404 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.404 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.404 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.404 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20735 3.12 3.8d2c5fb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.404 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550632+0000", 2024-09-13T09:12:43.404 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31415742800000002, 2024-09-13T09:12:43.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31817441200000002, 2024-09-13T09:12:43.404 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.404 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.404 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.405 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.405 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.405 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20735 2024-09-13T09:12:43.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.405 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.405 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550632+0000", 2024-09-13T09:12:43.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.405 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550632+0000", 2024-09-13T09:12:43.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550631+0000", 2024-09-13T09:12:43.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550634+0000", 2024-09-13T09:12:43.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:12:43.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550642+0000", 2024-09-13T09:12:43.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4409999999999993e-06 2024-09-13T09:12:43.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550647+0000", 2024-09-13T09:12:43.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8659999999999997e-06 2024-09-13T09:12:43.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550957+0000", 2024-09-13T09:12:43.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00030977000000000001 2024-09-13T09:12:43.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550960+0000", 2024-09-13T09:12:43.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:12:43.408 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.408 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.408 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.408 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19935 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.408 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524454+0000", 2024-09-13T09:12:43.409 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34033466899999998, 2024-09-13T09:12:43.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34438846200000001, 2024-09-13T09:12:43.409 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.409 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.409 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.409 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.409 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.409 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19935 2024-09-13T09:12:43.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.409 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524454+0000", 2024-09-13T09:12:43.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524454+0000", 2024-09-13T09:12:43.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524454+0000", 2024-09-13T09:12:43.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524457+0000", 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4789999999999999e-06 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524463+0000", 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9730000000000002e-06 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524467+0000", 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3189999999999996e-06 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.411 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534131+0000", 2024-09-13T09:12:43.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096642849999999999 2024-09-13T09:12:43.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534138+0000", 2024-09-13T09:12:43.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7490000000000001e-06 2024-09-13T09:12:43.412 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.412 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.412 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.412 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20077 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.413 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528860+0000", 2024-09-13T09:12:43.413 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33592878199999998, 2024-09-13T09:12:43.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34001565299999997, 2024-09-13T09:12:43.413 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.413 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.413 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.413 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.413 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.413 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20077 2024-09-13T09:12:43.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.413 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528860+0000", 2024-09-13T09:12:43.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528860+0000", 2024-09-13T09:12:43.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528860+0000", 2024-09-13T09:12:43.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528862+0000", 2024-09-13T09:12:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:12:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528868+0000", 2024-09-13T09:12:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8250000000000001e-06 2024-09-13T09:12:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528872+0000", 2024-09-13T09:12:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.155e-06 2024-09-13T09:12:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535891+0000", 2024-09-13T09:12:43.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0070185070000000002 2024-09-13T09:12:43.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535902+0000", 2024-09-13T09:12:43.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1167e-05 2024-09-13T09:12:43.417 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.417 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.417 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.418 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20187 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.418 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538225+0000", 2024-09-13T09:12:43.418 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32656406900000001, 2024-09-13T09:12:43.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33068415000000001, 2024-09-13T09:12:43.418 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.418 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.419 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.419 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.419 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.419 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20187 2024-09-13T09:12:43.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.419 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538225+0000", 2024-09-13T09:12:43.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538225+0000", 2024-09-13T09:12:43.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538225+0000", 2024-09-13T09:12:43.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538227+0000", 2024-09-13T09:12:43.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:12:43.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538234+0000", 2024-09-13T09:12:43.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8839999999999997e-06 2024-09-13T09:12:43.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538239+0000", 2024-09-13T09:12:43.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7559999999999999e-06 2024-09-13T09:12:43.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.424 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538345+0000", 2024-09-13T09:12:43.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010563799999999999 2024-09-13T09:12:43.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538352+0000", 2024-09-13T09:12:43.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4789999999999999e-06 2024-09-13T09:12:43.425 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.425 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.425 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.426 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20330 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.426 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540960+0000", 2024-09-13T09:12:43.426 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32382916, 2024-09-13T09:12:43.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32798238000000002, 2024-09-13T09:12:43.426 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.426 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.426 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.426 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.426 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.426 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20330 2024-09-13T09:12:43.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.427 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540960+0000", 2024-09-13T09:12:43.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540960+0000", 2024-09-13T09:12:43.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540959+0000", 2024-09-13T09:12:43.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540962+0000", 2024-09-13T09:12:43.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9229999999999998e-06 2024-09-13T09:12:43.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540969+0000", 2024-09-13T09:12:43.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7120000000000003e-06 2024-09-13T09:12:43.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540974+0000", 2024-09-13T09:12:43.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8999999999999997e-06 2024-09-13T09:12:43.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541843+0000", 2024-09-13T09:12:43.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00086893599999999995 2024-09-13T09:12:43.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541850+0000", 2024-09-13T09:12:43.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.486e-06 2024-09-13T09:12:43.429 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.430 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.430 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.430 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20441 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.430 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543012+0000", 2024-09-13T09:12:43.430 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.321777549, 2024-09-13T09:12:43.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.325963907, 2024-09-13T09:12:43.430 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.430 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.430 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.430 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.430 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.430 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20441 2024-09-13T09:12:43.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.431 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543012+0000", 2024-09-13T09:12:43.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543012+0000", 2024-09-13T09:12:43.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543011+0000", 2024-09-13T09:12:43.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543013+0000", 2024-09-13T09:12:43.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0849999999999999e-06 2024-09-13T09:12:43.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543019+0000", 2024-09-13T09:12:43.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7590000000000003e-06 2024-09-13T09:12:43.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543024+0000", 2024-09-13T09:12:43.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4360000000000004e-06 2024-09-13T09:12:43.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543208+0000", 2024-09-13T09:12:43.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018434699999999999 2024-09-13T09:12:43.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543215+0000", 2024-09-13T09:12:43.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0380000000000002e-06 2024-09-13T09:12:43.434 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.434 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.434 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.434 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20473 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.434 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.544106+0000", 2024-09-13T09:12:43.434 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32068277499999998, 2024-09-13T09:12:43.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32490216500000002, 2024-09-13T09:12:43.434 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.435 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.435 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.435 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.435 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.435 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20473 2024-09-13T09:12:43.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.435 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544106+0000", 2024-09-13T09:12:43.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544106+0000", 2024-09-13T09:12:43.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544106+0000", 2024-09-13T09:12:43.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544108+0000", 2024-09-13T09:12:43.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1880000000000001e-06 2024-09-13T09:12:43.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544114+0000", 2024-09-13T09:12:43.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6670000000000001e-06 2024-09-13T09:12:43.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544118+0000", 2024-09-13T09:12:43.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.335e-06 2024-09-13T09:12:43.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544840+0000", 2024-09-13T09:12:43.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00072187300000000005 2024-09-13T09:12:43.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.438 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544849+0000", 2024-09-13T09:12:43.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8799999999999997e-06 2024-09-13T09:12:43.438 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.438 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.438 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.438 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20584 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.438 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547661+0000", 2024-09-13T09:12:43.438 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31712811299999999, 2024-09-13T09:12:43.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.321380261, 2024-09-13T09:12:43.438 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.438 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.438 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.438 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.439 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.439 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20584 2024-09-13T09:12:43.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.439 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547661+0000", 2024-09-13T09:12:43.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547661+0000", 2024-09-13T09:12:43.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547660+0000", 2024-09-13T09:12:43.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547663+0000", 2024-09-13T09:12:43.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7039999999999999e-06 2024-09-13T09:12:43.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547670+0000", 2024-09-13T09:12:43.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1420000000000004e-06 2024-09-13T09:12:43.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547675+0000", 2024-09-13T09:12:43.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:12:43.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.442 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547939+0000", 2024-09-13T09:12:43.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00026380799999999998 2024-09-13T09:12:43.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.442 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.442 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547949+0000", 2024-09-13T09:12:43.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:12:43.442 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.442 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.442 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.442 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20736 3.12 3.8d2c5fb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.442 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550652+0000", 2024-09-13T09:12:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31413730600000001, 2024-09-13T09:12:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.318422502, 2024-09-13T09:12:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20736 2024-09-13T09:12:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550652+0000", 2024-09-13T09:12:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.444 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550652+0000", 2024-09-13T09:12:43.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.444 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550652+0000", 2024-09-13T09:12:43.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.444 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550654+0000", 2024-09-13T09:12:43.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2239999999999998e-06 2024-09-13T09:12:43.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550673+0000", 2024-09-13T09:12:43.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9130000000000001e-05 2024-09-13T09:12:43.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550678+0000", 2024-09-13T09:12:43.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9899999999999997e-06 2024-09-13T09:12:43.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550970+0000", 2024-09-13T09:12:43.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00029204799999999999 2024-09-13T09:12:43.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550973+0000", 2024-09-13T09:12:43.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2200000000000001e-06 2024-09-13T09:12:43.446 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.446 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.446 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.446 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19936 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.446 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524472+0000", 2024-09-13T09:12:43.446 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34031748299999998, 2024-09-13T09:12:43.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.344636053, 2024-09-13T09:12:43.447 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.447 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.447 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.447 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.447 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.447 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19936 2024-09-13T09:12:43.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.447 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524472+0000", 2024-09-13T09:12:43.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524472+0000", 2024-09-13T09:12:43.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524471+0000", 2024-09-13T09:12:43.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524474+0000", 2024-09-13T09:12:43.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3410000000000001e-06 2024-09-13T09:12:43.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524480+0000", 2024-09-13T09:12:43.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1630000000000001e-06 2024-09-13T09:12:43.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524485+0000", 2024-09-13T09:12:43.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8620000000000002e-06 2024-09-13T09:12:43.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534162+0000", 2024-09-13T09:12:43.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096773610000000006 2024-09-13T09:12:43.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.450 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534169+0000", 2024-09-13T09:12:43.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0099999999999998e-06 2024-09-13T09:12:43.450 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.450 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.450 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.450 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20078 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.450 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528876+0000", 2024-09-13T09:12:43.451 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33591278299999999, 2024-09-13T09:12:43.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34026466100000002, 2024-09-13T09:12:43.451 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.451 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.451 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.451 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.451 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.451 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20078 2024-09-13T09:12:43.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.451 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528876+0000", 2024-09-13T09:12:43.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.452 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528876+0000", 2024-09-13T09:12:43.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.452 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528876+0000", 2024-09-13T09:12:43.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.452 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528878+0000", 2024-09-13T09:12:43.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2809999999999998e-06 2024-09-13T09:12:43.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528884+0000", 2024-09-13T09:12:43.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7309999999999998e-06 2024-09-13T09:12:43.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528889+0000", 2024-09-13T09:12:43.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5979999999999999e-06 2024-09-13T09:12:43.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536150+0000", 2024-09-13T09:12:43.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072612850000000001 2024-09-13T09:12:43.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536160+0000", 2024-09-13T09:12:43.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7990000000000005e-06 2024-09-13T09:12:43.454 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.454 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.454 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.455 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20188 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.455 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538244+0000", 2024-09-13T09:12:43.455 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32654560500000002, 2024-09-13T09:12:43.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33093103400000001, 2024-09-13T09:12:43.455 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.455 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.455 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.455 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.455 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.455 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20188 2024-09-13T09:12:43.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.455 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.456 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538244+0000", 2024-09-13T09:12:43.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.456 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538244+0000", 2024-09-13T09:12:43.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.456 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538243+0000", 2024-09-13T09:12:43.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538245+0000", 2024-09-13T09:12:43.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2330000000000001e-06 2024-09-13T09:12:43.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538251+0000", 2024-09-13T09:12:43.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2580000000000003e-06 2024-09-13T09:12:43.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538255+0000", 2024-09-13T09:12:43.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3109999999999999e-06 2024-09-13T09:12:43.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538378+0000", 2024-09-13T09:12:43.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000122686 2024-09-13T09:12:43.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538385+0000", 2024-09-13T09:12:43.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1729999999999998e-06 2024-09-13T09:12:43.459 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.459 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.459 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.459 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20331 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.459 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540979+0000", 2024-09-13T09:12:43.459 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32381051900000002, 2024-09-13T09:12:43.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32822915699999999, 2024-09-13T09:12:43.459 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.460 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.460 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.460 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.460 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.460 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20331 2024-09-13T09:12:43.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.460 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.460 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540979+0000", 2024-09-13T09:12:43.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540979+0000", 2024-09-13T09:12:43.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540978+0000", 2024-09-13T09:12:43.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540981+0000", 2024-09-13T09:12:43.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7149999999999998e-06 2024-09-13T09:12:43.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540987+0000", 2024-09-13T09:12:43.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3450000000000002e-06 2024-09-13T09:12:43.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540992+0000", 2024-09-13T09:12:43.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1379999999999998e-06 2024-09-13T09:12:43.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541878+0000", 2024-09-13T09:12:43.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000885486 2024-09-13T09:12:43.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.463 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541885+0000", 2024-09-13T09:12:43.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6380000000000004e-06 2024-09-13T09:12:43.463 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.463 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.463 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.464 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20442 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.464 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543028+0000", 2024-09-13T09:12:43.464 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.321761409, 2024-09-13T09:12:43.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32621314299999998, 2024-09-13T09:12:43.464 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.464 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.464 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.464 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.464 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.464 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20442 2024-09-13T09:12:43.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.464 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543028+0000", 2024-09-13T09:12:43.465 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543028+0000", 2024-09-13T09:12:43.465 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543027+0000", 2024-09-13T09:12:43.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.466 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543029+0000", 2024-09-13T09:12:43.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.888e-06 2024-09-13T09:12:43.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.466 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543035+0000", 2024-09-13T09:12:43.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6659999999999996e-06 2024-09-13T09:12:43.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543041+0000", 2024-09-13T09:12:43.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.626e-06 2024-09-13T09:12:43.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.467 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543322+0000", 2024-09-13T09:12:43.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00028158399999999998 2024-09-13T09:12:43.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.467 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543331+0000", 2024-09-13T09:12:43.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7960000000000001e-06 2024-09-13T09:12:43.467 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.467 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.468 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.468 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20474 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.468 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.544144+0000", 2024-09-13T09:12:43.468 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32064566300000003, 2024-09-13T09:12:43.468 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32513025600000001, 2024-09-13T09:12:43.468 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.468 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.468 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.469 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.469 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.469 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20474 2024-09-13T09:12:43.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.469 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.469 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.469 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544144+0000", 2024-09-13T09:12:43.469 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.469 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.469 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544144+0000", 2024-09-13T09:12:43.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544143+0000", 2024-09-13T09:12:43.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544146+0000", 2024-09-13T09:12:43.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:12:43.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544153+0000", 2024-09-13T09:12:43.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1690000000000003e-06 2024-09-13T09:12:43.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544158+0000", 2024-09-13T09:12:43.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1120000000000004e-06 2024-09-13T09:12:43.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544874+0000", 2024-09-13T09:12:43.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00071544399999999998 2024-09-13T09:12:43.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.472 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.472 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.472 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544880+0000", 2024-09-13T09:12:43.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8199999999999999e-06 2024-09-13T09:12:43.472 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.472 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.472 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.472 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.472 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20585 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.473 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547699+0000", 2024-09-13T09:12:43.473 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31709016600000001, 2024-09-13T09:12:43.473 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32160752799999998, 2024-09-13T09:12:43.473 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.473 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.473 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.473 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.473 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.473 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20585 2024-09-13T09:12:43.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.473 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.473 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.473 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547699+0000", 2024-09-13T09:12:43.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.474 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547699+0000", 2024-09-13T09:12:43.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.474 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547698+0000", 2024-09-13T09:12:43.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547701+0000", 2024-09-13T09:12:43.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8399999999999999e-06 2024-09-13T09:12:43.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547708+0000", 2024-09-13T09:12:43.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9890000000000003e-06 2024-09-13T09:12:43.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547713+0000", 2024-09-13T09:12:43.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0019999999999997e-06 2024-09-13T09:12:43.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547983+0000", 2024-09-13T09:12:43.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00026980599999999998 2024-09-13T09:12:43.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547991+0000", 2024-09-13T09:12:43.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8499999999999994e-06 2024-09-13T09:12:43.476 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.476 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.476 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.477 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20737 3.12 3.8d2c5fb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.477 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550685+0000", 2024-09-13T09:12:43.477 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.314104308, 2024-09-13T09:12:43.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31865484500000002, 2024-09-13T09:12:43.477 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.477 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.477 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.477 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.477 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.477 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20737 2024-09-13T09:12:43.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.477 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550685+0000", 2024-09-13T09:12:43.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.478 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550685+0000", 2024-09-13T09:12:43.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.478 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550682+0000", 2024-09-13T09:12:43.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999971 2024-09-13T09:12:43.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550687+0000", 2024-09-13T09:12:43.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1410000000000004e-06 2024-09-13T09:12:43.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550694+0000", 2024-09-13T09:12:43.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:12:43.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550698+0000", 2024-09-13T09:12:43.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0080000000000004e-06 2024-09-13T09:12:43.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550983+0000", 2024-09-13T09:12:43.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00028508400000000001 2024-09-13T09:12:43.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550986+0000", 2024-09-13T09:12:43.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1350000000000001e-06 2024-09-13T09:12:43.480 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.481 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.481 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.481 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19937 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.481 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524518+0000", 2024-09-13T09:12:43.481 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34027152900000002, 2024-09-13T09:12:43.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34485521099999999, 2024-09-13T09:12:43.481 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.481 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.481 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.481 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.481 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.481 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19937 2024-09-13T09:12:43.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.482 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524518+0000", 2024-09-13T09:12:43.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524518+0000", 2024-09-13T09:12:43.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524517+0000", 2024-09-13T09:12:43.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524520+0000", 2024-09-13T09:12:43.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3060000000000002e-06 2024-09-13T09:12:43.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524531+0000", 2024-09-13T09:12:43.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0694e-05 2024-09-13T09:12:43.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524537+0000", 2024-09-13T09:12:43.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:12:43.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534190+0000", 2024-09-13T09:12:43.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096528529999999994 2024-09-13T09:12:43.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534197+0000", 2024-09-13T09:12:43.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.002e-06 2024-09-13T09:12:43.484 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.485 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.485 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.485 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20079 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.485 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528893+0000", 2024-09-13T09:12:43.485 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33589636899999997, 2024-09-13T09:12:43.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34051292999999999, 2024-09-13T09:12:43.485 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.485 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.485 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.485 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.485 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.486 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20079 2024-09-13T09:12:43.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.486 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528893+0000", 2024-09-13T09:12:43.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528893+0000", 2024-09-13T09:12:43.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528892+0000", 2024-09-13T09:12:43.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528895+0000", 2024-09-13T09:12:43.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1749999999999999e-06 2024-09-13T09:12:43.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528913+0000", 2024-09-13T09:12:43.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8266000000000001e-05 2024-09-13T09:12:43.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528918+0000", 2024-09-13T09:12:43.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2290000000000002e-06 2024-09-13T09:12:43.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536192+0000", 2024-09-13T09:12:43.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007274076 2024-09-13T09:12:43.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536200+0000", 2024-09-13T09:12:43.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:12:43.489 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.489 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.489 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.490 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20189 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.490 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538269+0000", 2024-09-13T09:12:43.490 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32651970699999999, 2024-09-13T09:12:43.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.331169029, 2024-09-13T09:12:43.490 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.490 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.490 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.490 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.491 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.491 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20189 2024-09-13T09:12:43.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.491 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538269+0000", 2024-09-13T09:12:43.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538269+0000", 2024-09-13T09:12:43.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538269+0000", 2024-09-13T09:12:43.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538276+0000", 2024-09-13T09:12:43.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3080000000000002e-06 2024-09-13T09:12:43.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538283+0000", 2024-09-13T09:12:43.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5130000000000002e-06 2024-09-13T09:12:43.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538288+0000", 2024-09-13T09:12:43.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7190000000000001e-06 2024-09-13T09:12:43.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538631+0000", 2024-09-13T09:12:43.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034347300000000001 2024-09-13T09:12:43.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538639+0000", 2024-09-13T09:12:43.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:12:43.494 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.494 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.494 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.494 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20332 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.494 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540997+0000", 2024-09-13T09:12:43.494 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.323792266, 2024-09-13T09:12:43.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.328474987, 2024-09-13T09:12:43.494 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.494 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.494 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.495 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.495 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.495 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20332 2024-09-13T09:12:43.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.495 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540997+0000", 2024-09-13T09:12:43.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540997+0000", 2024-09-13T09:12:43.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540996+0000", 2024-09-13T09:12:43.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541006+0000", 2024-09-13T09:12:43.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0017000000000001e-05 2024-09-13T09:12:43.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541014+0000", 2024-09-13T09:12:43.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0589999999999997e-06 2024-09-13T09:12:43.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541024+0000", 2024-09-13T09:12:43.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9699999999999994e-06 2024-09-13T09:12:43.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541912+0000", 2024-09-13T09:12:43.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00088882800000000004 2024-09-13T09:12:43.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541920+0000", 2024-09-13T09:12:43.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3019999999999997e-06 2024-09-13T09:12:43.498 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.498 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.498 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.498 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20443 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.498 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543045+0000", 2024-09-13T09:12:43.498 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32174440599999998, 2024-09-13T09:12:43.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32645993099999998, 2024-09-13T09:12:43.498 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.498 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.498 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.499 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.499 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.499 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20443 2024-09-13T09:12:43.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.499 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543045+0000", 2024-09-13T09:12:43.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543045+0000", 2024-09-13T09:12:43.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543044+0000", 2024-09-13T09:12:43.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543046+0000", 2024-09-13T09:12:43.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.17e-06 2024-09-13T09:12:43.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543052+0000", 2024-09-13T09:12:43.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7509999999999997e-06 2024-09-13T09:12:43.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543056+0000", 2024-09-13T09:12:43.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2599999999999999e-06 2024-09-13T09:12:43.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543362+0000", 2024-09-13T09:12:43.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00030513300000000002 2024-09-13T09:12:43.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543369+0000", 2024-09-13T09:12:43.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1069999999999999e-06 2024-09-13T09:12:43.502 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.502 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.502 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.502 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20586 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.503 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547718+0000", 2024-09-13T09:12:43.503 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.3170712, 2024-09-13T09:12:43.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32182000900000002, 2024-09-13T09:12:43.503 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.503 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.503 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.503 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.503 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.503 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20586 2024-09-13T09:12:43.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.503 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547718+0000", 2024-09-13T09:12:43.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547718+0000", 2024-09-13T09:12:43.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547717+0000", 2024-09-13T09:12:43.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547720+0000", 2024-09-13T09:12:43.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:12:43.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547725+0000", 2024-09-13T09:12:43.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:12:43.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547730+0000", 2024-09-13T09:12:43.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3159999999999998e-06 2024-09-13T09:12:43.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:12:43.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00029942400000000001 2024-09-13T09:12:43.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548037+0000", 2024-09-13T09:12:43.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:12:43.507 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.507 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.507 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.507 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20704 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.507 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549380+0000", 2024-09-13T09:12:43.507 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31540896400000001, 2024-09-13T09:12:43.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32019115399999998, 2024-09-13T09:12:43.507 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.508 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.508 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.508 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.508 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.508 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20704 2024-09-13T09:12:43.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.508 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549380+0000", 2024-09-13T09:12:43.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549380+0000", 2024-09-13T09:12:43.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549380+0000", 2024-09-13T09:12:43.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549382+0000", 2024-09-13T09:12:43.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1919999999999999e-06 2024-09-13T09:12:43.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549388+0000", 2024-09-13T09:12:43.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.784e-06 2024-09-13T09:12:43.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549392+0000", 2024-09-13T09:12:43.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5730000000000001e-06 2024-09-13T09:12:43.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549446+0000", 2024-09-13T09:12:43.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4181999999999997e-05 2024-09-13T09:12:43.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549457+0000", 2024-09-13T09:12:43.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0247e-05 2024-09-13T09:12:43.511 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.511 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.512 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.512 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20738 3.12 3.8d2c5fb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.512 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550702+0000", 2024-09-13T09:12:43.512 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.314087161, 2024-09-13T09:12:43.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31890228199999998, 2024-09-13T09:12:43.512 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.512 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.512 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.512 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.512 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.512 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20738 2024-09-13T09:12:43.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.513 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550702+0000", 2024-09-13T09:12:43.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550702+0000", 2024-09-13T09:12:43.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.514 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.514 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550702+0000", 2024-09-13T09:12:43.514 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.514 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.514 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550704+0000", 2024-09-13T09:12:43.514 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.007e-06 2024-09-13T09:12:43.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.514 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.514 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550709+0000", 2024-09-13T09:12:43.514 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0150000000000003e-06 2024-09-13T09:12:43.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550722+0000", 2024-09-13T09:12:43.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2985e-05 2024-09-13T09:12:43.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550996+0000", 2024-09-13T09:12:43.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00027437099999999998 2024-09-13T09:12:43.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550999+0000", 2024-09-13T09:12:43.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1599999999999998e-06 2024-09-13T09:12:43.515 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.516 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.516 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.516 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19828 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.516 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.522171+0000", 2024-09-13T09:12:43.516 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34261801600000003, 2024-09-13T09:12:43.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.347466316, 2024-09-13T09:12:43.516 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.516 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.516 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.516 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.517 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.517 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19828 2024-09-13T09:12:43.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.517 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522171+0000", 2024-09-13T09:12:43.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522171+0000", 2024-09-13T09:12:43.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522171+0000", 2024-09-13T09:12:43.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522173+0000", 2024-09-13T09:12:43.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.041e-06 2024-09-13T09:12:43.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522177+0000", 2024-09-13T09:12:43.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:12:43.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522181+0000", 2024-09-13T09:12:43.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:12:43.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.532892+0000", 2024-09-13T09:12:43.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010711385 2024-09-13T09:12:43.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.520 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.532905+0000", 2024-09-13T09:12:43.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2306999999999999e-05 2024-09-13T09:12:43.520 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.520 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.520 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.520 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19938 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.520 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524542+0000", 2024-09-13T09:12:43.521 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34024674799999999, 2024-09-13T09:12:43.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34512815000000002, 2024-09-13T09:12:43.521 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.521 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.521 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.521 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.521 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.521 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19938 2024-09-13T09:12:43.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.521 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524542+0000", 2024-09-13T09:12:43.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524542+0000", 2024-09-13T09:12:43.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524542+0000", 2024-09-13T09:12:43.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524545+0000", 2024-09-13T09:12:43.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6259999999999999e-06 2024-09-13T09:12:43.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524551+0000", 2024-09-13T09:12:43.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.404e-06 2024-09-13T09:12:43.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524556+0000", 2024-09-13T09:12:43.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8049999999999997e-06 2024-09-13T09:12:43.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534220+0000", 2024-09-13T09:12:43.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.009663764 2024-09-13T09:12:43.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534226+0000", 2024-09-13T09:12:43.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:12:43.524 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.524 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.524 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.524 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.524 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:12:43.524 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33586518799999998, 2024-09-13T09:12:43.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34077945599999998, 2024-09-13T09:12:43.525 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.525 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.525 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.525 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.525 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.525 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:12:43.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.525 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:12:43.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:12:43.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:12:43.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:12:43.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:12:43.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:12:43.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:12:43.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:12:43.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:12:43.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:12:43.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:12:43.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:12:43.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:12:43.528 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.528 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.528 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.528 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20190 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.529 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538306+0000", 2024-09-13T09:12:43.529 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32648360999999998, 2024-09-13T09:12:43.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33143059000000002, 2024-09-13T09:12:43.529 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.529 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.529 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.529 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.529 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.529 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20190 2024-09-13T09:12:43.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.529 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538306+0000", 2024-09-13T09:12:43.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538306+0000", 2024-09-13T09:12:43.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538305+0000", 2024-09-13T09:12:43.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538308+0000", 2024-09-13T09:12:43.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5620000000000002e-06 2024-09-13T09:12:43.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538314+0000", 2024-09-13T09:12:43.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.455e-06 2024-09-13T09:12:43.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.532 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538320+0000", 2024-09-13T09:12:43.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:12:43.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.532 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538661+0000", 2024-09-13T09:12:43.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034017699999999999 2024-09-13T09:12:43.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538668+0000", 2024-09-13T09:12:43.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4059999999999999e-06 2024-09-13T09:12:43.533 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.533 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.533 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.533 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20333 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.533 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.541041+0000", 2024-09-13T09:12:43.533 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32374822199999997, 2024-09-13T09:12:43.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32872798800000003, 2024-09-13T09:12:43.534 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.534 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.534 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.534 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.534 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.534 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20333 2024-09-13T09:12:43.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.534 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541041+0000", 2024-09-13T09:12:43.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541041+0000", 2024-09-13T09:12:43.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541040+0000", 2024-09-13T09:12:43.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541043+0000", 2024-09-13T09:12:43.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1140000000000002e-06 2024-09-13T09:12:43.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541052+0000", 2024-09-13T09:12:43.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8459999999999997e-06 2024-09-13T09:12:43.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541057+0000", 2024-09-13T09:12:43.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0989999999999998e-06 2024-09-13T09:12:43.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541944+0000", 2024-09-13T09:12:43.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00088673099999999998 2024-09-13T09:12:43.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541951+0000", 2024-09-13T09:12:43.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1199999999999996e-06 2024-09-13T09:12:43.537 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.537 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.537 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.537 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20444 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.537 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543060+0000", 2024-09-13T09:12:43.538 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32172869799999998, 2024-09-13T09:12:43.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.326741487, 2024-09-13T09:12:43.538 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.538 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.538 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.538 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.538 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.538 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20444 2024-09-13T09:12:43.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.538 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543060+0000", 2024-09-13T09:12:43.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543060+0000", 2024-09-13T09:12:43.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543060+0000", 2024-09-13T09:12:43.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543062+0000", 2024-09-13T09:12:43.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9400000000000001e-06 2024-09-13T09:12:43.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543067+0000", 2024-09-13T09:12:43.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4160000000000003e-06 2024-09-13T09:12:43.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543072+0000", 2024-09-13T09:12:43.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0670000000000002e-06 2024-09-13T09:12:43.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.541 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543395+0000", 2024-09-13T09:12:43.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00032303399999999998 2024-09-13T09:12:43.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.541 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543401+0000", 2024-09-13T09:12:43.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8580000000000002e-06 2024-09-13T09:12:43.542 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.542 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.542 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.542 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20587 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.542 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547745+0000", 2024-09-13T09:12:43.542 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31704390399999999, 2024-09-13T09:12:43.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.322094094, 2024-09-13T09:12:43.542 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.542 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.542 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.543 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.543 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.543 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20587 2024-09-13T09:12:43.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.543 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.543 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.543 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547745+0000", 2024-09-13T09:12:43.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.543 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.543 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547745+0000", 2024-09-13T09:12:43.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547745+0000", 2024-09-13T09:12:43.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547747+0000", 2024-09-13T09:12:43.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6970000000000002e-06 2024-09-13T09:12:43.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547754+0000", 2024-09-13T09:12:43.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:12:43.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547760+0000", 2024-09-13T09:12:43.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7209999999999999e-06 2024-09-13T09:12:43.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548072+0000", 2024-09-13T09:12:43.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00031280199999999999 2024-09-13T09:12:43.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548080+0000", 2024-09-13T09:12:43.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9419999999999995e-06 2024-09-13T09:12:43.546 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.546 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.546 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.546 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20705 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.546 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549401+0000", 2024-09-13T09:12:43.546 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.315388314, 2024-09-13T09:12:43.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.320471744, 2024-09-13T09:12:43.546 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.546 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20705 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549401+0000", 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549401+0000", 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549400+0000", 2024-09-13T09:12:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549403+0000", 2024-09-13T09:12:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3539999999999998e-06 2024-09-13T09:12:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549409+0000", 2024-09-13T09:12:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:12:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549427+0000", 2024-09-13T09:12:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7949e-05 2024-09-13T09:12:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549486+0000", 2024-09-13T09:12:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9465000000000001e-05 2024-09-13T09:12:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549495+0000", 2024-09-13T09:12:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3750000000000003e-06 2024-09-13T09:12:43.550 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.550 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.550 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.550 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20739 3.12 3.8d2c5fb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.550 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550727+0000", 2024-09-13T09:12:43.550 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31406220299999998, 2024-09-13T09:12:43.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31917834899999997, 2024-09-13T09:12:43.550 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.550 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.550 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.550 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.550 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.551 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20739 2024-09-13T09:12:43.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.551 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550727+0000", 2024-09-13T09:12:43.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550727+0000", 2024-09-13T09:12:43.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550727+0000", 2024-09-13T09:12:43.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550729+0000", 2024-09-13T09:12:43.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.667e-06 2024-09-13T09:12:43.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550737+0000", 2024-09-13T09:12:43.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7819999999999994e-06 2024-09-13T09:12:43.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550742+0000", 2024-09-13T09:12:43.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4670000000000003e-06 2024-09-13T09:12:43.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.551009+0000", 2024-09-13T09:12:43.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00026664800000000003 2024-09-13T09:12:43.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.551012+0000", 2024-09-13T09:12:43.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0939999999999999e-06 2024-09-13T09:12:43.554 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.554 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.554 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.554 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19829 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.554 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.522184+0000", 2024-09-13T09:12:43.554 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.342604674, 2024-09-13T09:12:43.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34775395100000001, 2024-09-13T09:12:43.555 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.555 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.555 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.555 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.555 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.555 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19829 2024-09-13T09:12:43.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.555 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522184+0000", 2024-09-13T09:12:43.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522184+0000", 2024-09-13T09:12:43.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522184+0000", 2024-09-13T09:12:43.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522186+0000", 2024-09-13T09:12:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.73e-06 2024-09-13T09:12:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522191+0000", 2024-09-13T09:12:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:12:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522194+0000", 2024-09-13T09:12:43.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6389999999999998e-06 2024-09-13T09:12:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533302+0000", 2024-09-13T09:12:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011107883000000001 2024-09-13T09:12:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533310+0000", 2024-09-13T09:12:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1489999999999994e-06 2024-09-13T09:12:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.558 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19939 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524561+0000", 2024-09-13T09:12:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.340228486, 2024-09-13T09:12:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34541055100000001, 2024-09-13T09:12:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19939 2024-09-13T09:12:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.559 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524561+0000", 2024-09-13T09:12:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524561+0000", 2024-09-13T09:12:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524560+0000", 2024-09-13T09:12:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524562+0000", 2024-09-13T09:12:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2060000000000001e-06 2024-09-13T09:12:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524571+0000", 2024-09-13T09:12:43.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8200000000000003e-06 2024-09-13T09:12:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524576+0000", 2024-09-13T09:12:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.454e-06 2024-09-13T09:12:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534248+0000", 2024-09-13T09:12:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096718020000000002 2024-09-13T09:12:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534255+0000", 2024-09-13T09:12:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0670000000000003e-06 2024-09-13T09:12:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:12:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33584715300000001, 2024-09-13T09:12:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34106207700000002, 2024-09-13T09:12:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.563 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.564 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.564 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:12:43.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.564 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:12:43.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:12:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:12:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:12:43.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:12:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:12:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:12:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:12:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:12:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:12:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:12:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:12:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:12:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20191 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538340+0000", 2024-09-13T09:12:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32644932799999998, 2024-09-13T09:12:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33169731099999999, 2024-09-13T09:12:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20191 2024-09-13T09:12:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538340+0000", 2024-09-13T09:12:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538340+0000", 2024-09-13T09:12:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538339+0000", 2024-09-13T09:12:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538342+0000", 2024-09-13T09:12:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3889999999999999e-06 2024-09-13T09:12:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538348+0000", 2024-09-13T09:12:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5080000000000002e-06 2024-09-13T09:12:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538353+0000", 2024-09-13T09:12:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:12:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538690+0000", 2024-09-13T09:12:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000337034 2024-09-13T09:12:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538697+0000", 2024-09-13T09:12:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9639999999999997e-06 2024-09-13T09:12:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.571 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20334 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.572 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.541062+0000", 2024-09-13T09:12:43.572 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32372680100000001, 2024-09-13T09:12:43.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32900769899999999, 2024-09-13T09:12:43.572 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.572 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.572 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.572 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.572 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.572 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20334 2024-09-13T09:12:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541062+0000", 2024-09-13T09:12:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541062+0000", 2024-09-13T09:12:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541062+0000", 2024-09-13T09:12:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541064+0000", 2024-09-13T09:12:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3989999999999998e-06 2024-09-13T09:12:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541070+0000", 2024-09-13T09:12:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8529999999999997e-06 2024-09-13T09:12:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541075+0000", 2024-09-13T09:12:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0200000000000002e-06 2024-09-13T09:12:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541979+0000", 2024-09-13T09:12:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00090434499999999998 2024-09-13T09:12:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541986+0000", 2024-09-13T09:12:43.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9210000000000003e-06 2024-09-13T09:12:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20445 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543081+0000", 2024-09-13T09:12:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32170782599999997, 2024-09-13T09:12:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32702192099999999, 2024-09-13T09:12:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.576 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20445 2024-09-13T09:12:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543081+0000", 2024-09-13T09:12:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543081+0000", 2024-09-13T09:12:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543081+0000", 2024-09-13T09:12:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543083+0000", 2024-09-13T09:12:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4119999999999999e-06 2024-09-13T09:12:43.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543089+0000", 2024-09-13T09:12:43.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1090000000000003e-06 2024-09-13T09:12:43.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543094+0000", 2024-09-13T09:12:43.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4519999999999999e-06 2024-09-13T09:12:43.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543425+0000", 2024-09-13T09:12:43.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000331274 2024-09-13T09:12:43.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543432+0000", 2024-09-13T09:12:43.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6109999999999999e-06 2024-09-13T09:12:43.580 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.580 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.580 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20588 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547783+0000", 2024-09-13T09:12:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31700624999999999, 2024-09-13T09:12:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32235322599999999, 2024-09-13T09:12:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20588 2024-09-13T09:12:43.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.582 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547783+0000", 2024-09-13T09:12:43.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547783+0000", 2024-09-13T09:12:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547782+0000", 2024-09-13T09:12:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547785+0000", 2024-09-13T09:12:43.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6220000000000001e-06 2024-09-13T09:12:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547792+0000", 2024-09-13T09:12:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6250000000000001e-06 2024-09-13T09:12:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547803+0000", 2024-09-13T09:12:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1063000000000001e-05 2024-09-13T09:12:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548113+0000", 2024-09-13T09:12:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00031036699999999998 2024-09-13T09:12:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548121+0000", 2024-09-13T09:12:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7829999999999999e-06 2024-09-13T09:12:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20706 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549439+0000", 2024-09-13T09:12:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31534967200000003, 2024-09-13T09:12:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32072973599999999, 2024-09-13T09:12:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20706 2024-09-13T09:12:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549439+0000", 2024-09-13T09:12:43.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549439+0000", 2024-09-13T09:12:43.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549439+0000", 2024-09-13T09:12:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549448+0000", 2024-09-13T09:12:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3570000000000004e-06 2024-09-13T09:12:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549457+0000", 2024-09-13T09:12:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8570000000000001e-06 2024-09-13T09:12:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549463+0000", 2024-09-13T09:12:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:12:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549524+0000", 2024-09-13T09:12:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0343999999999997e-05 2024-09-13T09:12:43.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549532+0000", 2024-09-13T09:12:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1159999999999999e-06 2024-09-13T09:12:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20740 3.12 3.8d2c5fb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.590 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.550747+0000", 2024-09-13T09:12:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31404263999999998, 2024-09-13T09:12:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.31945549400000001, 2024-09-13T09:12:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20740 2024-09-13T09:12:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550747+0000", 2024-09-13T09:12:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550747+0000", 2024-09-13T09:12:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550746+0000", 2024-09-13T09:12:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550749+0000", 2024-09-13T09:12:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:12:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550756+0000", 2024-09-13T09:12:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.729e-06 2024-09-13T09:12:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.550761+0000", 2024-09-13T09:12:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3599999999999998e-06 2024-09-13T09:12:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.551022+0000", 2024-09-13T09:12:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00026134699999999999 2024-09-13T09:12:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.551025+0000", 2024-09-13T09:12:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.129e-06 2024-09-13T09:12:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.594 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.595 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.595 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19830 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.595 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.522198+0000", 2024-09-13T09:12:43.595 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34259126099999998, 2024-09-13T09:12:43.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34803721300000001, 2024-09-13T09:12:43.595 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.595 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.595 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.595 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.595 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.595 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19830 2024-09-13T09:12:43.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.595 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522198+0000", 2024-09-13T09:12:43.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522198+0000", 2024-09-13T09:12:43.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522197+0000", 2024-09-13T09:12:43.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522199+0000", 2024-09-13T09:12:43.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9039999999999999e-06 2024-09-13T09:12:43.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522204+0000", 2024-09-13T09:12:43.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6770000000000003e-06 2024-09-13T09:12:43.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522208+0000", 2024-09-13T09:12:43.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5080000000000001e-06 2024-09-13T09:12:43.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533414+0000", 2024-09-13T09:12:43.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011206763999999999 2024-09-13T09:12:43.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533423+0000", 2024-09-13T09:12:43.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5650000000000002e-06 2024-09-13T09:12:43.598 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.598 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.598 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.599 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19940 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.599 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524581+0000", 2024-09-13T09:12:43.599 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34020843899999997, 2024-09-13T09:12:43.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34568748300000002, 2024-09-13T09:12:43.599 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.599 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.599 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.599 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.599 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.599 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19940 2024-09-13T09:12:43.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.599 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524581+0000", 2024-09-13T09:12:43.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524581+0000", 2024-09-13T09:12:43.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524580+0000", 2024-09-13T09:12:43.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524583+0000", 2024-09-13T09:12:43.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5409999999999999e-06 2024-09-13T09:12:43.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524589+0000", 2024-09-13T09:12:43.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2269999999999998e-06 2024-09-13T09:12:43.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524594+0000", 2024-09-13T09:12:43.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5800000000000002e-06 2024-09-13T09:12:43.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.602 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534277+0000", 2024-09-13T09:12:43.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096838819999999996 2024-09-13T09:12:43.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.602 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534285+0000", 2024-09-13T09:12:43.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1210000000000001e-06 2024-09-13T09:12:43.602 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.602 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.602 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.603 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.603 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:12:43.603 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33583048300000001, 2024-09-13T09:12:43.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34134276600000002, 2024-09-13T09:12:43.603 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.603 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.603 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.603 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.603 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.603 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:12:43.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.604 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:12:43.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:12:43.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:12:43.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:12:43.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:12:43.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:12:43.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:12:43.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:12:43.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:12:43.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:12:43.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:12:43.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:12:43.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:12:43.606 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.606 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.607 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.607 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20192 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.607 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538374+0000", 2024-09-13T09:12:43.607 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32641516700000001, 2024-09-13T09:12:43.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33196065499999999, 2024-09-13T09:12:43.607 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.607 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.607 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.607 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.607 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.607 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20192 2024-09-13T09:12:43.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.608 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538374+0000", 2024-09-13T09:12:43.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538374+0000", 2024-09-13T09:12:43.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538373+0000", 2024-09-13T09:12:43.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538376+0000", 2024-09-13T09:12:43.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4279999999999999e-06 2024-09-13T09:12:43.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538382+0000", 2024-09-13T09:12:43.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4169999999999997e-06 2024-09-13T09:12:43.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538389+0000", 2024-09-13T09:12:43.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:12:43.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538718+0000", 2024-09-13T09:12:43.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00032951500000000001 2024-09-13T09:12:43.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538726+0000", 2024-09-13T09:12:43.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3730000000000004e-06 2024-09-13T09:12:43.611 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.611 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.611 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.611 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20335 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.611 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.541080+0000", 2024-09-13T09:12:43.611 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32370965099999999, 2024-09-13T09:12:43.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32928843899999999, 2024-09-13T09:12:43.611 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.611 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.612 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.612 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.612 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.612 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20335 2024-09-13T09:12:43.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.612 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541080+0000", 2024-09-13T09:12:43.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541080+0000", 2024-09-13T09:12:43.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541079+0000", 2024-09-13T09:12:43.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541081+0000", 2024-09-13T09:12:43.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.154e-06 2024-09-13T09:12:43.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541087+0000", 2024-09-13T09:12:43.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9710000000000001e-06 2024-09-13T09:12:43.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541092+0000", 2024-09-13T09:12:43.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7559999999999999e-06 2024-09-13T09:12:43.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542011+0000", 2024-09-13T09:12:43.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00091921500000000003 2024-09-13T09:12:43.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542018+0000", 2024-09-13T09:12:43.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9380000000000003e-06 2024-09-13T09:12:43.616 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.616 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.616 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.616 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20446 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.616 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543098+0000", 2024-09-13T09:12:43.616 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32169133799999999, 2024-09-13T09:12:43.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32730333700000003, 2024-09-13T09:12:43.616 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.616 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.617 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.617 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.617 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.617 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20446 2024-09-13T09:12:43.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.617 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543098+0000", 2024-09-13T09:12:43.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543098+0000", 2024-09-13T09:12:43.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543097+0000", 2024-09-13T09:12:43.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543099+0000", 2024-09-13T09:12:43.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:12:43.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543105+0000", 2024-09-13T09:12:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5650000000000001e-06 2024-09-13T09:12:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543109+0000", 2024-09-13T09:12:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2359999999999998e-06 2024-09-13T09:12:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543461+0000", 2024-09-13T09:12:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035159200000000002 2024-09-13T09:12:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543468+0000", 2024-09-13T09:12:43.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:12:43.620 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.620 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.620 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.620 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20591 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.620 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547831+0000", 2024-09-13T09:12:43.620 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31695768000000002, 2024-09-13T09:12:43.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32260238099999999, 2024-09-13T09:12:43.620 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.621 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.621 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.621 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.621 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.621 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20591 2024-09-13T09:12:43.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.621 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547831+0000", 2024-09-13T09:12:43.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547831+0000", 2024-09-13T09:12:43.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547831+0000", 2024-09-13T09:12:43.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547834+0000", 2024-09-13T09:12:43.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.224e-06 2024-09-13T09:12:43.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547843+0000", 2024-09-13T09:12:43.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4379999999999995e-06 2024-09-13T09:12:43.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547848+0000", 2024-09-13T09:12:43.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.575e-06 2024-09-13T09:12:43.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548165+0000", 2024-09-13T09:12:43.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00031634000000000002 2024-09-13T09:12:43.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548174+0000", 2024-09-13T09:12:43.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0699999999999996e-06 2024-09-13T09:12:43.624 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.624 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.624 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.625 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20707 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.625 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549481+0000", 2024-09-13T09:12:43.625 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31530788100000001, 2024-09-13T09:12:43.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32098528599999998, 2024-09-13T09:12:43.625 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.625 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.625 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.625 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.625 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.625 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20707 2024-09-13T09:12:43.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.625 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549481+0000", 2024-09-13T09:12:43.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549481+0000", 2024-09-13T09:12:43.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549481+0000", 2024-09-13T09:12:43.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549483+0000", 2024-09-13T09:12:43.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5339999999999998e-06 2024-09-13T09:12:43.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549490+0000", 2024-09-13T09:12:43.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7939999999999997e-06 2024-09-13T09:12:43.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549495+0000", 2024-09-13T09:12:43.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0540000000000002e-06 2024-09-13T09:12:43.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549558+0000", 2024-09-13T09:12:43.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2514000000000005e-05 2024-09-13T09:12:43.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549566+0000", 2024-09-13T09:12:43.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3019999999999995e-06 2024-09-13T09:12:43.628 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.628 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.628 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.629 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19831 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.629 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.522211+0000", 2024-09-13T09:12:43.629 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34257840099999998, 2024-09-13T09:12:43.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34828905999999998, 2024-09-13T09:12:43.629 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.629 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.629 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.629 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.629 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.629 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19831 2024-09-13T09:12:43.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.629 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522211+0000", 2024-09-13T09:12:43.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522211+0000", 2024-09-13T09:12:43.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522210+0000", 2024-09-13T09:12:43.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522212+0000", 2024-09-13T09:12:43.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:12:43.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522217+0000", 2024-09-13T09:12:43.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.622e-06 2024-09-13T09:12:43.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522226+0000", 2024-09-13T09:12:43.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0589999999999992e-06 2024-09-13T09:12:43.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533452+0000", 2024-09-13T09:12:43.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011226165 2024-09-13T09:12:43.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533460+0000", 2024-09-13T09:12:43.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5279999999999998e-06 2024-09-13T09:12:43.632 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.632 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.632 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.633 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19941 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.633 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524598+0000", 2024-09-13T09:12:43.633 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34019089000000002, 2024-09-13T09:12:43.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34593467, 2024-09-13T09:12:43.633 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.633 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.633 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.633 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.633 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.633 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19941 2024-09-13T09:12:43.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.633 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524598+0000", 2024-09-13T09:12:43.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524598+0000", 2024-09-13T09:12:43.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524598+0000", 2024-09-13T09:12:43.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524600+0000", 2024-09-13T09:12:43.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:12:43.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524606+0000", 2024-09-13T09:12:43.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4820000000000002e-06 2024-09-13T09:12:43.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524612+0000", 2024-09-13T09:12:43.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0059999999999998e-06 2024-09-13T09:12:43.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534306+0000", 2024-09-13T09:12:43.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096945490000000002 2024-09-13T09:12:43.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534314+0000", 2024-09-13T09:12:43.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5059999999999998e-06 2024-09-13T09:12:43.636 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.637 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.637 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.637 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.637 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:12:43.637 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33581285100000002, 2024-09-13T09:12:43.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.341589847, 2024-09-13T09:12:43.637 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.637 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.637 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.637 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.637 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.637 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:12:43.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.638 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:12:43.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:12:43.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:12:43.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:12:43.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:12:43.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:12:43.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:12:43.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:12:43.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:12:43.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:12:43.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:12:43.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:12:43.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:12:43.640 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.640 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.640 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.641 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20193 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.641 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538399+0000", 2024-09-13T09:12:43.641 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32639038199999998, 2024-09-13T09:12:43.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.332200261, 2024-09-13T09:12:43.641 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.641 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.641 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.641 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.641 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.641 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20193 2024-09-13T09:12:43.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.641 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538399+0000", 2024-09-13T09:12:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538399+0000", 2024-09-13T09:12:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538398+0000", 2024-09-13T09:12:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538401+0000", 2024-09-13T09:12:43.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.694e-06 2024-09-13T09:12:43.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538407+0000", 2024-09-13T09:12:43.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7339999999999996e-06 2024-09-13T09:12:43.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538411+0000", 2024-09-13T09:12:43.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2209999999999999e-06 2024-09-13T09:12:43.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538747+0000", 2024-09-13T09:12:43.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033589500000000001 2024-09-13T09:12:43.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538754+0000", 2024-09-13T09:12:43.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2010000000000002e-06 2024-09-13T09:12:43.644 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.644 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.644 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.645 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20336 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.645 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.541097+0000", 2024-09-13T09:12:43.645 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32369265800000002, 2024-09-13T09:12:43.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.329535578, 2024-09-13T09:12:43.645 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.645 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.645 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.645 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.645 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.645 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20336 2024-09-13T09:12:43.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.646 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541097+0000", 2024-09-13T09:12:43.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541097+0000", 2024-09-13T09:12:43.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541096+0000", 2024-09-13T09:12:43.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541099+0000", 2024-09-13T09:12:43.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5500000000000001e-06 2024-09-13T09:12:43.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541105+0000", 2024-09-13T09:12:43.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4949999999999996e-06 2024-09-13T09:12:43.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541110+0000", 2024-09-13T09:12:43.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5310000000000003e-06 2024-09-13T09:12:43.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542042+0000", 2024-09-13T09:12:43.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00093276600000000002 2024-09-13T09:12:43.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542050+0000", 2024-09-13T09:12:43.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5730000000000002e-06 2024-09-13T09:12:43.648 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.649 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.649 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.649 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20447 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.649 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543113+0000", 2024-09-13T09:12:43.649 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32167604, 2024-09-13T09:12:43.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32755239800000002, 2024-09-13T09:12:43.649 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.649 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.649 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.649 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.649 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.649 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20447 2024-09-13T09:12:43.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.650 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543113+0000", 2024-09-13T09:12:43.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543113+0000", 2024-09-13T09:12:43.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543113+0000", 2024-09-13T09:12:43.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543124+0000", 2024-09-13T09:12:43.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1440999999999999e-05 2024-09-13T09:12:43.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543134+0000", 2024-09-13T09:12:43.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0020999999999999e-05 2024-09-13T09:12:43.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543139+0000", 2024-09-13T09:12:43.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:12:43.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543496+0000", 2024-09-13T09:12:43.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035717300000000002 2024-09-13T09:12:43.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543503+0000", 2024-09-13T09:12:43.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8070000000000002e-06 2024-09-13T09:12:43.653 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.653 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.653 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.653 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20593 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.653 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547910+0000", 2024-09-13T09:12:43.653 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31687870099999998, 2024-09-13T09:12:43.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32278784799999999, 2024-09-13T09:12:43.653 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.653 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.653 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.654 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.654 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.654 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20593 2024-09-13T09:12:43.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.654 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547910+0000", 2024-09-13T09:12:43.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547910+0000", 2024-09-13T09:12:43.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547910+0000", 2024-09-13T09:12:43.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547914+0000", 2024-09-13T09:12:43.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.72e-06 2024-09-13T09:12:43.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547924+0000", 2024-09-13T09:12:43.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0446e-05 2024-09-13T09:12:43.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547930+0000", 2024-09-13T09:12:43.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:12:43.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548212+0000", 2024-09-13T09:12:43.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00028146600000000002 2024-09-13T09:12:43.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548220+0000", 2024-09-13T09:12:43.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0190000000000006e-06 2024-09-13T09:12:43.657 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.657 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.657 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.657 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20708 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.657 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549516+0000", 2024-09-13T09:12:43.657 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31527270899999998, 2024-09-13T09:12:43.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32121501099999999, 2024-09-13T09:12:43.658 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.658 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.658 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.658 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.658 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.658 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20708 2024-09-13T09:12:43.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.658 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549516+0000", 2024-09-13T09:12:43.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549516+0000", 2024-09-13T09:12:43.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549516+0000", 2024-09-13T09:12:43.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549525+0000", 2024-09-13T09:12:43.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2860000000000005e-06 2024-09-13T09:12:43.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549532+0000", 2024-09-13T09:12:43.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0759999999999997e-06 2024-09-13T09:12:43.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549538+0000", 2024-09-13T09:12:43.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:12:43.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549594+0000", 2024-09-13T09:12:43.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6496999999999998e-05 2024-09-13T09:12:43.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549603+0000", 2024-09-13T09:12:43.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9190000000000005e-06 2024-09-13T09:12:43.661 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.661 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.661 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.661 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19832 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.661 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.522230+0000", 2024-09-13T09:12:43.662 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34255920099999998, 2024-09-13T09:12:43.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.348534862, 2024-09-13T09:12:43.662 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.662 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.662 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.662 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.662 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.662 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19832 2024-09-13T09:12:43.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.662 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522230+0000", 2024-09-13T09:12:43.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522230+0000", 2024-09-13T09:12:43.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522229+0000", 2024-09-13T09:12:43.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522232+0000", 2024-09-13T09:12:43.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0899999999999999e-06 2024-09-13T09:12:43.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522236+0000", 2024-09-13T09:12:43.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.916e-06 2024-09-13T09:12:43.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522240+0000", 2024-09-13T09:12:43.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5250000000000001e-06 2024-09-13T09:12:43.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533485+0000", 2024-09-13T09:12:43.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011244762 2024-09-13T09:12:43.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533492+0000", 2024-09-13T09:12:43.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9430000000000003e-06 2024-09-13T09:12:43.665 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.665 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.665 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.665 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19942 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.666 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524617+0000", 2024-09-13T09:12:43.666 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34017230999999998, 2024-09-13T09:12:43.666 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34618126399999999, 2024-09-13T09:12:43.666 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.666 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.666 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.666 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.666 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.666 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19942 2024-09-13T09:12:43.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.666 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.666 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.666 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524617+0000", 2024-09-13T09:12:43.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524617+0000", 2024-09-13T09:12:43.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524616+0000", 2024-09-13T09:12:43.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524619+0000", 2024-09-13T09:12:43.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3939999999999999e-06 2024-09-13T09:12:43.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524626+0000", 2024-09-13T09:12:43.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9210000000000003e-06 2024-09-13T09:12:43.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524638+0000", 2024-09-13T09:12:43.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2159e-05 2024-09-13T09:12:43.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534336+0000", 2024-09-13T09:12:43.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096983499999999997 2024-09-13T09:12:43.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534360+0000", 2024-09-13T09:12:43.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4264999999999999e-05 2024-09-13T09:12:43.669 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.669 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.669 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.670 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.670 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:12:43.670 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.335766546, 2024-09-13T09:12:43.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34181117799999999, 2024-09-13T09:12:43.670 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.670 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.670 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.670 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.670 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.670 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:12:43.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.670 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:12:43.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:12:43.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:12:43.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:12:43.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:12:43.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:12:43.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:12:43.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.673 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.673 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:12:43.673 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:12:43.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.673 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.673 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:12:43.673 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:12:43.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.673 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.673 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:12:43.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:12:43.674 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.674 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.674 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.674 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20194 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.674 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538415+0000", 2024-09-13T09:12:43.674 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32637426200000003, 2024-09-13T09:12:43.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33245213299999998, 2024-09-13T09:12:43.674 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.674 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.674 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.674 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.675 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.675 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20194 2024-09-13T09:12:43.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.675 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538415+0000", 2024-09-13T09:12:43.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538415+0000", 2024-09-13T09:12:43.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538415+0000", 2024-09-13T09:12:43.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538417+0000", 2024-09-13T09:12:43.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:12:43.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538422+0000", 2024-09-13T09:12:43.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7259999999999999e-06 2024-09-13T09:12:43.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538428+0000", 2024-09-13T09:12:43.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7429999999999999e-06 2024-09-13T09:12:43.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538779+0000", 2024-09-13T09:12:43.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000351219 2024-09-13T09:12:43.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538786+0000", 2024-09-13T09:12:43.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7580000000000003e-06 2024-09-13T09:12:43.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.678 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.678 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20337 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.678 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.541114+0000", 2024-09-13T09:12:43.678 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32367538899999998, 2024-09-13T09:12:43.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.329785948, 2024-09-13T09:12:43.678 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.679 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.679 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20337 2024-09-13T09:12:43.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.679 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.679 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.679 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541114+0000", 2024-09-13T09:12:43.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541114+0000", 2024-09-13T09:12:43.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541113+0000", 2024-09-13T09:12:43.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541116+0000", 2024-09-13T09:12:43.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1100000000000001e-06 2024-09-13T09:12:43.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541121+0000", 2024-09-13T09:12:43.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5829999999999997e-06 2024-09-13T09:12:43.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541126+0000", 2024-09-13T09:12:43.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9450000000000001e-06 2024-09-13T09:12:43.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542076+0000", 2024-09-13T09:12:43.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00094991400000000005 2024-09-13T09:12:43.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542083+0000", 2024-09-13T09:12:43.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8789999999999997e-06 2024-09-13T09:12:43.682 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.682 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.682 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.682 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20448 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.682 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543144+0000", 2024-09-13T09:12:43.682 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32164521699999998, 2024-09-13T09:12:43.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32778876099999998, 2024-09-13T09:12:43.682 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.683 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.683 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.683 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.683 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.683 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20448 2024-09-13T09:12:43.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.683 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543144+0000", 2024-09-13T09:12:43.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543144+0000", 2024-09-13T09:12:43.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543144+0000", 2024-09-13T09:12:43.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543146+0000", 2024-09-13T09:12:43.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0509999999999999e-06 2024-09-13T09:12:43.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543152+0000", 2024-09-13T09:12:43.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2890000000000003e-06 2024-09-13T09:12:43.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543156+0000", 2024-09-13T09:12:43.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3900000000000003e-06 2024-09-13T09:12:43.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543534+0000", 2024-09-13T09:12:43.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00037729099999999998 2024-09-13T09:12:43.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.686 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.686 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543540+0000", 2024-09-13T09:12:43.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6749999999999996e-06 2024-09-13T09:12:43.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.686 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.686 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20595 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.686 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547935+0000", 2024-09-13T09:12:43.687 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31685386199999999, 2024-09-13T09:12:43.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32303067299999999, 2024-09-13T09:12:43.687 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.687 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.687 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.687 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.687 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.687 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20595 2024-09-13T09:12:43.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.688 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547935+0000", 2024-09-13T09:12:43.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547935+0000", 2024-09-13T09:12:43.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547935+0000", 2024-09-13T09:12:43.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547942+0000", 2024-09-13T09:12:43.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9410000000000001e-06 2024-09-13T09:12:43.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547950+0000", 2024-09-13T09:12:43.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9559999999999997e-06 2024-09-13T09:12:43.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547958+0000", 2024-09-13T09:12:43.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.405e-06 2024-09-13T09:12:43.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548264+0000", 2024-09-13T09:12:43.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000305834 2024-09-13T09:12:43.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548273+0000", 2024-09-13T09:12:43.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5299999999999996e-06 2024-09-13T09:12:43.691 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.691 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.691 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.691 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20709 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.691 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549552+0000", 2024-09-13T09:12:43.691 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31523668500000002, 2024-09-13T09:12:43.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32144661899999999, 2024-09-13T09:12:43.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.691 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.692 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.692 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20709 2024-09-13T09:12:43.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.692 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549552+0000", 2024-09-13T09:12:43.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549552+0000", 2024-09-13T09:12:43.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549552+0000", 2024-09-13T09:12:43.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549555+0000", 2024-09-13T09:12:43.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.819e-06 2024-09-13T09:12:43.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549562+0000", 2024-09-13T09:12:43.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0049999999999998e-06 2024-09-13T09:12:43.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549568+0000", 2024-09-13T09:12:43.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:12:43.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549631+0000", 2024-09-13T09:12:43.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2982000000000001e-05 2024-09-13T09:12:43.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549639+0000", 2024-09-13T09:12:43.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7069999999999992e-06 2024-09-13T09:12:43.695 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.695 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.695 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.695 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19833 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.695 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.522244+0000", 2024-09-13T09:12:43.695 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34254559099999998, 2024-09-13T09:12:43.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34878915500000002, 2024-09-13T09:12:43.695 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.695 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.695 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.695 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.696 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.696 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19833 2024-09-13T09:12:43.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.696 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522244+0000", 2024-09-13T09:12:43.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522244+0000", 2024-09-13T09:12:43.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522243+0000", 2024-09-13T09:12:43.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522245+0000", 2024-09-13T09:12:43.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9420000000000002e-06 2024-09-13T09:12:43.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522250+0000", 2024-09-13T09:12:43.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7500000000000003e-06 2024-09-13T09:12:43.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522258+0000", 2024-09-13T09:12:43.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6470000000000004e-06 2024-09-13T09:12:43.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533597+0000", 2024-09-13T09:12:43.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011338686000000001 2024-09-13T09:12:43.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533608+0000", 2024-09-13T09:12:43.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1133e-05 2024-09-13T09:12:43.699 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.699 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.699 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.699 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19943 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.699 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524643+0000", 2024-09-13T09:12:43.699 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34014643300000003, 2024-09-13T09:12:43.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34642310999999998, 2024-09-13T09:12:43.699 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.699 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.700 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.700 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.700 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.700 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19943 2024-09-13T09:12:43.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.700 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524643+0000", 2024-09-13T09:12:43.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524643+0000", 2024-09-13T09:12:43.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524642+0000", 2024-09-13T09:12:43.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524645+0000", 2024-09-13T09:12:43.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:12:43.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524651+0000", 2024-09-13T09:12:43.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1380000000000004e-06 2024-09-13T09:12:43.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524655+0000", 2024-09-13T09:12:43.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1810000000000003e-06 2024-09-13T09:12:43.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534389+0000", 2024-09-13T09:12:43.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0097339680000000008 2024-09-13T09:12:43.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534396+0000", 2024-09-13T09:12:43.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0360000000000001e-06 2024-09-13T09:12:43.703 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.703 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.703 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.703 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.703 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:12:43.704 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33574124500000002, 2024-09-13T09:12:43.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34205122399999999, 2024-09-13T09:12:43.704 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.704 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.704 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.704 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.704 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.704 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:12:43.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.704 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:12:43.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:12:43.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:12:43.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:12:43.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:12:43.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:12:43.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:12:43.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:12:43.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.707 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20195 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.708 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538432+0000", 2024-09-13T09:12:43.708 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.326356912, 2024-09-13T09:12:43.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33269970799999998, 2024-09-13T09:12:43.708 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.708 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.708 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.708 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.708 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.708 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20195 2024-09-13T09:12:43.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.708 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538432+0000", 2024-09-13T09:12:43.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538432+0000", 2024-09-13T09:12:43.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538432+0000", 2024-09-13T09:12:43.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538440+0000", 2024-09-13T09:12:43.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0940000000000008e-06 2024-09-13T09:12:43.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538446+0000", 2024-09-13T09:12:43.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.088e-06 2024-09-13T09:12:43.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538456+0000", 2024-09-13T09:12:43.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8609999999999993e-06 2024-09-13T09:12:43.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538832+0000", 2024-09-13T09:12:43.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000376084 2024-09-13T09:12:43.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538840+0000", 2024-09-13T09:12:43.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2670000000000006e-06 2024-09-13T09:12:43.711 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.711 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.711 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.712 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20338 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.712 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.541136+0000", 2024-09-13T09:12:43.712 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32365302000000001, 2024-09-13T09:12:43.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33002870000000001, 2024-09-13T09:12:43.712 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.712 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.712 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.712 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.712 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.712 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20338 2024-09-13T09:12:43.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.713 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541136+0000", 2024-09-13T09:12:43.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541136+0000", 2024-09-13T09:12:43.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541136+0000", 2024-09-13T09:12:43.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541138+0000", 2024-09-13T09:12:43.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.559e-06 2024-09-13T09:12:43.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541144+0000", 2024-09-13T09:12:43.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1530000000000002e-06 2024-09-13T09:12:43.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541150+0000", 2024-09-13T09:12:43.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1950000000000002e-06 2024-09-13T09:12:43.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542110+0000", 2024-09-13T09:12:43.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00096005099999999996 2024-09-13T09:12:43.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542117+0000", 2024-09-13T09:12:43.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0350000000000004e-06 2024-09-13T09:12:43.716 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.716 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.716 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.716 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20449 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.716 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543160+0000", 2024-09-13T09:12:43.716 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32162875200000002, 2024-09-13T09:12:43.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.328037668, 2024-09-13T09:12:43.716 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.716 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.716 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.716 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.717 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.717 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20449 2024-09-13T09:12:43.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.717 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543160+0000", 2024-09-13T09:12:43.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.718 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.718 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543160+0000", 2024-09-13T09:12:43.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.718 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.718 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543160+0000", 2024-09-13T09:12:43.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543162+0000", 2024-09-13T09:12:43.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.154e-06 2024-09-13T09:12:43.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543168+0000", 2024-09-13T09:12:43.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5659999999999997e-06 2024-09-13T09:12:43.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543173+0000", 2024-09-13T09:12:43.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8799999999999996e-06 2024-09-13T09:12:43.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543574+0000", 2024-09-13T09:12:43.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00040009300000000002 2024-09-13T09:12:43.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543580+0000", 2024-09-13T09:12:43.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8510000000000001e-06 2024-09-13T09:12:43.720 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.720 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.720 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.721 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20596 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.721 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547979+0000", 2024-09-13T09:12:43.721 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31681021399999998, 2024-09-13T09:12:43.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32325246400000002, 2024-09-13T09:12:43.721 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.721 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.721 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.721 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.721 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.721 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20596 2024-09-13T09:12:43.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.721 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547979+0000", 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547979+0000", 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547978+0000", 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547981+0000", 2024-09-13T09:12:43.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6000000000000001e-06 2024-09-13T09:12:43.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547988+0000", 2024-09-13T09:12:43.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5509999999999996e-06 2024-09-13T09:12:43.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547992+0000", 2024-09-13T09:12:43.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4229999999999998e-06 2024-09-13T09:12:43.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548305+0000", 2024-09-13T09:12:43.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00031277300000000002 2024-09-13T09:12:43.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548313+0000", 2024-09-13T09:12:43.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4570000000000005e-06 2024-09-13T09:12:43.724 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.724 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.724 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.725 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20710 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.725 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549588+0000", 2024-09-13T09:12:43.725 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.315200755, 2024-09-13T09:12:43.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32167627100000001, 2024-09-13T09:12:43.725 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.725 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.725 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.725 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.725 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.725 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20710 2024-09-13T09:12:43.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.726 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549588+0000", 2024-09-13T09:12:43.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549588+0000", 2024-09-13T09:12:43.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549588+0000", 2024-09-13T09:12:43.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549590+0000", 2024-09-13T09:12:43.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4269999999999998e-06 2024-09-13T09:12:43.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549597+0000", 2024-09-13T09:12:43.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.793e-06 2024-09-13T09:12:43.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549604+0000", 2024-09-13T09:12:43.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.584e-06 2024-09-13T09:12:43.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549665+0000", 2024-09-13T09:12:43.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0989999999999997e-05 2024-09-13T09:12:43.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549672+0000", 2024-09-13T09:12:43.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6499999999999996e-06 2024-09-13T09:12:43.729 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.729 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.729 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.729 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19834 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.729 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.522262+0000", 2024-09-13T09:12:43.729 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34252682600000001, 2024-09-13T09:12:43.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34903569499999998, 2024-09-13T09:12:43.729 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.729 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.730 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.730 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.730 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.730 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19834 2024-09-13T09:12:43.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.730 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522262+0000", 2024-09-13T09:12:43.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522262+0000", 2024-09-13T09:12:43.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522262+0000", 2024-09-13T09:12:43.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.732 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522264+0000", 2024-09-13T09:12:43.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.956e-06 2024-09-13T09:12:43.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.732 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.732 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522269+0000", 2024-09-13T09:12:43.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4920000000000001e-06 2024-09-13T09:12:43.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.732 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.732 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.522273+0000", 2024-09-13T09:12:43.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8800000000000001e-06 2024-09-13T09:12:43.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.733 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.733 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533658+0000", 2024-09-13T09:12:43.733 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011384905000000001 2024-09-13T09:12:43.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.733 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.733 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.533667+0000", 2024-09-13T09:12:43.733 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3110000000000006e-06 2024-09-13T09:12:43.733 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.733 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.733 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.734 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19944 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.734 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524660+0000", 2024-09-13T09:12:43.734 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34012965299999998, 2024-09-13T09:12:43.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34667199900000001, 2024-09-13T09:12:43.734 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.734 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.734 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.734 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.734 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.734 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19944 2024-09-13T09:12:43.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.735 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524660+0000", 2024-09-13T09:12:43.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524660+0000", 2024-09-13T09:12:43.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524659+0000", 2024-09-13T09:12:43.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524661+0000", 2024-09-13T09:12:43.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1569999999999998e-06 2024-09-13T09:12:43.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524668+0000", 2024-09-13T09:12:43.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2639999999999997e-06 2024-09-13T09:12:43.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.737 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524672+0000", 2024-09-13T09:12:43.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3930000000000001e-06 2024-09-13T09:12:43.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.737 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534421+0000", 2024-09-13T09:12:43.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0097489629999999994 2024-09-13T09:12:43.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534427+0000", 2024-09-13T09:12:43.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:12:43.738 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.738 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.738 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.738 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.738 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:12:43.738 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33572301599999999, 2024-09-13T09:12:43.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34229818299999998, 2024-09-13T09:12:43.738 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.739 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.739 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.739 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.739 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.739 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:12:43.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.739 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.739 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.739 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:12:43.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.739 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.740 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:12:43.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.740 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:12:43.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.740 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:12:43.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:12:43.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:12:43.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:12:43.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:12:43.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:12:43.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:12:43.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:12:43.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:12:43.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:12:43.742 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.742 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.742 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.743 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20196 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.743 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538460+0000", 2024-09-13T09:12:43.743 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32632892400000002, 2024-09-13T09:12:43.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33293660600000002, 2024-09-13T09:12:43.743 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.743 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.743 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.743 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.743 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.743 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20196 2024-09-13T09:12:43.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.743 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.744 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538460+0000", 2024-09-13T09:12:43.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.744 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.744 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538460+0000", 2024-09-13T09:12:43.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.744 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.744 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538460+0000", 2024-09-13T09:12:43.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538462+0000", 2024-09-13T09:12:43.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:12:43.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538468+0000", 2024-09-13T09:12:43.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6699999999999999e-06 2024-09-13T09:12:43.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538472+0000", 2024-09-13T09:12:43.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4499999999999997e-06 2024-09-13T09:12:43.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538865+0000", 2024-09-13T09:12:43.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00039235499999999999 2024-09-13T09:12:43.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538872+0000", 2024-09-13T09:12:43.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:12:43.746 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.747 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.747 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.747 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20339 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.747 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.541154+0000", 2024-09-13T09:12:43.747 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32363492100000002, 2024-09-13T09:12:43.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33027522599999998, 2024-09-13T09:12:43.747 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.747 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.747 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.747 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.747 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.748 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20339 2024-09-13T09:12:43.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.748 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.748 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.748 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541154+0000", 2024-09-13T09:12:43.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.748 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.748 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541154+0000", 2024-09-13T09:12:43.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.749 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.749 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541154+0000", 2024-09-13T09:12:43.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.749 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.749 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541162+0000", 2024-09-13T09:12:43.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3410000000000003e-06 2024-09-13T09:12:43.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541169+0000", 2024-09-13T09:12:43.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.635e-06 2024-09-13T09:12:43.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541174+0000", 2024-09-13T09:12:43.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8450000000000002e-06 2024-09-13T09:12:43.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542141+0000", 2024-09-13T09:12:43.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00096779699999999995 2024-09-13T09:12:43.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542148+0000", 2024-09-13T09:12:43.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8800000000000002e-06 2024-09-13T09:12:43.751 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.751 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.751 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.751 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20450 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.751 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543177+0000", 2024-09-13T09:12:43.751 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32161167099999999, 2024-09-13T09:12:43.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32828522399999999, 2024-09-13T09:12:43.752 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.752 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.752 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.752 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.752 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.752 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20450 2024-09-13T09:12:43.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.752 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.752 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543177+0000", 2024-09-13T09:12:43.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.753 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543177+0000", 2024-09-13T09:12:43.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.753 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543177+0000", 2024-09-13T09:12:43.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543179+0000", 2024-09-13T09:12:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9319999999999999e-06 2024-09-13T09:12:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543184+0000", 2024-09-13T09:12:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2650000000000004e-06 2024-09-13T09:12:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543189+0000", 2024-09-13T09:12:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4490000000000001e-06 2024-09-13T09:12:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543674+0000", 2024-09-13T09:12:43.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00048485299999999999 2024-09-13T09:12:43.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543683+0000", 2024-09-13T09:12:43.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.054e-06 2024-09-13T09:12:43.755 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.755 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.755 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.756 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20597 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.756 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547996+0000", 2024-09-13T09:12:43.756 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31679299900000002, 2024-09-13T09:12:43.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.323499971, 2024-09-13T09:12:43.756 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.756 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.756 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.756 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.756 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.756 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20597 2024-09-13T09:12:43.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.756 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547996+0000", 2024-09-13T09:12:43.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547996+0000", 2024-09-13T09:12:43.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547996+0000", 2024-09-13T09:12:43.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547998+0000", 2024-09-13T09:12:43.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2759999999999999e-06 2024-09-13T09:12:43.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548004+0000", 2024-09-13T09:12:43.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8660000000000002e-06 2024-09-13T09:12:43.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548008+0000", 2024-09-13T09:12:43.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5739999999999997e-06 2024-09-13T09:12:43.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548341+0000", 2024-09-13T09:12:43.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033242200000000002 2024-09-13T09:12:43.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548349+0000", 2024-09-13T09:12:43.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8769999999999993e-06 2024-09-13T09:12:43.759 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.759 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.760 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.760 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20711 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.760 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549625+0000", 2024-09-13T09:12:43.760 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31516374600000002, 2024-09-13T09:12:43.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32190378200000003, 2024-09-13T09:12:43.760 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.760 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.760 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.760 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.760 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.761 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20711 2024-09-13T09:12:43.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.761 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.761 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.761 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.761 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549625+0000", 2024-09-13T09:12:43.761 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.761 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.761 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.761 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549625+0000", 2024-09-13T09:12:43.761 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.761 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.762 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.762 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549625+0000", 2024-09-13T09:12:43.762 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.762 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.762 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549628+0000", 2024-09-13T09:12:43.762 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4990000000000001e-06 2024-09-13T09:12:43.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.762 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.762 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549635+0000", 2024-09-13T09:12:43.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.469e-06 2024-09-13T09:12:43.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.763 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549642+0000", 2024-09-13T09:12:43.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9149999999999999e-06 2024-09-13T09:12:43.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.763 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549699+0000", 2024-09-13T09:12:43.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7021000000000002e-05 2024-09-13T09:12:43.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.764 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549707+0000", 2024-09-13T09:12:43.764 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6410000000000002e-06 2024-09-13T09:12:43.764 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.764 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.764 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.764 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19945 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.764 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524676+0000", 2024-09-13T09:12:43.764 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34011311799999999, 2024-09-13T09:12:43.764 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34688639199999999, 2024-09-13T09:12:43.764 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.764 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.765 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.765 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.765 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.765 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19945 2024-09-13T09:12:43.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.765 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.765 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.765 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524676+0000", 2024-09-13T09:12:43.765 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.765 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.766 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524676+0000", 2024-09-13T09:12:43.766 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.766 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.766 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524676+0000", 2024-09-13T09:12:43.766 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.766 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.766 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524687+0000", 2024-09-13T09:12:43.766 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1673e-05 2024-09-13T09:12:43.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524695+0000", 2024-09-13T09:12:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2649999999999999e-06 2024-09-13T09:12:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524700+0000", 2024-09-13T09:12:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9450000000000001e-06 2024-09-13T09:12:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534452+0000", 2024-09-13T09:12:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0097520030000000004 2024-09-13T09:12:43.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534459+0000", 2024-09-13T09:12:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9330000000000004e-06 2024-09-13T09:12:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:12:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.335675697, 2024-09-13T09:12:43.768 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34248152300000001, 2024-09-13T09:12:43.769 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.769 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.769 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.769 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.769 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.769 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:12:43.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.769 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.769 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.769 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:12:43.769 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:12:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:12:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.770 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:12:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:12:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:12:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:12:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:12:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:12:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:12:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:12:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:12:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:12:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20197 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538476+0000", 2024-09-13T09:12:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32631276799999998, 2024-09-13T09:12:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33315165299999999, 2024-09-13T09:12:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20197 2024-09-13T09:12:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.773 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538476+0000", 2024-09-13T09:12:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538476+0000", 2024-09-13T09:12:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538476+0000", 2024-09-13T09:12:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.775 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.775 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538478+0000", 2024-09-13T09:12:43.775 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9539999999999998e-06 2024-09-13T09:12:43.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.775 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.775 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538483+0000", 2024-09-13T09:12:43.775 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:12:43.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.775 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.775 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538487+0000", 2024-09-13T09:12:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6059999999999999e-06 2024-09-13T09:12:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538895+0000", 2024-09-13T09:12:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00040795399999999998 2024-09-13T09:12:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:12:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.481e-06 2024-09-13T09:12:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.776 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20340 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.541178+0000", 2024-09-13T09:12:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32361114400000002, 2024-09-13T09:12:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33048295900000002, 2024-09-13T09:12:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.777 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20340 2024-09-13T09:12:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541178+0000", 2024-09-13T09:12:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541178+0000", 2024-09-13T09:12:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541178+0000", 2024-09-13T09:12:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541180+0000", 2024-09-13T09:12:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2060000000000001e-06 2024-09-13T09:12:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541186+0000", 2024-09-13T09:12:43.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1519999999999997e-06 2024-09-13T09:12:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541191+0000", 2024-09-13T09:12:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:12:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542172+0000", 2024-09-13T09:12:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098181700000000006 2024-09-13T09:12:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.780 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542179+0000", 2024-09-13T09:12:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9210000000000003e-06 2024-09-13T09:12:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20451 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543208+0000", 2024-09-13T09:12:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32158067099999998, 2024-09-13T09:12:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32848560999999998, 2024-09-13T09:12:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.781 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20451 2024-09-13T09:12:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543208+0000", 2024-09-13T09:12:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.782 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543208+0000", 2024-09-13T09:12:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543208+0000", 2024-09-13T09:12:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543211+0000", 2024-09-13T09:12:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4819999999999999e-06 2024-09-13T09:12:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543218+0000", 2024-09-13T09:12:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2469999999999997e-06 2024-09-13T09:12:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543223+0000", 2024-09-13T09:12:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0259999999999998e-06 2024-09-13T09:12:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543718+0000", 2024-09-13T09:12:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049518800000000003 2024-09-13T09:12:43.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543725+0000", 2024-09-13T09:12:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4939999999999997e-06 2024-09-13T09:12:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20598 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.548024+0000", 2024-09-13T09:12:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.316765201, 2024-09-13T09:12:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32370338900000001, 2024-09-13T09:12:43.785 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20598 2024-09-13T09:12:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548024+0000", 2024-09-13T09:12:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.786 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548024+0000", 2024-09-13T09:12:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548024+0000", 2024-09-13T09:12:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548033+0000", 2024-09-13T09:12:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4269999999999997e-06 2024-09-13T09:12:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.788 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.788 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548040+0000", 2024-09-13T09:12:43.788 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8770000000000004e-06 2024-09-13T09:12:43.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.788 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.788 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548044+0000", 2024-09-13T09:12:43.788 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5539999999999999e-06 2024-09-13T09:12:43.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548386+0000", 2024-09-13T09:12:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034201199999999999 2024-09-13T09:12:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548404+0000", 2024-09-13T09:12:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7850999999999999e-05 2024-09-13T09:12:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20712 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549660+0000", 2024-09-13T09:12:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31512910199999999, 2024-09-13T09:12:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32210013799999998, 2024-09-13T09:12:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20712 2024-09-13T09:12:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549660+0000", 2024-09-13T09:12:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549660+0000", 2024-09-13T09:12:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549660+0000", 2024-09-13T09:12:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549662+0000", 2024-09-13T09:12:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4980000000000001e-06 2024-09-13T09:12:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549669+0000", 2024-09-13T09:12:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8210000000000004e-06 2024-09-13T09:12:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.792 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549674+0000", 2024-09-13T09:12:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:12:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549734+0000", 2024-09-13T09:12:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0263999999999998e-05 2024-09-13T09:12:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549742+0000", 2024-09-13T09:12:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7360000000000001e-06 2024-09-13T09:12:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.793 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19946 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524705+0000", 2024-09-13T09:12:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.340084473, 2024-09-13T09:12:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.347088589, 2024-09-13T09:12:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19946 2024-09-13T09:12:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.794 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524705+0000", 2024-09-13T09:12:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524705+0000", 2024-09-13T09:12:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524704+0000", 2024-09-13T09:12:43.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524707+0000", 2024-09-13T09:12:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.542e-06 2024-09-13T09:12:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524713+0000", 2024-09-13T09:12:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3280000000000002e-06 2024-09-13T09:12:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.796 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524718+0000", 2024-09-13T09:12:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7720000000000002e-06 2024-09-13T09:12:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534486+0000", 2024-09-13T09:12:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0097688110000000005 2024-09-13T09:12:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534494+0000", 2024-09-13T09:12:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9950000000000005e-06 2024-09-13T09:12:43.797 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20056 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528484+0000", 2024-09-13T09:12:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33630565400000001, 2024-09-13T09:12:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34334546300000002, 2024-09-13T09:12:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.798 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20056 2024-09-13T09:12:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528484+0000", 2024-09-13T09:12:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528484+0000", 2024-09-13T09:12:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.799 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528483+0000", 2024-09-13T09:12:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528485+0000", 2024-09-13T09:12:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2960000000000001e-06 2024-09-13T09:12:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528491+0000", 2024-09-13T09:12:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:12:43.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.801 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528496+0000", 2024-09-13T09:12:43.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6500000000000004e-06 2024-09-13T09:12:43.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.801 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535060+0000", 2024-09-13T09:12:43.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0065642620000000004 2024-09-13T09:12:43.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.801 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535067+0000", 2024-09-13T09:12:43.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7399999999999998e-06 2024-09-13T09:12:43.802 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.802 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.802 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.802 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.802 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.802 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:12:43.802 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33516145899999999, 2024-09-13T09:12:43.802 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34223456800000002, 2024-09-13T09:12:43.802 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.802 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.802 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.802 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.802 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.802 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:12:43.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.803 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:12:43.803 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:12:43.803 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.804 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:12:43.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:43.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.804 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:12:43.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:12:43.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.804 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:12:43.805 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:12:43.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.805 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.805 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:12:43.805 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:12:43.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.805 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.805 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:12:43.805 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:12:43.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.806 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.806 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:12:43.806 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:12:43.806 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.806 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.806 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.806 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.806 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20198 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.806 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538491+0000", 2024-09-13T09:12:43.806 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32629837099999998, 2024-09-13T09:12:43.806 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.333404596, 2024-09-13T09:12:43.807 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.807 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.807 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.807 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.807 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.807 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20198 2024-09-13T09:12:43.807 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.807 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.807 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.807 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.807 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538491+0000", 2024-09-13T09:12:43.807 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.807 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.807 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.808 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.808 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538491+0000", 2024-09-13T09:12:43.808 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.808 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.808 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.808 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.808 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538490+0000", 2024-09-13T09:12:43.808 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.808 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.808 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.808 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.808 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538492+0000", 2024-09-13T09:12:43.808 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.03e-06 2024-09-13T09:12:43.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.809 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.809 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.809 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538498+0000", 2024-09-13T09:12:43.809 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3909999999999997e-06 2024-09-13T09:12:43.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.809 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.809 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.809 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538502+0000", 2024-09-13T09:12:43.809 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9940000000000002e-06 2024-09-13T09:12:43.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.809 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.809 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.809 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538926+0000", 2024-09-13T09:12:43.810 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00042400199999999998 2024-09-13T09:12:43.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.810 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.810 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.810 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538933+0000", 2024-09-13T09:12:43.810 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6760000000000007e-06 2024-09-13T09:12:43.810 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.810 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.810 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.810 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.810 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20341 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.810 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.541196+0000", 2024-09-13T09:12:43.811 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32359310400000002, 2024-09-13T09:12:43.811 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33073282399999998, 2024-09-13T09:12:43.811 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.811 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.811 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.811 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.811 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.811 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20341 2024-09-13T09:12:43.811 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.811 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.811 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.811 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.811 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541196+0000", 2024-09-13T09:12:43.811 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.812 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.812 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541196+0000", 2024-09-13T09:12:43.812 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.812 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.812 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541195+0000", 2024-09-13T09:12:43.812 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.812 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.812 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541198+0000", 2024-09-13T09:12:43.813 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.621e-06 2024-09-13T09:12:43.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.813 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.813 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541204+0000", 2024-09-13T09:12:43.813 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:12:43.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.813 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.813 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541220+0000", 2024-09-13T09:12:43.813 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5920999999999999e-05 2024-09-13T09:12:43.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.814 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.814 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542206+0000", 2024-09-13T09:12:43.814 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098555800000000009 2024-09-13T09:12:43.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.814 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.814 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542213+0000", 2024-09-13T09:12:43.814 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.109e-06 2024-09-13T09:12:43.814 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.814 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.814 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20452 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543241+0000", 2024-09-13T09:12:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.321547894, 2024-09-13T09:12:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32872080100000001, 2024-09-13T09:12:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20452 2024-09-13T09:12:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.815 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.816 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543241+0000", 2024-09-13T09:12:43.816 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.816 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.816 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.816 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543241+0000", 2024-09-13T09:12:43.816 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.816 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.816 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.816 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543241+0000", 2024-09-13T09:12:43.816 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.816 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.817 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.817 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543243+0000", 2024-09-13T09:12:43.817 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4889999999999998e-06 2024-09-13T09:12:43.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.817 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.817 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543250+0000", 2024-09-13T09:12:43.817 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:12:43.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.817 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.817 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543255+0000", 2024-09-13T09:12:43.817 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7149999999999997e-06 2024-09-13T09:12:43.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.818 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.818 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543762+0000", 2024-09-13T09:12:43.818 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00050772600000000001 2024-09-13T09:12:43.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.818 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.818 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543770+0000", 2024-09-13T09:12:43.818 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.452e-06 2024-09-13T09:12:43.818 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.818 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.818 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.819 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20599 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.819 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.548069+0000", 2024-09-13T09:12:43.819 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.316720322, 2024-09-13T09:12:43.819 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32392644100000001, 2024-09-13T09:12:43.819 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.819 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.819 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.819 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.819 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.819 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20599 2024-09-13T09:12:43.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.819 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.820 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.820 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548069+0000", 2024-09-13T09:12:43.820 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.820 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.820 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.820 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548069+0000", 2024-09-13T09:12:43.820 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.820 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.820 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.820 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:12:43.820 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.821 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.821 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548071+0000", 2024-09-13T09:12:43.821 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7350000000000001e-06 2024-09-13T09:12:43.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.821 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.821 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548078+0000", 2024-09-13T09:12:43.821 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8870000000000003e-06 2024-09-13T09:12:43.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.821 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.822 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548084+0000", 2024-09-13T09:12:43.822 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3500000000000002e-06 2024-09-13T09:12:43.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.822 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.822 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.822 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548444+0000", 2024-09-13T09:12:43.822 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035942299999999999 2024-09-13T09:12:43.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.822 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.822 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.822 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548448+0000", 2024-09-13T09:12:43.822 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7500000000000003e-06 2024-09-13T09:12:43.822 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.822 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.823 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.823 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.823 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20713 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.823 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549694+0000", 2024-09-13T09:12:43.823 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31509532800000001, 2024-09-13T09:12:43.823 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32233668999999998, 2024-09-13T09:12:43.823 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.823 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.823 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.823 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.823 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.823 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20713 2024-09-13T09:12:43.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.824 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.824 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.824 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549694+0000", 2024-09-13T09:12:43.824 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.824 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.824 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549694+0000", 2024-09-13T09:12:43.824 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.824 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.824 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549693+0000", 2024-09-13T09:12:43.825 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.825 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.825 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549696+0000", 2024-09-13T09:12:43.825 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4710000000000001e-06 2024-09-13T09:12:43.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.825 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.825 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549703+0000", 2024-09-13T09:12:43.825 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8770000000000004e-06 2024-09-13T09:12:43.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.826 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.826 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549708+0000", 2024-09-13T09:12:43.826 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:12:43.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.826 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.826 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.826 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549770+0000", 2024-09-13T09:12:43.826 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2446000000000005e-05 2024-09-13T09:12:43.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.826 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.826 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.826 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549780+0000", 2024-09-13T09:12:43.826 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6870000000000006e-06 2024-09-13T09:12:43.826 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.827 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.827 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.827 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.827 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19947 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.827 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524722+0000", 2024-09-13T09:12:43.827 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34006682799999999, 2024-09-13T09:12:43.827 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34734143000000001, 2024-09-13T09:12:43.827 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.827 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.827 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.827 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.827 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.828 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19947 2024-09-13T09:12:43.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.828 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.828 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.828 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524722+0000", 2024-09-13T09:12:43.828 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.828 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.828 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524722+0000", 2024-09-13T09:12:43.828 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.829 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.829 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524722+0000", 2024-09-13T09:12:43.829 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.829 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.829 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524724+0000", 2024-09-13T09:12:43.829 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3089999999999998e-06 2024-09-13T09:12:43.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.829 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.829 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524730+0000", 2024-09-13T09:12:43.830 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:12:43.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.830 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.830 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524734+0000", 2024-09-13T09:12:43.830 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2509999999999996e-06 2024-09-13T09:12:43.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.830 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.830 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534520+0000", 2024-09-13T09:12:43.830 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0097851020000000004 2024-09-13T09:12:43.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.831 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.831 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.831 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534527+0000", 2024-09-13T09:12:43.831 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0160000000000003e-06 2024-09-13T09:12:43.831 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.831 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.831 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.831 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.831 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20057 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.831 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528501+0000", 2024-09-13T09:12:43.831 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33628861700000001, 2024-09-13T09:12:43.831 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34359621899999998, 2024-09-13T09:12:43.831 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.832 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.832 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.832 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.832 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.832 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20057 2024-09-13T09:12:43.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.832 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.832 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.832 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.832 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528501+0000", 2024-09-13T09:12:43.832 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.832 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.832 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.833 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528501+0000", 2024-09-13T09:12:43.833 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.833 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.833 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.833 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528500+0000", 2024-09-13T09:12:43.833 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.833 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.833 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.833 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528502+0000", 2024-09-13T09:12:43.833 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2189999999999998e-06 2024-09-13T09:12:43.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.833 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.834 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.834 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528508+0000", 2024-09-13T09:12:43.834 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4650000000000002e-06 2024-09-13T09:12:43.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.834 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.834 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.834 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528513+0000", 2024-09-13T09:12:43.834 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3070000000000002e-06 2024-09-13T09:12:43.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.834 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.834 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.834 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535090+0000", 2024-09-13T09:12:43.834 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.006576747 2024-09-13T09:12:43.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.835 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.835 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535096+0000", 2024-09-13T09:12:43.835 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6779999999999997e-06 2024-09-13T09:12:43.835 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.835 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.835 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.835 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.835 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:12:43.835 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33512692900000002, 2024-09-13T09:12:43.835 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34246760300000001, 2024-09-13T09:12:43.836 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.836 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.836 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.836 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.836 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.836 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:12:43.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.836 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.836 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.836 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.836 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:12:43.836 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.837 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.837 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.837 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:12:43.837 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.837 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.837 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.837 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:12:43.837 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.837 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.837 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.837 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:12:43.838 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:12:43.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.838 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.838 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.838 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:12:43.838 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:12:43.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.838 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.838 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.838 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:12:43.838 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:12:43.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.838 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.838 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.839 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:12:43.839 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:12:43.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.839 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.839 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.839 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:12:43.839 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:12:43.839 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.839 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.839 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.839 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.839 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20199 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.840 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538506+0000", 2024-09-13T09:12:43.840 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.326283304, 2024-09-13T09:12:43.840 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33365719500000002, 2024-09-13T09:12:43.840 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.840 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.840 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.840 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.840 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.840 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20199 2024-09-13T09:12:43.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.840 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.840 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.840 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.840 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538506+0000", 2024-09-13T09:12:43.841 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.841 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.841 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.841 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538506+0000", 2024-09-13T09:12:43.841 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.841 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.841 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.841 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538506+0000", 2024-09-13T09:12:43.841 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.841 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.842 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.842 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538508+0000", 2024-09-13T09:12:43.842 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.119e-06 2024-09-13T09:12:43.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.842 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.842 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.842 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538513+0000", 2024-09-13T09:12:43.842 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.536e-06 2024-09-13T09:12:43.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.842 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.842 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.842 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538517+0000", 2024-09-13T09:12:43.842 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.16e-06 2024-09-13T09:12:43.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.843 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.843 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538958+0000", 2024-09-13T09:12:43.843 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00044043899999999999 2024-09-13T09:12:43.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.843 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.843 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538964+0000", 2024-09-13T09:12:43.843 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3149999999999997e-06 2024-09-13T09:12:43.843 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.843 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.843 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.844 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20342 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.844 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.541226+0000", 2024-09-13T09:12:43.844 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32356309300000002, 2024-09-13T09:12:43.844 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33097026400000001, 2024-09-13T09:12:43.844 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.844 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.844 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.844 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.844 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.844 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20342 2024-09-13T09:12:43.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.844 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.844 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.844 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.845 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541226+0000", 2024-09-13T09:12:43.845 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.845 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.845 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.845 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541226+0000", 2024-09-13T09:12:43.845 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.845 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.845 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.845 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541226+0000", 2024-09-13T09:12:43.845 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.845 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.846 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.846 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541228+0000", 2024-09-13T09:12:43.846 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6359999999999998e-06 2024-09-13T09:12:43.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.846 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.846 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.846 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541234+0000", 2024-09-13T09:12:43.846 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3060000000000003e-06 2024-09-13T09:12:43.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.846 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.846 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.846 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541239+0000", 2024-09-13T09:12:43.846 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8019999999999999e-06 2024-09-13T09:12:43.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.847 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.847 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.847 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542237+0000", 2024-09-13T09:12:43.847 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00099727600000000007 2024-09-13T09:12:43.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.847 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.847 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.847 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542243+0000", 2024-09-13T09:12:43.847 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8560000000000001e-06 2024-09-13T09:12:43.847 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.847 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.847 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.848 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20453 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.848 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543259+0000", 2024-09-13T09:12:43.848 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32153032199999998, 2024-09-13T09:12:43.848 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32897055800000002, 2024-09-13T09:12:43.848 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.848 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.848 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.848 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.848 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.848 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20453 2024-09-13T09:12:43.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.848 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.849 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.849 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543259+0000", 2024-09-13T09:12:43.849 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.849 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.849 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.849 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543259+0000", 2024-09-13T09:12:43.849 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.849 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.849 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.849 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543258+0000", 2024-09-13T09:12:43.849 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.850 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.850 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543261+0000", 2024-09-13T09:12:43.850 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.232e-06 2024-09-13T09:12:43.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.850 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.850 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543266+0000", 2024-09-13T09:12:43.850 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4859999999999997e-06 2024-09-13T09:12:43.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.850 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.850 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543270+0000", 2024-09-13T09:12:43.851 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1840000000000001e-06 2024-09-13T09:12:43.851 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.851 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.851 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.851 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543800+0000", 2024-09-13T09:12:43.851 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052973899999999999 2024-09-13T09:12:43.851 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.851 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.851 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.851 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543815+0000", 2024-09-13T09:12:43.851 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5125e-05 2024-09-13T09:12:43.851 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.851 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.851 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.852 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.852 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20600 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.852 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.548090+0000", 2024-09-13T09:12:43.852 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31669962400000001, 2024-09-13T09:12:43.852 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32417289900000001, 2024-09-13T09:12:43.852 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.852 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.852 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.852 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.852 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.852 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20600 2024-09-13T09:12:43.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.852 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.853 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.853 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548090+0000", 2024-09-13T09:12:43.853 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.853 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.853 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548090+0000", 2024-09-13T09:12:43.853 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.853 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.853 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:12:43.853 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.854 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.854 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548092+0000", 2024-09-13T09:12:43.854 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5040000000000001e-06 2024-09-13T09:12:43.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.854 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.854 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548098+0000", 2024-09-13T09:12:43.854 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0680000000000002e-06 2024-09-13T09:12:43.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.855 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.855 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548102+0000", 2024-09-13T09:12:43.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8189999999999999e-06 2024-09-13T09:12:43.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.855 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.855 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548478+0000", 2024-09-13T09:12:43.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00037528399999999998 2024-09-13T09:12:43.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.855 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.855 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548482+0000", 2024-09-13T09:12:43.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3680000000000004e-06 2024-09-13T09:12:43.856 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.856 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.856 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.856 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.856 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20714 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.856 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549728+0000", 2024-09-13T09:12:43.856 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.315061333, 2024-09-13T09:12:43.856 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32256763500000002, 2024-09-13T09:12:43.856 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.856 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.856 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.856 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.857 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.857 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20714 2024-09-13T09:12:43.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.857 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.857 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.857 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549728+0000", 2024-09-13T09:12:43.857 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.857 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.857 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549728+0000", 2024-09-13T09:12:43.857 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.858 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549727+0000", 2024-09-13T09:12:43.858 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.858 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549730+0000", 2024-09-13T09:12:43.858 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4899999999999999e-06 2024-09-13T09:12:43.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.858 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549743+0000", 2024-09-13T09:12:43.859 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3063e-05 2024-09-13T09:12:43.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.859 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.859 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549753+0000", 2024-09-13T09:12:43.859 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042e-05 2024-09-13T09:12:43.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.859 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.859 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549804+0000", 2024-09-13T09:12:43.859 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0206e-05 2024-09-13T09:12:43.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.859 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.859 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549820+0000", 2024-09-13T09:12:43.860 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6331e-05 2024-09-13T09:12:43.860 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.860 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.860 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.860 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.860 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19948 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.860 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524739+0000", 2024-09-13T09:12:43.860 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34005009899999999, 2024-09-13T09:12:43.860 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34758951700000001, 2024-09-13T09:12:43.860 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.860 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.860 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.860 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.860 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.861 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19948 2024-09-13T09:12:43.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.861 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524739+0000", 2024-09-13T09:12:43.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524739+0000", 2024-09-13T09:12:43.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524739+0000", 2024-09-13T09:12:43.862 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524742+0000", 2024-09-13T09:12:43.862 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6219999999999998e-06 2024-09-13T09:12:43.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524749+0000", 2024-09-13T09:12:43.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3860000000000004e-06 2024-09-13T09:12:43.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.863 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.863 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.863 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524753+0000", 2024-09-13T09:12:43.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6720000000000003e-06 2024-09-13T09:12:43.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.863 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.863 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.863 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534551+0000", 2024-09-13T09:12:43.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0097980889999999994 2024-09-13T09:12:43.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.863 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.863 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.864 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534558+0000", 2024-09-13T09:12:43.864 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5910000000000001e-06 2024-09-13T09:12:43.864 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.864 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.864 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.864 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20058 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.864 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528517+0000", 2024-09-13T09:12:43.864 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33627258900000001, 2024-09-13T09:12:43.864 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34405429999999998, 2024-09-13T09:12:43.864 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.864 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.865 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.865 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.865 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.865 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20058 2024-09-13T09:12:43.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.865 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.865 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.865 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528517+0000", 2024-09-13T09:12:43.865 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.866 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528517+0000", 2024-09-13T09:12:43.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.866 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528516+0000", 2024-09-13T09:12:43.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.866 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528518+0000", 2024-09-13T09:12:43.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9149999999999999e-06 2024-09-13T09:12:43.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528524+0000", 2024-09-13T09:12:43.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3700000000000003e-06 2024-09-13T09:12:43.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528528+0000", 2024-09-13T09:12:43.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1099999999999996e-06 2024-09-13T09:12:43.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535121+0000", 2024-09-13T09:12:43.868 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0065929099999999996 2024-09-13T09:12:43.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.868 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.868 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.868 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535127+0000", 2024-09-13T09:12:43.868 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5799999999999997e-06 2024-09-13T09:12:43.868 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.868 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.868 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.868 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.868 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.868 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:12:43.869 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33510902199999998, 2024-09-13T09:12:43.869 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34292465900000002, 2024-09-13T09:12:43.869 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.869 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.869 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.869 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.869 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.869 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:12:43.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.869 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.869 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.869 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.869 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:12:43.869 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:12:43.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:12:43.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:12:43.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:12:43.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.871 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:12:43.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:12:43.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.871 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:12:43.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:12:43.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:12:43.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:12:43.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:12:43.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:12:43.872 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.872 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.872 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.873 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20200 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.873 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538524+0000", 2024-09-13T09:12:43.873 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.326265533, 2024-09-13T09:12:43.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33411651399999998, 2024-09-13T09:12:43.873 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.873 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.873 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.873 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.873 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.873 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20200 2024-09-13T09:12:43.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.873 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.873 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538524+0000", 2024-09-13T09:12:43.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538524+0000", 2024-09-13T09:12:43.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538523+0000", 2024-09-13T09:12:43.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.875 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538525+0000", 2024-09-13T09:12:43.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0530000000000001e-06 2024-09-13T09:12:43.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.875 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538531+0000", 2024-09-13T09:12:43.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2339999999999999e-06 2024-09-13T09:12:43.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.875 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538536+0000", 2024-09-13T09:12:43.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2069999999999997e-06 2024-09-13T09:12:43.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.876 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.876 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538986+0000", 2024-09-13T09:12:43.876 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00045007700000000001 2024-09-13T09:12:43.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.876 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.876 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538992+0000", 2024-09-13T09:12:43.876 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7340000000000002e-06 2024-09-13T09:12:43.876 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.876 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.876 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.877 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20343 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.877 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.541265+0000", 2024-09-13T09:12:43.877 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32352437099999998, 2024-09-13T09:12:43.877 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33140866899999999, 2024-09-13T09:12:43.877 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.877 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.877 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.877 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.877 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.877 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20343 2024-09-13T09:12:43.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.877 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.878 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541265+0000", 2024-09-13T09:12:43.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.878 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541265+0000", 2024-09-13T09:12:43.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.878 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541264+0000", 2024-09-13T09:12:43.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541267+0000", 2024-09-13T09:12:43.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1e-06 2024-09-13T09:12:43.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541275+0000", 2024-09-13T09:12:43.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4680000000000003e-06 2024-09-13T09:12:43.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541280+0000", 2024-09-13T09:12:43.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5670000000000002e-06 2024-09-13T09:12:43.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.880 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542271+0000", 2024-09-13T09:12:43.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00099022200000000002 2024-09-13T09:12:43.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.880 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542277+0000", 2024-09-13T09:12:43.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8689999999999998e-06 2024-09-13T09:12:43.880 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.880 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.881 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.881 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20454 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.881 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543462+0000", 2024-09-13T09:12:43.881 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.321327266, 2024-09-13T09:12:43.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.329246074, 2024-09-13T09:12:43.881 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.881 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.881 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.881 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.881 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.881 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20454 2024-09-13T09:12:43.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.882 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543462+0000", 2024-09-13T09:12:43.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543462+0000", 2024-09-13T09:12:43.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543281+0000", 2024-09-13T09:12:43.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998193 2024-09-13T09:12:43.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543467+0000", 2024-09-13T09:12:43.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018578900000000001 2024-09-13T09:12:43.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543479+0000", 2024-09-13T09:12:43.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2325e-05 2024-09-13T09:12:43.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543487+0000", 2024-09-13T09:12:43.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7139999999999993e-06 2024-09-13T09:12:43.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543846+0000", 2024-09-13T09:12:43.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000359086 2024-09-13T09:12:43.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543854+0000", 2024-09-13T09:12:43.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:12:43.885 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.885 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.885 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.885 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20601 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.885 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.548111+0000", 2024-09-13T09:12:43.885 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31667842299999999, 2024-09-13T09:12:43.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32463027, 2024-09-13T09:12:43.886 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.886 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.886 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.886 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.886 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.886 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20601 2024-09-13T09:12:43.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.886 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.886 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548111+0000", 2024-09-13T09:12:43.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548111+0000", 2024-09-13T09:12:43.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:12:43.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548112+0000", 2024-09-13T09:12:43.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1239999999999999e-06 2024-09-13T09:12:43.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548119+0000", 2024-09-13T09:12:43.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1929999999999998e-06 2024-09-13T09:12:43.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548136+0000", 2024-09-13T09:12:43.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6841999999999999e-05 2024-09-13T09:12:43.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548508+0000", 2024-09-13T09:12:43.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00037210299999999998 2024-09-13T09:12:43.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548512+0000", 2024-09-13T09:12:43.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8920000000000002e-06 2024-09-13T09:12:43.889 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.890 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.890 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.890 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20715 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.890 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549764+0000", 2024-09-13T09:12:43.890 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31502466699999998, 2024-09-13T09:12:43.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32301109500000003, 2024-09-13T09:12:43.890 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.890 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.890 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.890 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.890 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.890 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20715 2024-09-13T09:12:43.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.891 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549764+0000", 2024-09-13T09:12:43.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549764+0000", 2024-09-13T09:12:43.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549764+0000", 2024-09-13T09:12:43.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549767+0000", 2024-09-13T09:12:43.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6570000000000001e-06 2024-09-13T09:12:43.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549773+0000", 2024-09-13T09:12:43.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.63e-06 2024-09-13T09:12:43.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549779+0000", 2024-09-13T09:12:43.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.57e-06 2024-09-13T09:12:43.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549849+0000", 2024-09-13T09:12:43.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0369000000000003e-05 2024-09-13T09:12:43.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549857+0000", 2024-09-13T09:12:43.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9999999999999996e-06 2024-09-13T09:12:43.894 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.894 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.894 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.894 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19949 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.894 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524758+0000", 2024-09-13T09:12:43.894 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.34003132899999999, 2024-09-13T09:12:43.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34805535999999998, 2024-09-13T09:12:43.894 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.894 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.894 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.894 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.894 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.895 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19949 2024-09-13T09:12:43.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.895 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524758+0000", 2024-09-13T09:12:43.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524758+0000", 2024-09-13T09:12:43.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524757+0000", 2024-09-13T09:12:43.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524760+0000", 2024-09-13T09:12:43.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3649999999999998e-06 2024-09-13T09:12:43.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524766+0000", 2024-09-13T09:12:43.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9939999999999997e-06 2024-09-13T09:12:43.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524786+0000", 2024-09-13T09:12:43.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0533000000000001e-05 2024-09-13T09:12:43.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534583+0000", 2024-09-13T09:12:43.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0097967690000000003 2024-09-13T09:12:43.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534590+0000", 2024-09-13T09:12:43.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0870000000000001e-06 2024-09-13T09:12:43.898 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.898 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.898 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.898 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20059 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.898 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528531+0000", 2024-09-13T09:12:43.898 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33625803900000001, 2024-09-13T09:12:43.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34431580499999997, 2024-09-13T09:12:43.898 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.899 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.899 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.899 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.899 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.899 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20059 2024-09-13T09:12:43.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.899 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528531+0000", 2024-09-13T09:12:43.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528531+0000", 2024-09-13T09:12:43.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528531+0000", 2024-09-13T09:12:43.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528533+0000", 2024-09-13T09:12:43.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.238e-06 2024-09-13T09:12:43.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528540+0000", 2024-09-13T09:12:43.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9770000000000003e-06 2024-09-13T09:12:43.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528545+0000", 2024-09-13T09:12:43.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.481e-06 2024-09-13T09:12:43.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535152+0000", 2024-09-13T09:12:43.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0066077979999999998 2024-09-13T09:12:43.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535160+0000", 2024-09-13T09:12:43.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3089999999999998e-06 2024-09-13T09:12:43.902 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.902 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.902 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.902 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20201 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.902 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538540+0000", 2024-09-13T09:12:43.902 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32624940099999999, 2024-09-13T09:12:43.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33434219599999998, 2024-09-13T09:12:43.903 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.903 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.903 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.903 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.903 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.903 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20201 2024-09-13T09:12:43.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.903 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538540+0000", 2024-09-13T09:12:43.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538540+0000", 2024-09-13T09:12:43.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538539+0000", 2024-09-13T09:12:43.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538547+0000", 2024-09-13T09:12:43.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5939999999999996e-06 2024-09-13T09:12:43.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538553+0000", 2024-09-13T09:12:43.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4370000000000004e-06 2024-09-13T09:12:43.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538558+0000", 2024-09-13T09:12:43.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7040000000000002e-06 2024-09-13T09:12:43.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539012+0000", 2024-09-13T09:12:43.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00045435499999999998 2024-09-13T09:12:43.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539019+0000", 2024-09-13T09:12:43.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6189999999999996e-06 2024-09-13T09:12:43.906 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.906 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.906 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.906 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20312 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.907 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540501+0000", 2024-09-13T09:12:43.907 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32428789299999999, 2024-09-13T09:12:43.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.332413666, 2024-09-13T09:12:43.907 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.907 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.907 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.907 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.907 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.907 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20312 2024-09-13T09:12:43.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.907 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540501+0000", 2024-09-13T09:12:43.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540501+0000", 2024-09-13T09:12:43.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540501+0000", 2024-09-13T09:12:43.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540503+0000", 2024-09-13T09:12:43.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.295e-06 2024-09-13T09:12:43.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540508+0000", 2024-09-13T09:12:43.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3129999999999998e-06 2024-09-13T09:12:43.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540512+0000", 2024-09-13T09:12:43.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9249999999999997e-06 2024-09-13T09:12:43.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540571+0000", 2024-09-13T09:12:43.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8292999999999999e-05 2024-09-13T09:12:43.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540579+0000", 2024-09-13T09:12:43.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7269999999999996e-06 2024-09-13T09:12:43.910 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.910 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.911 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.911 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20344 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.911 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.541285+0000", 2024-09-13T09:12:43.911 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32350418399999997, 2024-09-13T09:12:43.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.331665196, 2024-09-13T09:12:43.911 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.911 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.911 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.911 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.911 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.912 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20344 2024-09-13T09:12:43.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.912 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541285+0000", 2024-09-13T09:12:43.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541285+0000", 2024-09-13T09:12:43.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541285+0000", 2024-09-13T09:12:43.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541293+0000", 2024-09-13T09:12:43.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3250000000000008e-06 2024-09-13T09:12:43.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541300+0000", 2024-09-13T09:12:43.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6889999999999998e-06 2024-09-13T09:12:43.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541304+0000", 2024-09-13T09:12:43.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6970000000000001e-06 2024-09-13T09:12:43.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542302+0000", 2024-09-13T09:12:43.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00099800699999999997 2024-09-13T09:12:43.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542306+0000", 2024-09-13T09:12:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0219999999999998e-06 2024-09-13T09:12:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20455 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543494+0000", 2024-09-13T09:12:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.321295302, 2024-09-13T09:12:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32948949799999999, 2024-09-13T09:12:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20455 2024-09-13T09:12:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543494+0000", 2024-09-13T09:12:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543494+0000", 2024-09-13T09:12:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543493+0000", 2024-09-13T09:12:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543496+0000", 2024-09-13T09:12:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:12:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543503+0000", 2024-09-13T09:12:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5050000000000004e-06 2024-09-13T09:12:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543508+0000", 2024-09-13T09:12:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9019999999999998e-06 2024-09-13T09:12:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543881+0000", 2024-09-13T09:12:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00037357299999999998 2024-09-13T09:12:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543889+0000", 2024-09-13T09:12:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4000000000000003e-06 2024-09-13T09:12:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20602 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.548158+0000", 2024-09-13T09:12:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.316631143, 2024-09-13T09:12:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32485993699999999, 2024-09-13T09:12:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20602 2024-09-13T09:12:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548158+0000", 2024-09-13T09:12:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548158+0000", 2024-09-13T09:12:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548157+0000", 2024-09-13T09:12:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548160+0000", 2024-09-13T09:12:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8049999999999998e-06 2024-09-13T09:12:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548168+0000", 2024-09-13T09:12:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2649999999999999e-06 2024-09-13T09:12:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548173+0000", 2024-09-13T09:12:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9729999999999997e-06 2024-09-13T09:12:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548562+0000", 2024-09-13T09:12:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00038955100000000002 2024-09-13T09:12:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548566+0000", 2024-09-13T09:12:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8410000000000002e-06 2024-09-13T09:12:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20716 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549784+0000", 2024-09-13T09:12:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31500564800000003, 2024-09-13T09:12:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32326903800000001, 2024-09-13T09:12:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20716 2024-09-13T09:12:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549784+0000", 2024-09-13T09:12:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549784+0000", 2024-09-13T09:12:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549783+0000", 2024-09-13T09:12:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549785+0000", 2024-09-13T09:12:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2010000000000002e-06 2024-09-13T09:12:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549791+0000", 2024-09-13T09:12:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0769999999999996e-06 2024-09-13T09:12:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549798+0000", 2024-09-13T09:12:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1750000000000002e-06 2024-09-13T09:12:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549882+0000", 2024-09-13T09:12:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4833000000000006e-05 2024-09-13T09:12:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549890+0000", 2024-09-13T09:12:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:12:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19950 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524793+0000", 2024-09-13T09:12:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33999597199999998, 2024-09-13T09:12:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34829280099999999, 2024-09-13T09:12:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19950 2024-09-13T09:12:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524793+0000", 2024-09-13T09:12:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524793+0000", 2024-09-13T09:12:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524793+0000", 2024-09-13T09:12:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524796+0000", 2024-09-13T09:12:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0000000000000001e-06 2024-09-13T09:12:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524803+0000", 2024-09-13T09:12:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.883e-06 2024-09-13T09:12:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524812+0000", 2024-09-13T09:12:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2350000000000005e-06 2024-09-13T09:12:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534616+0000", 2024-09-13T09:12:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0098046090000000006 2024-09-13T09:12:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534624+0000", 2024-09-13T09:12:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:12:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20060 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528549+0000", 2024-09-13T09:12:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33624047600000001, 2024-09-13T09:12:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.344572143, 2024-09-13T09:12:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20060 2024-09-13T09:12:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528549+0000", 2024-09-13T09:12:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528549+0000", 2024-09-13T09:12:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528548+0000", 2024-09-13T09:12:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528551+0000", 2024-09-13T09:12:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2079999999999999e-06 2024-09-13T09:12:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528557+0000", 2024-09-13T09:12:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3199999999999996e-06 2024-09-13T09:12:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528562+0000", 2024-09-13T09:12:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5010000000000004e-06 2024-09-13T09:12:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535186+0000", 2024-09-13T09:12:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0066234639999999999 2024-09-13T09:12:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535193+0000", 2024-09-13T09:12:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8789999999999997e-06 2024-09-13T09:12:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20202 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538564+0000", 2024-09-13T09:12:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32622479500000001, 2024-09-13T09:12:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33458966200000001, 2024-09-13T09:12:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20202 2024-09-13T09:12:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538564+0000", 2024-09-13T09:12:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538564+0000", 2024-09-13T09:12:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538564+0000", 2024-09-13T09:12:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538566+0000", 2024-09-13T09:12:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6079999999999998e-06 2024-09-13T09:12:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538573+0000", 2024-09-13T09:12:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5950000000000004e-06 2024-09-13T09:12:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538578+0000", 2024-09-13T09:12:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8189999999999999e-06 2024-09-13T09:12:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539040+0000", 2024-09-13T09:12:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00046227399999999999 2024-09-13T09:12:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539046+0000", 2024-09-13T09:12:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:12:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:12:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32426557499999997, 2024-09-13T09:12:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33266526099999999, 2024-09-13T09:12:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:12:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:12:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:12:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:12:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:12:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:12:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:12:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:12:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:12:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:12:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:12:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:12:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:12:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:12:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20345 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.541310+0000", 2024-09-13T09:12:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32347876399999997, 2024-09-13T09:12:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33191163200000001, 2024-09-13T09:12:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20345 2024-09-13T09:12:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541310+0000", 2024-09-13T09:12:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541310+0000", 2024-09-13T09:12:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541310+0000", 2024-09-13T09:12:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541312+0000", 2024-09-13T09:12:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4820000000000001e-06 2024-09-13T09:12:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541318+0000", 2024-09-13T09:12:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0090000000000004e-06 2024-09-13T09:12:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541323+0000", 2024-09-13T09:12:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.882e-06 2024-09-13T09:12:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542340+0000", 2024-09-13T09:12:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010169529999999999 2024-09-13T09:12:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542348+0000", 2024-09-13T09:12:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7600000000000002e-06 2024-09-13T09:12:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20456 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543533+0000", 2024-09-13T09:12:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32125581399999997, 2024-09-13T09:12:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32972329900000003, 2024-09-13T09:12:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20456 2024-09-13T09:12:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543533+0000", 2024-09-13T09:12:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543533+0000", 2024-09-13T09:12:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543533+0000", 2024-09-13T09:12:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543536+0000", 2024-09-13T09:12:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8499999999999998e-06 2024-09-13T09:12:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543543+0000", 2024-09-13T09:12:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7179999999999996e-06 2024-09-13T09:12:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543549+0000", 2024-09-13T09:12:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3970000000000002e-06 2024-09-13T09:12:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543910+0000", 2024-09-13T09:12:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000361534 2024-09-13T09:12:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543917+0000", 2024-09-13T09:12:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1049999999999997e-06 2024-09-13T09:12:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20603 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.548177+0000", 2024-09-13T09:12:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31661206200000003, 2024-09-13T09:12:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32511567099999999, 2024-09-13T09:12:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20603 2024-09-13T09:12:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548177+0000", 2024-09-13T09:12:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548177+0000", 2024-09-13T09:12:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548177+0000", 2024-09-13T09:12:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548179+0000", 2024-09-13T09:12:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:12:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548185+0000", 2024-09-13T09:12:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0769999999999996e-06 2024-09-13T09:12:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548190+0000", 2024-09-13T09:12:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.254e-06 2024-09-13T09:12:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548596+0000", 2024-09-13T09:12:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00040566400000000002 2024-09-13T09:12:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548603+0000", 2024-09-13T09:12:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3270000000000003e-06 2024-09-13T09:12:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20717 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549802+0000", 2024-09-13T09:12:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31498707399999998, 2024-09-13T09:12:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32352386399999999, 2024-09-13T09:12:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20717 2024-09-13T09:12:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549802+0000", 2024-09-13T09:12:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549802+0000", 2024-09-13T09:12:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549802+0000", 2024-09-13T09:12:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549804+0000", 2024-09-13T09:12:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2469999999999998e-06 2024-09-13T09:12:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549818+0000", 2024-09-13T09:12:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4047e-05 2024-09-13T09:12:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549824+0000", 2024-09-13T09:12:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7209999999999999e-06 2024-09-13T09:12:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549917+0000", 2024-09-13T09:12:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3601e-05 2024-09-13T09:12:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549927+0000", 2024-09-13T09:12:43.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3589999999999997e-06 2024-09-13T09:12:43.961 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.961 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.961 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.961 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:19951 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.961 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.524817+0000", 2024-09-13T09:12:43.961 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33997166400000001, 2024-09-13T09:12:43.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34854321300000002, 2024-09-13T09:12:43.961 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.961 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.962 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.962 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.962 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.962 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 19951 2024-09-13T09:12:43.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.962 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524817+0000", 2024-09-13T09:12:43.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524817+0000", 2024-09-13T09:12:43.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524817+0000", 2024-09-13T09:12:43.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524820+0000", 2024-09-13T09:12:43.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4980000000000001e-06 2024-09-13T09:12:43.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.524826+0000", 2024-09-13T09:12:43.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:12:43.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.525016+0000", 2024-09-13T09:12:43.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000189691 2024-09-13T09:12:43.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534649+0000", 2024-09-13T09:12:43.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096330970000000002 2024-09-13T09:12:43.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.534657+0000", 2024-09-13T09:12:43.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1110000000000008e-06 2024-09-13T09:12:43.965 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.965 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.965 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.965 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20061 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.965 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528566+0000", 2024-09-13T09:12:43.966 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.33622274499999999, 2024-09-13T09:12:43.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.34482722399999999, 2024-09-13T09:12:43.966 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.966 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.966 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.966 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.966 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.966 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20061 2024-09-13T09:12:43.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.966 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528566+0000", 2024-09-13T09:12:43.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528566+0000", 2024-09-13T09:12:43.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528566+0000", 2024-09-13T09:12:43.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528568+0000", 2024-09-13T09:12:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9690000000000001e-06 2024-09-13T09:12:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528574+0000", 2024-09-13T09:12:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7729999999999996e-06 2024-09-13T09:12:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528579+0000", 2024-09-13T09:12:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6890000000000001e-06 2024-09-13T09:12:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535222+0000", 2024-09-13T09:12:43.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0066423150000000002 2024-09-13T09:12:43.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.535229+0000", 2024-09-13T09:12:43.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9e-06 2024-09-13T09:12:43.969 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.969 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.969 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.970 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20203 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.970 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538583+0000", 2024-09-13T09:12:43.970 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32620647400000002, 2024-09-13T09:12:43.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.334845583, 2024-09-13T09:12:43.970 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.970 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.970 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.970 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.970 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.970 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20203 2024-09-13T09:12:43.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.970 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538583+0000", 2024-09-13T09:12:43.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538583+0000", 2024-09-13T09:12:43.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538582+0000", 2024-09-13T09:12:43.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538585+0000", 2024-09-13T09:12:43.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4159999999999998e-06 2024-09-13T09:12:43.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538591+0000", 2024-09-13T09:12:43.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2029999999999997e-06 2024-09-13T09:12:43.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538596+0000", 2024-09-13T09:12:43.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6670000000000004e-06 2024-09-13T09:12:43.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539068+0000", 2024-09-13T09:12:43.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000472239 2024-09-13T09:12:43.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539075+0000", 2024-09-13T09:12:43.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:12:43.973 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.973 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.973 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.974 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.974 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:12:43.974 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32422552399999999, 2024-09-13T09:12:43.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.33289792899999998, 2024-09-13T09:12:43.974 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.974 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.974 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.974 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.974 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.974 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:12:43.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.974 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:12:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:12:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:12:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:12:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:12:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:12:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:12:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:12:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:12:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:12:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:12:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:12:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:12:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20346 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.541328+0000", 2024-09-13T09:12:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32346140000000001, 2024-09-13T09:12:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.332168727, 2024-09-13T09:12:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20346 2024-09-13T09:12:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541328+0000", 2024-09-13T09:12:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541328+0000", 2024-09-13T09:12:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541327+0000", 2024-09-13T09:12:43.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541330+0000", 2024-09-13T09:12:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:12:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541336+0000", 2024-09-13T09:12:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0140000000000004e-06 2024-09-13T09:12:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.541340+0000", 2024-09-13T09:12:43.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4170000000000002e-06 2024-09-13T09:12:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542372+0000", 2024-09-13T09:12:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001031761 2024-09-13T09:12:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.542379+0000", 2024-09-13T09:12:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2470000000000002e-06 2024-09-13T09:12:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20457 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.543554+0000", 2024-09-13T09:12:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.32123538299999999, 2024-09-13T09:12:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32997764499999999, 2024-09-13T09:12:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20457 2024-09-13T09:12:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.982 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543554+0000", 2024-09-13T09:12:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543554+0000", 2024-09-13T09:12:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543553+0000", 2024-09-13T09:12:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543556+0000", 2024-09-13T09:12:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4109999999999999e-06 2024-09-13T09:12:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543562+0000", 2024-09-13T09:12:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2099999999999998e-06 2024-09-13T09:12:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.543566+0000", 2024-09-13T09:12:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4510000000000002e-06 2024-09-13T09:12:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544099+0000", 2024-09-13T09:12:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053284600000000001 2024-09-13T09:12:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.544108+0000", 2024-09-13T09:12:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2360000000000004e-06 2024-09-13T09:12:43.985 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20568 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.547205+0000", 2024-09-13T09:12:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31758408900000001, 2024-09-13T09:12:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32635982600000002, 2024-09-13T09:12:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.986 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20568 2024-09-13T09:12:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547205+0000", 2024-09-13T09:12:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547205+0000", 2024-09-13T09:12:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547205+0000", 2024-09-13T09:12:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547207+0000", 2024-09-13T09:12:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4159999999999998e-06 2024-09-13T09:12:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547213+0000", 2024-09-13T09:12:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8189999999999997e-06 2024-09-13T09:12:43.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547217+0000", 2024-09-13T09:12:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6399999999999996e-06 2024-09-13T09:12:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547262+0000", 2024-09-13T09:12:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4277000000000003e-05 2024-09-13T09:12:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.547271+0000", 2024-09-13T09:12:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7030000000000001e-06 2024-09-13T09:12:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20604 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.548207+0000", 2024-09-13T09:12:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31658265499999999, 2024-09-13T09:12:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.325392878, 2024-09-13T09:12:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.990 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20604 2024-09-13T09:12:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548207+0000", 2024-09-13T09:12:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548207+0000", 2024-09-13T09:12:43.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548206+0000", 2024-09-13T09:12:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548209+0000", 2024-09-13T09:12:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.509e-06 2024-09-13T09:12:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548215+0000", 2024-09-13T09:12:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3969999999999999e-06 2024-09-13T09:12:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548220+0000", 2024-09-13T09:12:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:12:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548631+0000", 2024-09-13T09:12:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00041156899999999998 2024-09-13T09:12:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.548638+0000", 2024-09-13T09:12:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.126e-06 2024-09-13T09:12:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20718 3.12 3.8d2c5fb2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.549844+0000", 2024-09-13T09:12:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.31494502899999999, 2024-09-13T09:12:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.32378876200000001, 2024-09-13T09:12:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:12:43.994 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20718 2024-09-13T09:12:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549844+0000", 2024-09-13T09:12:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:43.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549844+0000", 2024-09-13T09:12:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549844+0000", 2024-09-13T09:12:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549850+0000", 2024-09-13T09:12:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8700000000000003e-06 2024-09-13T09:12:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549858+0000", 2024-09-13T09:12:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7230000000000004e-06 2024-09-13T09:12:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549863+0000", 2024-09-13T09:12:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0930000000000002e-06 2024-09-13T09:12:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549952+0000", 2024-09-13T09:12:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8595000000000001e-05 2024-09-13T09:12:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:43.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:43.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:43.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.549960+0000", 2024-09-13T09:12:43.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5920000000000001e-06 2024-09-13T09:12:43.998 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.998 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:43.998 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.998 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:43.998 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:12:43.998 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 256 2024-09-13T09:12:43.998 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:43.999 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:43.999+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:44.000 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:44.001 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-09-13T09:12:44.016 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:12:44.100 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:44.100+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:44.152 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:44.152 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:44.152 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:44.152 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:44.152 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:44.162 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:44.202 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:44.201+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:44.297 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:44.297 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:44.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:44.297 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:44.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.297 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:13792 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 7251968~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.298 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:41.962995+0000", 2024-09-13T09:12:44.298 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.3319454459999998, 2024-09-13T09:12:44.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032521330000000001, 2024-09-13T09:12:44.298 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.298 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:44.298 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.298 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.298 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.298 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 13792 2024-09-13T09:12:44.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.298 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.962995+0000", 2024-09-13T09:12:44.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.962995+0000", 2024-09-13T09:12:44.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.962995+0000", 2024-09-13T09:12:44.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:44.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.962997+0000", 2024-09-13T09:12:44.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2010000000000002e-06 2024-09-13T09:12:44.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963003+0000", 2024-09-13T09:12:44.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8760000000000002e-06 2024-09-13T09:12:44.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963009+0000", 2024-09-13T09:12:44.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7039999999999999e-06 2024-09-13T09:12:44.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.995394+0000", 2024-09-13T09:12:44.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032385381999999997 2024-09-13T09:12:44.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.995450+0000", 2024-09-13T09:12:44.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.554e-05 2024-09-13T09:12:44.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.995517+0000", 2024-09-13T09:12:44.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7003999999999997e-05 2024-09-13T09:12:44.302 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.302 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.302 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.302 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:13793 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 7274496~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.302 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:41.963025+0000", 2024-09-13T09:12:44.302 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.3319159539999998, 2024-09-13T09:12:44.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032671635999999997, 2024-09-13T09:12:44.302 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.302 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:44.302 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.302 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.302 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.303 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 13793 2024-09-13T09:12:44.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.303 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963025+0000", 2024-09-13T09:12:44.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963025+0000", 2024-09-13T09:12:44.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963024+0000", 2024-09-13T09:12:44.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:44.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963027+0000", 2024-09-13T09:12:44.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9189999999999999e-06 2024-09-13T09:12:44.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963035+0000", 2024-09-13T09:12:44.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5360000000000004e-06 2024-09-13T09:12:44.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963040+0000", 2024-09-13T09:12:44.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.276e-06 2024-09-13T09:12:44.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.995548+0000", 2024-09-13T09:12:44.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032508294 2024-09-13T09:12:44.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.995621+0000", 2024-09-13T09:12:44.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2539000000000005e-05 2024-09-13T09:12:44.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.306 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.995697+0000", 2024-09-13T09:12:44.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5610000000000003e-05 2024-09-13T09:12:44.306 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.306 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.306 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.306 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:13794 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 7297024~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.306 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:41.963045+0000", 2024-09-13T09:12:44.306 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.3318955360000002, 2024-09-13T09:12:44.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032753049999999999, 2024-09-13T09:12:44.307 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.307 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:44.307 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.307 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.307 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.307 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 13794 2024-09-13T09:12:44.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.307 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963045+0000", 2024-09-13T09:12:44.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963045+0000", 2024-09-13T09:12:44.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963045+0000", 2024-09-13T09:12:44.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:44.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963047+0000", 2024-09-13T09:12:44.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.334e-06 2024-09-13T09:12:44.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963053+0000", 2024-09-13T09:12:44.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7760000000000003e-06 2024-09-13T09:12:44.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963058+0000", 2024-09-13T09:12:44.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9520000000000002e-06 2024-09-13T09:12:44.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.995712+0000", 2024-09-13T09:12:44.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032654023999999997 2024-09-13T09:12:44.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.995752+0000", 2024-09-13T09:12:44.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9538e-05 2024-09-13T09:12:44.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.995798+0000", 2024-09-13T09:12:44.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6875000000000001e-05 2024-09-13T09:12:44.310 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.310 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.310 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.311 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:13795 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 7319552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.311 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:41.963063+0000", 2024-09-13T09:12:44.311 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.3318783829999998, 2024-09-13T09:12:44.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032845599000000003, 2024-09-13T09:12:44.311 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.311 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:44.311 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.311 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.311 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.311 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 13795 2024-09-13T09:12:44.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.311 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963063+0000", 2024-09-13T09:12:44.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963063+0000", 2024-09-13T09:12:44.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963062+0000", 2024-09-13T09:12:44.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:44.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963064+0000", 2024-09-13T09:12:44.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3329999999999999e-06 2024-09-13T09:12:44.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963071+0000", 2024-09-13T09:12:44.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1260000000000003e-06 2024-09-13T09:12:44.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.963075+0000", 2024-09-13T09:12:44.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.685e-06 2024-09-13T09:12:44.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.995824+0000", 2024-09-13T09:12:44.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032748372999999997 2024-09-13T09:12:44.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.995863+0000", 2024-09-13T09:12:44.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9172999999999999e-05 2024-09-13T09:12:44.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.995908+0000", 2024-09-13T09:12:44.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5272999999999998e-05 2024-09-13T09:12:44.315 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.315 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.315 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.315 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:13923 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 7321600~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.315 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:41.973111+0000", 2024-09-13T09:12:44.316 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.3218303100000002, 2024-09-13T09:12:44.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032601640000000001, 2024-09-13T09:12:44.316 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.316 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:44.316 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.316 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.316 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.316 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 13923 2024-09-13T09:12:44.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.316 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.973111+0000", 2024-09-13T09:12:44.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.973111+0000", 2024-09-13T09:12:44.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.973110+0000", 2024-09-13T09:12:44.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:44.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.973118+0000", 2024-09-13T09:12:44.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9310000000000008e-06 2024-09-13T09:12:44.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.973125+0000", 2024-09-13T09:12:44.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6379999999999998e-06 2024-09-13T09:12:44.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.973129+0000", 2024-09-13T09:12:44.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1409999999999998e-06 2024-09-13T09:12:44.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.005525+0000", 2024-09-13T09:12:44.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032395588000000003 2024-09-13T09:12:44.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.005613+0000", 2024-09-13T09:12:44.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8485000000000004e-05 2024-09-13T09:12:44.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.005712+0000", 2024-09-13T09:12:44.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9227000000000004e-05 2024-09-13T09:12:44.320 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.320 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.320 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.320 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:14042 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 7075840~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.320 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:41.981213+0000", 2024-09-13T09:12:44.320 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.3137283370000001, 2024-09-13T09:12:44.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032395556999999998, 2024-09-13T09:12:44.320 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.320 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:44.320 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.320 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.321 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.321 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 14042 2024-09-13T09:12:44.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.321 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.981213+0000", 2024-09-13T09:12:44.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.981213+0000", 2024-09-13T09:12:44.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.981212+0000", 2024-09-13T09:12:44.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:44.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.981214+0000", 2024-09-13T09:12:44.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1840000000000002e-06 2024-09-13T09:12:44.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.981221+0000", 2024-09-13T09:12:44.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3509999999999998e-06 2024-09-13T09:12:44.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.981225+0000", 2024-09-13T09:12:44.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3390000000000003e-06 2024-09-13T09:12:44.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.013519+0000", 2024-09-13T09:12:44.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032294014000000003 2024-09-13T09:12:44.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.324 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.013565+0000", 2024-09-13T09:12:44.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5816999999999999e-05 2024-09-13T09:12:44.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.324 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.324 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.013608+0000", 2024-09-13T09:12:44.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3210999999999997e-05 2024-09-13T09:12:44.324 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.324 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.324 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.324 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:14043 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 7098368~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.325 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:41.981230+0000", 2024-09-13T09:12:44.325 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.313711192, 2024-09-13T09:12:44.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032597419000000002, 2024-09-13T09:12:44.325 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.325 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:44.325 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.325 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.325 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.325 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 14043 2024-09-13T09:12:44.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.325 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.981230+0000", 2024-09-13T09:12:44.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.981230+0000", 2024-09-13T09:12:44.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.981229+0000", 2024-09-13T09:12:44.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:44.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.981232+0000", 2024-09-13T09:12:44.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5840000000000002e-06 2024-09-13T09:12:44.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.981238+0000", 2024-09-13T09:12:44.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:12:44.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.981242+0000", 2024-09-13T09:12:44.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5020000000000003e-06 2024-09-13T09:12:44.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.013655+0000", 2024-09-13T09:12:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032412458999999998 2024-09-13T09:12:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.013729+0000", 2024-09-13T09:12:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3926999999999999e-05 2024-09-13T09:12:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.013827+0000", 2024-09-13T09:12:44.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8309999999999999e-05 2024-09-13T09:12:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:14052 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 7301120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:41.983025+0000", 2024-09-13T09:12:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.3119155390000001, 2024-09-13T09:12:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032430392000000002, 2024-09-13T09:12:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.329 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 14052 2024-09-13T09:12:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.983025+0000", 2024-09-13T09:12:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.983025+0000", 2024-09-13T09:12:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.983025+0000", 2024-09-13T09:12:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.983027+0000", 2024-09-13T09:12:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2079999999999999e-06 2024-09-13T09:12:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.983034+0000", 2024-09-13T09:12:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4699999999999999e-06 2024-09-13T09:12:44.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.983049+0000", 2024-09-13T09:12:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4936999999999999e-05 2024-09-13T09:12:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.015361+0000", 2024-09-13T09:12:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032312343 2024-09-13T09:12:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.015403+0000", 2024-09-13T09:12:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1910999999999999e-05 2024-09-13T09:12:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.015456+0000", 2024-09-13T09:12:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2882e-05 2024-09-13T09:12:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:14053 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 7323648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.333 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:41.983054+0000", 2024-09-13T09:12:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.3118871059999999, 2024-09-13T09:12:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032564276000000003, 2024-09-13T09:12:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 14053 2024-09-13T09:12:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.983054+0000", 2024-09-13T09:12:44.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.983054+0000", 2024-09-13T09:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.983053+0000", 2024-09-13T09:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.983056+0000", 2024-09-13T09:12:44.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.954e-06 2024-09-13T09:12:44.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.983063+0000", 2024-09-13T09:12:44.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.207e-06 2024-09-13T09:12:44.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:41.983069+0000", 2024-09-13T09:12:44.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1530000000000002e-06 2024-09-13T09:12:44.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.015486+0000", 2024-09-13T09:12:44.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032417348999999998 2024-09-13T09:12:44.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.015562+0000", 2024-09-13T09:12:44.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5605999999999994e-05 2024-09-13T09:12:44.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.015618+0000", 2024-09-13T09:12:44.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6421000000000001e-05 2024-09-13T09:12:44.337 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.338 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.338 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.338 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:15646 3.15 3:abeebc43:::rbd_object_map.112326fb43d3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.338 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.034485+0000", 2024-09-13T09:12:44.338 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.2604557989999998, 2024-09-13T09:12:44.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032541249000000001, 2024-09-13T09:12:44.338 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.338 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:44.338 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.338 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.338 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.338 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 15646 2024-09-13T09:12:44.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.339 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034485+0000", 2024-09-13T09:12:44.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034485+0000", 2024-09-13T09:12:44.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034485+0000", 2024-09-13T09:12:44.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:44.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034487+0000", 2024-09-13T09:12:44.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5409999999999999e-06 2024-09-13T09:12:44.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034493+0000", 2024-09-13T09:12:44.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4670000000000003e-06 2024-09-13T09:12:44.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034497+0000", 2024-09-13T09:12:44.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3019999999999996e-06 2024-09-13T09:12:44.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.059937+0000", 2024-09-13T09:12:44.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.025440372999999999 2024-09-13T09:12:44.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.060027+0000", 2024-09-13T09:12:44.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9438999999999995e-05 2024-09-13T09:12:44.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:44.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.060439+0000", 2024-09-13T09:12:44.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00041170999999999997 2024-09-13T09:12:44.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:44.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.064323+0000", 2024-09-13T09:12:44.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0038843380000000002 2024-09-13T09:12:44.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:44.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.066941+0000", 2024-09-13T09:12:44.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026179749999999998 2024-09-13T09:12:44.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:44.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.066991+0000", 2024-09-13T09:12:44.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0467999999999999e-05 2024-09-13T09:12:44.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.067026+0000", 2024-09-13T09:12:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5033000000000002e-05 2024-09-13T09:12:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:15771 3.15 3:abeebc43:::rbd_object_map.112326fb43d3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.034501+0000", 2024-09-13T09:12:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.2604395149999998, 2024-09-13T09:12:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.034313918999999998, 2024-09-13T09:12:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.344 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:44.345 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.345 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.345 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.345 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 15771 2024-09-13T09:12:44.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.345 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034501+0000", 2024-09-13T09:12:44.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034501+0000", 2024-09-13T09:12:44.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034501+0000", 2024-09-13T09:12:44.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:44.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034504+0000", 2024-09-13T09:12:44.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7379999999999999e-06 2024-09-13T09:12:44.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034510+0000", 2024-09-13T09:12:44.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0789999999999997e-06 2024-09-13T09:12:44.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034514+0000", 2024-09-13T09:12:44.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4299999999999999e-06 2024-09-13T09:12:44.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.060995+0000", 2024-09-13T09:12:44.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.026480877999999999 2024-09-13T09:12:44.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.061049+0000", 2024-09-13T09:12:44.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4216999999999999e-05 2024-09-13T09:12:44.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:44.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.061290+0000", 2024-09-13T09:12:44.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000240836 2024-09-13T09:12:44.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:44.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.064330+0000", 2024-09-13T09:12:44.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0030397390000000001 2024-09-13T09:12:44.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:44.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.068716+0000", 2024-09-13T09:12:44.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0043857150000000001 2024-09-13T09:12:44.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:44.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.068769+0000", 2024-09-13T09:12:44.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3037000000000001e-05 2024-09-13T09:12:44.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.068815+0000", 2024-09-13T09:12:44.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6604000000000003e-05 2024-09-13T09:12:44.351 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.351 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.351 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.351 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:15916 3.15 3:abeebc43:::rbd_object_map.112326fb43d3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.351 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.034518+0000", 2024-09-13T09:12:44.352 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.2604224290000001, 2024-09-13T09:12:44.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.034441181000000001, 2024-09-13T09:12:44.352 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.352 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:44.352 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.353 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.353 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.353 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 15916 2024-09-13T09:12:44.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.353 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034518+0000", 2024-09-13T09:12:44.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034518+0000", 2024-09-13T09:12:44.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.355 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034518+0000", 2024-09-13T09:12:44.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:44.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034525+0000", 2024-09-13T09:12:44.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2389999999999996e-06 2024-09-13T09:12:44.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.357 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034532+0000", 2024-09-13T09:12:44.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.178e-06 2024-09-13T09:12:44.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.358 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.358 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034538+0000", 2024-09-13T09:12:44.358 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4880000000000004e-06 2024-09-13T09:12:44.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.358 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.061610+0000", 2024-09-13T09:12:44.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.027071517 2024-09-13T09:12:44.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.359 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.061708+0000", 2024-09-13T09:12:44.360 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8067999999999997e-05 2024-09-13T09:12:44.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.360 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:44.360 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.062028+0000", 2024-09-13T09:12:44.360 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000320367 2024-09-13T09:12:44.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:44.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.066863+0000", 2024-09-13T09:12:44.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048348890000000002 2024-09-13T09:12:44.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:44.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.068894+0000", 2024-09-13T09:12:44.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002031026 2024-09-13T09:12:44.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:44.363 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.068928+0000", 2024-09-13T09:12:44.363 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3880000000000001e-05 2024-09-13T09:12:44.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.363 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.363 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.068960+0000", 2024-09-13T09:12:44.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1915000000000002e-05 2024-09-13T09:12:44.364 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.364 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.364 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.365 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:15922 3.15 3:abeebc43:::rbd_object_map.112326fb43d3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.365 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.034543+0000", 2024-09-13T09:12:44.365 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.2603982889999998, 2024-09-13T09:12:44.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.034507517000000001, 2024-09-13T09:12:44.365 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.365 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:44.365 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.366 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.366 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.366 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 15922 2024-09-13T09:12:44.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.366 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.367 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034543+0000", 2024-09-13T09:12:44.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.368 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.368 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034543+0000", 2024-09-13T09:12:44.368 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.368 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034542+0000", 2024-09-13T09:12:44.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:44.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034545+0000", 2024-09-13T09:12:44.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0189999999999998e-06 2024-09-13T09:12:44.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034551+0000", 2024-09-13T09:12:44.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:12:44.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034562+0000", 2024-09-13T09:12:44.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0642000000000001e-05 2024-09-13T09:12:44.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.062410+0000", 2024-09-13T09:12:44.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.027848030999999999 2024-09-13T09:12:44.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.062499+0000", 2024-09-13T09:12:44.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9664000000000001e-05 2024-09-13T09:12:44.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:44.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.062985+0000", 2024-09-13T09:12:44.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00048576700000000001 2024-09-13T09:12:44.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:44.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.066869+0000", 2024-09-13T09:12:44.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0038834680000000002 2024-09-13T09:12:44.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:44.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069002+0000", 2024-09-13T09:12:44.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021332439999999999 2024-09-13T09:12:44.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:44.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069030+0000", 2024-09-13T09:12:44.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8572999999999999e-05 2024-09-13T09:12:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069050+0000", 2024-09-13T09:12:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.967e-05 2024-09-13T09:12:44.377 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.378 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.378 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.378 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:16064 3.15 3:abeebc43:::rbd_object_map.112326fb43d3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.378 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.034793+0000", 2024-09-13T09:12:44.379 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.2601476319999998, 2024-09-13T09:12:44.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.034336431000000001, 2024-09-13T09:12:44.379 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.379 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:44.379 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.379 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.380 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.380 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 16064 2024-09-13T09:12:44.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.380 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034793+0000", 2024-09-13T09:12:44.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034793+0000", 2024-09-13T09:12:44.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034793+0000", 2024-09-13T09:12:44.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:44.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.383 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034803+0000", 2024-09-13T09:12:44.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0532e-05 2024-09-13T09:12:44.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034820+0000", 2024-09-13T09:12:44.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6990000000000002e-05 2024-09-13T09:12:44.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034825+0000", 2024-09-13T09:12:44.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4970000000000003e-06 2024-09-13T09:12:44.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.063502+0000", 2024-09-13T09:12:44.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.028677316000000001 2024-09-13T09:12:44.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.063585+0000", 2024-09-13T09:12:44.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3363999999999997e-05 2024-09-13T09:12:44.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:44.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.063961+0000", 2024-09-13T09:12:44.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00037612799999999999 2024-09-13T09:12:44.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.388 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:44.388 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.066874+0000", 2024-09-13T09:12:44.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029124239999999998 2024-09-13T09:12:44.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:44.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069088+0000", 2024-09-13T09:12:44.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002214223 2024-09-13T09:12:44.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:44.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069111+0000", 2024-09-13T09:12:44.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2955999999999998e-05 2024-09-13T09:12:44.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069130+0000", 2024-09-13T09:12:44.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8674000000000001e-05 2024-09-13T09:12:44.392 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.392 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.392 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.392 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:16160 3.15 3:abeebc43:::rbd_object_map.112326fb43d3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.393 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.034829+0000", 2024-09-13T09:12:44.393 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.2601122149999999, 2024-09-13T09:12:44.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.034373048000000003, 2024-09-13T09:12:44.393 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.393 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:44.393 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.394 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.394 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.394 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 16160 2024-09-13T09:12:44.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.394 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034829+0000", 2024-09-13T09:12:44.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.396 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034829+0000", 2024-09-13T09:12:44.396 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034828+0000", 2024-09-13T09:12:44.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:44.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.397 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034831+0000", 2024-09-13T09:12:44.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.424e-06 2024-09-13T09:12:44.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.398 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.398 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034834+0000", 2024-09-13T09:12:44.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6239999999999999e-06 2024-09-13T09:12:44.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034839+0000", 2024-09-13T09:12:44.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2429999999999999e-06 2024-09-13T09:12:44.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.400 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.064359+0000", 2024-09-13T09:12:44.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.029520380999999998 2024-09-13T09:12:44.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.401 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.401 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.064404+0000", 2024-09-13T09:12:44.401 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4530000000000002e-05 2024-09-13T09:12:44.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.402 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:44.402 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.064586+0000", 2024-09-13T09:12:44.402 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018262199999999999 2024-09-13T09:12:44.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.402 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:44.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.066880+0000", 2024-09-13T09:12:44.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022941060000000002 2024-09-13T09:12:44.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:44.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069164+0000", 2024-09-13T09:12:44.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002283321 2024-09-13T09:12:44.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.404 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:44.404 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069185+0000", 2024-09-13T09:12:44.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1815e-05 2024-09-13T09:12:44.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.405 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069202+0000", 2024-09-13T09:12:44.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6370000000000001e-05 2024-09-13T09:12:44.405 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.406 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.406 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.406 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:16198 3.15 3:abeebc43:::rbd_object_map.112326fb43d3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.406 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.034841+0000", 2024-09-13T09:12:44.407 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.260099876, 2024-09-13T09:12:44.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.034477018999999998, 2024-09-13T09:12:44.407 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.407 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:44.407 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.407 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.408 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.408 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 16198 2024-09-13T09:12:44.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.408 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034841+0000", 2024-09-13T09:12:44.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034841+0000", 2024-09-13T09:12:44.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.411 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.411 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034841+0000", 2024-09-13T09:12:44.411 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:12:44.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034843+0000", 2024-09-13T09:12:44.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.759e-06 2024-09-13T09:12:44.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034846+0000", 2024-09-13T09:12:44.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0800000000000002e-06 2024-09-13T09:12:44.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034848+0000", 2024-09-13T09:12:44.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6019999999999998e-06 2024-09-13T09:12:44.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.415 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.064876+0000", 2024-09-13T09:12:44.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.030027457 2024-09-13T09:12:44.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.064955+0000", 2024-09-13T09:12:44.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9311e-05 2024-09-13T09:12:44.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:44.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.065395+0000", 2024-09-13T09:12:44.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00043995999999999998 2024-09-13T09:12:44.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.417 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:44.417 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.067775+0000", 2024-09-13T09:12:44.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023797660000000002 2024-09-13T09:12:44.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.418 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:44.418 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069246+0000", 2024-09-13T09:12:44.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0014712709999999999 2024-09-13T09:12:44.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:44.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069279+0000", 2024-09-13T09:12:44.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3238999999999998e-05 2024-09-13T09:12:44.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069318+0000", 2024-09-13T09:12:44.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8798999999999999e-05 2024-09-13T09:12:44.420 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.420 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.421 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.421 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:16326 3.15 3:abeebc43:::rbd_object_map.112326fb43d3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.421 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.034851+0000", 2024-09-13T09:12:44.422 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.2600903629999998, 2024-09-13T09:12:44.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.034544643999999999, 2024-09-13T09:12:44.422 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.422 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:44.422 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.422 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.423 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.423 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 16326 2024-09-13T09:12:44.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.423 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034851+0000", 2024-09-13T09:12:44.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.424 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034851+0000", 2024-09-13T09:12:44.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034850+0000", 2024-09-13T09:12:44.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:12:44.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034852+0000", 2024-09-13T09:12:44.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.545e-06 2024-09-13T09:12:44.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034855+0000", 2024-09-13T09:12:44.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9249999999999999e-06 2024-09-13T09:12:44.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034857+0000", 2024-09-13T09:12:44.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.429e-06 2024-09-13T09:12:44.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.065853+0000", 2024-09-13T09:12:44.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.030995759000000001 2024-09-13T09:12:44.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.065942+0000", 2024-09-13T09:12:44.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8678999999999996e-05 2024-09-13T09:12:44.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:44.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.066317+0000", 2024-09-13T09:12:44.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00037573899999999999 2024-09-13T09:12:44.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:44.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.068830+0000", 2024-09-13T09:12:44.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025130360000000002 2024-09-13T09:12:44.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:44.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069356+0000", 2024-09-13T09:12:44.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052552000000000002 2024-09-13T09:12:44.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:44.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069379+0000", 2024-09-13T09:12:44.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2688999999999999e-05 2024-09-13T09:12:44.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069395+0000", 2024-09-13T09:12:44.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6551000000000001e-05 2024-09-13T09:12:44.429 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.430 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.430 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.430 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:16403 3.15 3:abeebc43:::rbd_object_map.112326fb43d3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.430 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.034859+0000", 2024-09-13T09:12:44.430 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.260081531, 2024-09-13T09:12:44.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.034626109000000002, 2024-09-13T09:12:44.430 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.430 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:44.430 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.431 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.431 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.431 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 16403 2024-09-13T09:12:44.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.431 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034859+0000", 2024-09-13T09:12:44.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034859+0000", 2024-09-13T09:12:44.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034859+0000", 2024-09-13T09:12:44.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:12:44.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034861+0000", 2024-09-13T09:12:44.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6780000000000001e-06 2024-09-13T09:12:44.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034864+0000", 2024-09-13T09:12:44.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7970000000000001e-06 2024-09-13T09:12:44.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034866+0000", 2024-09-13T09:12:44.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.441e-06 2024-09-13T09:12:44.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.067068+0000", 2024-09-13T09:12:44.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0322017 2024-09-13T09:12:44.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.067119+0000", 2024-09-13T09:12:44.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1286999999999999e-05 2024-09-13T09:12:44.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:44.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.067398+0000", 2024-09-13T09:12:44.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00027907000000000002 2024-09-13T09:12:44.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:44.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069061+0000", 2024-09-13T09:12:44.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0016628019999999999 2024-09-13T09:12:44.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:44.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069429+0000", 2024-09-13T09:12:44.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00036764499999999999 2024-09-13T09:12:44.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:44.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069452+0000", 2024-09-13T09:12:44.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3822999999999999e-05 2024-09-13T09:12:44.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069485+0000", 2024-09-13T09:12:44.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3074999999999998e-05 2024-09-13T09:12:44.436 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.436 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.436 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.436 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:16448 3.15 3:abeebc43:::rbd_object_map.112326fb43d3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.436 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.034868+0000", 2024-09-13T09:12:44.436 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.2600727620000001, 2024-09-13T09:12:44.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.034694738000000003, 2024-09-13T09:12:44.436 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.436 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:44.436 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.436 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.437 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.437 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 16448 2024-09-13T09:12:44.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.437 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034868+0000", 2024-09-13T09:12:44.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034868+0000", 2024-09-13T09:12:44.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.438 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.438 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034868+0000", 2024-09-13T09:12:44.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:12:44.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.438 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.438 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034869+0000", 2024-09-13T09:12:44.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4640000000000001e-06 2024-09-13T09:12:44.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.438 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.438 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034872+0000", 2024-09-13T09:12:44.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8100000000000002e-06 2024-09-13T09:12:44.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.034875+0000", 2024-09-13T09:12:44.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.367e-06 2024-09-13T09:12:44.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.067825+0000", 2024-09-13T09:12:44.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032950578000000001 2024-09-13T09:12:44.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.067913+0000", 2024-09-13T09:12:44.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7396000000000001e-05 2024-09-13T09:12:44.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:44.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.068286+0000", 2024-09-13T09:12:44.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000372968 2024-09-13T09:12:44.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:44.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069217+0000", 2024-09-13T09:12:44.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00093195500000000004 2024-09-13T09:12:44.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:44.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069524+0000", 2024-09-13T09:12:44.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00030699299999999998 2024-09-13T09:12:44.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:44.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069547+0000", 2024-09-13T09:12:44.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2289e-05 2024-09-13T09:12:44.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.069563+0000", 2024-09-13T09:12:44.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6137000000000002e-05 2024-09-13T09:12:44.441 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.441 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.442 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.442 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:16634 3.15 3:abeebc43:::rbd_object_map.112326fb43d3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:44.442 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.041620+0000", 2024-09-13T09:12:44.442 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.2533212389999999, 2024-09-13T09:12:44.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032813740000000001, 2024-09-13T09:12:44.442 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:44.442 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:44.442 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:44.442 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:44.442 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:44.442 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 16634 2024-09-13T09:12:44.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.443 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:44.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:44.443 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.041620+0000", 2024-09-13T09:12:44.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:44.443 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.041620+0000", 2024-09-13T09:12:44.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:44.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:44.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.041619+0000", 2024-09-13T09:12:44.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:44.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.444 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:44.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.041624+0000", 2024-09-13T09:12:44.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6590000000000004e-06 2024-09-13T09:12:44.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.444 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:44.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.041638+0000", 2024-09-13T09:12:44.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:12:44.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:44.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.041646+0000", 2024-09-13T09:12:44.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6909999999999997e-06 2024-09-13T09:12:44.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:44.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.070212+0000", 2024-09-13T09:12:44.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.028566059000000001 2024-09-13T09:12:44.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:44.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.070260+0000", 2024-09-13T09:12:44.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7666000000000001e-05 2024-09-13T09:12:44.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 4", 2024-09-13T09:12:44.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.070541+0000", 2024-09-13T09:12:44.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000280928 2024-09-13T09:12:44.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:44.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.072745+0000", 2024-09-13T09:12:44.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022047490000000002 2024-09-13T09:12:44.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:44.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.074376+0000", 2024-09-13T09:12:44.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001630835 2024-09-13T09:12:44.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:44.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.074401+0000", 2024-09-13T09:12:44.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4621e-05 2024-09-13T09:12:44.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:44.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:44.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:44.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.074433+0000", 2024-09-13T09:12:44.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2481999999999998e-05 2024-09-13T09:12:44.447 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.447 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.448 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.448 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:44.448 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:44.448 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:44.449 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:44.450 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:44.448+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:44.550 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:44.549+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:44.590 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:44.590 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:44.615 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:44.616 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:44.632 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:44.632 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:44.639 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:44.640 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:44.643 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:44.662 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:44.662 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:44.662 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:44.661+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:44.688 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:44.688 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:44.714 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:44.714 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:44.750 INFO:teuthology.orchestra.run.smithi157.stdout:2062 write 0x97e4800 thru 0x97e6cf3 (0x24f4 bytes) 2024-09-13T09:12:44.751 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:44.750+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:44.763 INFO:teuthology.orchestra.run.smithi157.stdout:2064 trunc from 0xba4e400 to 0x8245800 2024-09-13T09:12:44.805 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:44.806 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:44.815 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:44.831 INFO:teuthology.orchestra.run.smithi157.stdout:2065 punch from 0x102d200 to 0x103ba00, (0xe800 bytes) 2024-09-13T09:12:44.838 INFO:teuthology.orchestra.run.smithi157.stdout:2068 read 0x5666400 thru 0x566f14d (0x8d4e bytes) 2024-09-13T09:12:44.839 INFO:teuthology.orchestra.run.smithi157.stdout:2075 punch from 0x6a44000 to 0x6a4a200, (0x6200 bytes) 2024-09-13T09:12:44.843 INFO:teuthology.orchestra.run.smithi157.stdout:2076 trunc from 0x8245800 to 0xac0ba00 2024-09-13T09:12:44.851 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:44.850+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:44.883 INFO:teuthology.orchestra.run.smithi157.stdout:2080 read 0xbf9a00 thru 0xc073e7 (0xd9e8 bytes) 2024-09-13T09:12:44.888 INFO:teuthology.orchestra.run.smithi157.stdout:2083 read 0x9b5e600 thru 0x9b6a1f4 (0xbbf5 bytes) 2024-09-13T09:12:44.888 INFO:teuthology.orchestra.run.smithi157.stdout:2084 punch from 0x757ae00 to 0x7583400, (0x8600 bytes) 2024-09-13T09:12:44.888 INFO:teuthology.orchestra.run.smithi157.stdout:2086 read 0x5e73a00 thru 0x5e824a7 (0xeaa8 bytes) 2024-09-13T09:12:44.888 INFO:teuthology.orchestra.run.smithi157.stdout:2088 read 0x9809e00 thru 0x9810be4 (0x6de5 bytes) 2024-09-13T09:12:44.903 INFO:teuthology.orchestra.run.smithi157.stdout:2089 trunc from 0xac0ba00 to 0x15a800 2024-09-13T09:12:44.952 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:44.951+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:44.967 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:44.967 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:44.980 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:45.053 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:45.052+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:45.132 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:45.132 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:45.144 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:45.154 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:45.153+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:45.163 INFO:teuthology.orchestra.run.smithi157.stdout:2090 read 0x1f000 thru 0x21471 (0x2472 bytes) 2024-09-13T09:12:45.168 INFO:teuthology.orchestra.run.smithi157.stdout:2094 read 0xe1800 thru 0xefd5e (0xe55f bytes) 2024-09-13T09:12:45.169 INFO:teuthology.orchestra.run.smithi157.stdout:2095 read 0xfa200 thru 0xfc33f (0x2140 bytes) 2024-09-13T09:12:45.170 INFO:teuthology.orchestra.run.smithi157.stdout:2096 punch from 0xb6c00 to 0xc3200, (0xc600 bytes) 2024-09-13T09:12:45.184 INFO:teuthology.orchestra.run.smithi157.stdout:2097 write 0x1055200 thru 0x105c184 (0x6f85 bytes) 2024-09-13T09:12:45.185 INFO:teuthology.orchestra.run.smithi157.stdout:2098 write 0x8a2400 thru 0x8a4a7c (0x267d bytes) 2024-09-13T09:12:45.185 INFO:teuthology.orchestra.run.smithi157.stdout:2104 read 0x6eda00 thru 0x6efea0 (0x24a1 bytes) 2024-09-13T09:12:45.224 INFO:teuthology.orchestra.run.smithi157.stdout:2106 write 0x6cdbc00 thru 0x6ce19a0 (0x5da1 bytes) 2024-09-13T09:12:45.231 INFO:teuthology.orchestra.run.smithi157.stdout:2107 read 0x2605200 thru 0x260b1e8 (0x5fe9 bytes) 2024-09-13T09:12:45.255 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:45.254+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:45.301 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:45.301 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:45.309 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:45.356 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:45.355+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:45.456 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:45.457 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:45.457 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:45.456+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:45.465 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:45.558 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:45.557+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:45.568 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:45.569 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:45.577 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:45.659 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:45.658+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:45.680 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:45.680 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:45.688 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:45.760 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:45.759+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:45.789 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:45.790 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:45.798 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:45.861 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:45.860+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:45.901 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:45.901 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:45.911 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:45.995 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:45.994+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:46.002 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:12:46.052 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:46.053 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:46.061 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:46.062 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:46.061+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:46.164 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:46.162+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:46.203 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:46.203 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:46.212 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:46.264 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:46.263+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:46.272 INFO:teuthology.orchestra.run.smithi157.stdout:2108 write 0x9f4fc00 thru 0x9f4fca2 (0xa3 bytes) 2024-09-13T09:12:46.272 INFO:teuthology.orchestra.run.smithi157.stdout:2109 read 0x8853800 thru 0x8861c06 (0xe407 bytes) 2024-09-13T09:12:46.273 INFO:teuthology.orchestra.run.smithi157.stdout:2112 write 0x30ef000 thru 0x30f7cac (0x8cad bytes) 2024-09-13T09:12:46.273 INFO:teuthology.orchestra.run.smithi157.stdout:2113 read 0x313fc00 thru 0x31464bc (0x68bd bytes) 2024-09-13T09:12:46.302 INFO:teuthology.orchestra.run.smithi157.stdout:2118 write 0xd969800 thru 0xd96b2f0 (0x1af1 bytes) 2024-09-13T09:12:46.303 INFO:teuthology.orchestra.run.smithi157.stdout:2119 trunc from 0xd96b400 to 0xd889200 2024-09-13T09:12:46.326 INFO:teuthology.orchestra.run.smithi157.stdout:2120 read 0x4117600 thru 0x412430a (0xcd0b bytes) 2024-09-13T09:12:46.332 INFO:teuthology.orchestra.run.smithi157.stdout:2123 write 0x2c1de00 thru 0x2c2285c (0x4a5d bytes) 2024-09-13T09:12:46.337 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:46.337 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:46.347 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:46.351 INFO:teuthology.orchestra.run.smithi157.stdout:2124 trunc from 0xd889200 to 0x1c2e000 2024-09-13T09:12:46.391 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:46.390+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:46.414 INFO:teuthology.orchestra.run.smithi157.stdout:2125 trunc from 0x1c2e000 to 0x9c75600 2024-09-13T09:12:46.430 INFO:teuthology.orchestra.run.smithi157.stdout:2126 trunc from 0x9c75600 to 0x43d1200 2024-09-13T09:12:46.443 INFO:teuthology.orchestra.run.smithi157.stdout:2129 trunc from 0x43d1200 to 0x23c8000 2024-09-13T09:12:46.467 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:46.466+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:46.468 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:46.469 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:46.474 INFO:teuthology.orchestra.run.smithi157.stdout:2131 write 0xbdd2600 thru 0xbdd9a7b (0x747c bytes) 2024-09-13T09:12:46.475 INFO:teuthology.orchestra.run.smithi157.stdout:2132 write 0xb178400 thru 0xb1808f6 (0x84f7 bytes) 2024-09-13T09:12:46.475 INFO:teuthology.orchestra.run.smithi157.stdout:2134 trunc from 0xbdd9c00 to 0xc18e400 2024-09-13T09:12:46.480 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:46.518 INFO:teuthology.orchestra.run.smithi157.stdout:2136 write 0xc3df200 thru 0xc3eecb3 (0xfab4 bytes) 2024-09-13T09:12:46.519 INFO:teuthology.orchestra.run.smithi157.stdout:2137 trunc from 0xc3eee00 to 0xb318c00 2024-09-13T09:12:46.559 INFO:teuthology.orchestra.run.smithi157.stdout:2139 trunc from 0xb318c00 to 0x6c9b800 2024-09-13T09:12:46.568 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:46.567+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:46.576 INFO:teuthology.orchestra.run.smithi157.stdout:2140 write 0x42fc800 thru 0x4307ebd (0xb6be bytes) 2024-09-13T09:12:46.577 INFO:teuthology.orchestra.run.smithi157.stdout:2141 read 0x401f000 thru 0x402a17b (0xb17c bytes) 2024-09-13T09:12:46.577 INFO:teuthology.orchestra.run.smithi157.stdout:2142 punch from 0xe15600 to 0xe20200, (0xac00 bytes) 2024-09-13T09:12:46.600 INFO:teuthology.orchestra.run.smithi157.stdout:2145 write 0x797b000 thru 0x79813a7 (0x63a8 bytes) 2024-09-13T09:12:46.601 INFO:teuthology.orchestra.run.smithi157.stdout:2146 write 0x6ebae00 thru 0x6ebc707 (0x1908 bytes) 2024-09-13T09:12:46.603 INFO:teuthology.orchestra.run.smithi157.stdout:2149 trunc from 0x7981400 to 0x60dea00 2024-09-13T09:12:46.645 INFO:teuthology.orchestra.run.smithi157.stdout:2151 write 0x3f0e00 thru 0x3f7f10 (0x7111 bytes) 2024-09-13T09:12:46.655 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:46.656 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:46.668 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:46.669 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:46.668+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:46.673 INFO:teuthology.orchestra.run.smithi157.stdout:2153 write 0xbf27800 thru 0xbf33544 (0xbd45 bytes) 2024-09-13T09:12:46.674 INFO:teuthology.orchestra.run.smithi157.stdout:2154 read 0x6c9fa00 thru 0x6ca02fd (0x8fe bytes) 2024-09-13T09:12:46.674 INFO:teuthology.orchestra.run.smithi157.stdout:2156 read 0xbfa800 thru 0xc08d67 (0xe568 bytes) 2024-09-13T09:12:46.675 INFO:teuthology.orchestra.run.smithi157.stdout:2157 read 0x4db400 thru 0x4e4efb (0x9afc bytes) 2024-09-13T09:12:46.679 INFO:teuthology.orchestra.run.smithi157.stdout:2159 read 0x7a98000 thru 0x7a9fc86 (0x7c87 bytes) 2024-09-13T09:12:46.679 INFO:teuthology.orchestra.run.smithi157.stdout:2161 read 0x4294600 thru 0x42a1935 (0xd336 bytes) 2024-09-13T09:12:46.680 INFO:teuthology.orchestra.run.smithi157.stdout:2163 read 0x3f84c00 thru 0x3f92ef2 (0xe2f3 bytes) 2024-09-13T09:12:46.681 INFO:teuthology.orchestra.run.smithi157.stdout:2164 read 0x8797800 thru 0x87993e3 (0x1be4 bytes) 2024-09-13T09:12:46.681 INFO:teuthology.orchestra.run.smithi157.stdout:2166 punch from 0x836c800 to 0x8371a00, (0x5200 bytes) 2024-09-13T09:12:46.688 INFO:teuthology.orchestra.run.smithi157.stdout:2167 trunc from 0xbf33600 to 0x4fc7200 2024-09-13T09:12:46.727 INFO:teuthology.orchestra.run.smithi157.stdout:2168 trunc from 0x4fc7200 to 0x82d4000 2024-09-13T09:12:46.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:12:46.738 INFO:teuthology.orchestra.run.smithi157.stdout:2169 write 0x2ede800 thru 0x2ee347a (0x4c7b bytes) 2024-09-13T09:12:46.770 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:46.769+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:46.772 INFO:teuthology.orchestra.run.smithi157.stdout:2170 write 0xd952200 thru 0xd957999 (0x579a bytes) 2024-09-13T09:12:46.773 INFO:teuthology.orchestra.run.smithi157.stdout:2171 punch from 0xd946400 to 0xd947e00, (0x1a00 bytes) 2024-09-13T09:12:46.773 INFO:teuthology.orchestra.run.smithi157.stdout:2173 read 0x3ed7200 thru 0x3ee4514 (0xd315 bytes) 2024-09-13T09:12:46.774 INFO:teuthology.orchestra.run.smithi157.stdout:2175 clone 14 order 21 su 16384 sc 12 2024-09-13T09:12:46.791 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:46.791 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:46.801 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:46.860 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:46.861 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:46.871 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:46.870+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:46.877 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:46.877 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:46.893 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:46.893 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:46.908 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:46.908 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:46.908 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:46.908 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:46.908 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:46.910 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:46.910 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:46.917 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:46.927 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:46.927 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:46.943 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:46.944 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:46.996 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:46.995+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:47.026 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:47.026 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:47.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:47.026 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:47.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.027 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.027 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:12:47.027 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4949910019999999, 2024-09-13T09:12:47.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:12:47.027 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.027 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.027 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.027 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.027 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.027 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:12:47.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.027 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:12:47.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:12:47.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:12:47.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:12:47.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:12:47.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:12:47.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:12:47.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:12:47.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:12:47.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:12:47.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:12:47.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:12:47.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:12:47.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:12:47.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:12:47.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:12:47.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:12:47.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:12:47.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:12:47.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:12:47.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:12:47.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:12:47.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:12:47.032 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.032 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.033 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.033 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.033 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:12:47.033 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4949729669999998, 2024-09-13T09:12:47.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:12:47.033 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.033 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.033 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.033 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.033 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.033 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:12:47.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.034 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:12:47.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:12:47.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:12:47.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.036 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.036 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:12:47.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:12:47.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.036 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.036 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:12:47.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:12:47.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.036 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.036 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:12:47.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:12:47.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:12:47.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:12:47.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:12:47.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:12:47.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:12:47.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:12:47.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:12:47.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:12:47.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:12:47.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:12:47.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:12:47.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:12:47.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:12:47.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:12:47.039 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.039 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.039 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.040 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.040 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:12:47.040 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4949562969999999, 2024-09-13T09:12:47.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:12:47.040 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.040 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.040 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.040 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.040 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.040 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:12:47.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.040 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:12:47.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:12:47.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:12:47.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:12:47.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:12:47.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:12:47.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:12:47.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:12:47.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:12:47.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:12:47.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:12:47.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:12:47.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:12:47.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:12:47.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:12:47.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:12:47.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:12:47.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:12:47.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:12:47.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:12:47.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:12:47.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:12:47.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:12:47.046 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.046 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.046 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.046 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.046 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:12:47.046 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4949386650000003, 2024-09-13T09:12:47.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:12:47.046 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.046 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.047 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.047 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.047 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.047 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:12:47.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.047 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:12:47.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:12:47.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:12:47.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:12:47.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:12:47.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:12:47.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:12:47.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:12:47.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:12:47.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:12:47.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:12:47.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:12:47.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:12:47.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:12:47.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:12:47.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:12:47.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:12:47.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:12:47.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:12:47.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:12:47.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:12:47.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:12:47.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:12:47.052 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.052 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.052 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.052 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.052 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:12:47.052 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4948923599999997, 2024-09-13T09:12:47.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:12:47.052 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.052 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.053 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.053 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.053 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.053 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:12:47.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.053 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:12:47.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:12:47.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:12:47.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:12:47.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:12:47.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:12:47.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:12:47.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:12:47.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:12:47.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:12:47.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:12:47.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:12:47.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:12:47.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:12:47.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:12:47.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:12:47.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:12:47.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:12:47.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:12:47.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:12:47.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:12:47.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:12:47.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:12:47.058 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.058 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.058 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.058 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.058 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:12:47.058 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4948670589999997, 2024-09-13T09:12:47.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:12:47.058 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.058 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.058 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.059 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.059 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.059 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:12:47.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.059 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:12:47.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:12:47.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:12:47.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:12:47.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:12:47.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.061 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:12:47.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:12:47.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.061 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:12:47.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:12:47.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.061 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:12:47.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:12:47.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:12:47.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:12:47.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:12:47.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:12:47.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:12:47.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:12:47.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:12:47.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:12:47.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:12:47.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:12:47.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:12:47.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:12:47.064 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.064 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.064 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.064 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.064 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:12:47.064 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4948488300000005, 2024-09-13T09:12:47.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:12:47.064 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.064 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.064 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.065 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.065 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.065 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:12:47.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.065 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:12:47.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:12:47.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:12:47.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:12:47.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:12:47.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:12:47.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:12:47.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:12:47.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:12:47.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:12:47.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:12:47.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:12:47.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:12:47.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:12:47.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:12:47.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:12:47.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:12:47.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:12:47.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:12:47.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:12:47.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:12:47.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:12:47.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:12:47.070 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.070 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.070 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.070 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.070 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:12:47.070 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4948015110000004, 2024-09-13T09:12:47.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:12:47.070 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.070 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.070 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.070 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.071 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.071 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:12:47.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.071 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:12:47.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:12:47.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:12:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:12:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:12:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:12:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:12:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:12:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:12:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:12:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:12:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:12:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:12:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:12:47.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.075 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.075 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:12:47.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:12:47.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.075 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.075 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:12:47.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:12:47.075 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.075 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.075 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.075 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:12:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4942872730000003, 2024-09-13T09:12:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:12:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:12:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:12:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:12:47.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:12:47.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:47.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:12:47.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:12:47.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:12:47.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:12:47.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:12:47.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:12:47.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:12:47.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:12:47.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:12:47.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:12:47.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:12:47.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:12:47.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.080 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:12:47.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:12:47.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.080 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.080 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:12:47.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:12:47.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.080 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.080 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:12:47.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:12:47.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.081 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.081 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:12:47.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:12:47.081 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.081 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.081 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.081 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.081 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:12:47.081 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4942527429999997, 2024-09-13T09:12:47.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:12:47.081 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.082 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.082 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.082 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.082 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.082 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:12:47.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.082 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.082 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.082 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:12:47.082 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.083 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.083 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:12:47.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.083 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.083 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:12:47.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.083 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.083 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:12:47.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:12:47.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:12:47.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:12:47.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:12:47.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:12:47.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:12:47.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:12:47.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:12:47.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:12:47.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:12:47.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:12:47.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:12:47.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:12:47.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:12:47.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:12:47.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:12:47.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:12:47.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.087 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.087 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:12:47.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:12:47.087 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.087 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.087 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.087 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.087 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:12:47.087 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4942348360000004, 2024-09-13T09:12:47.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:12:47.087 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.087 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.088 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.088 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.088 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.088 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:12:47.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.088 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:12:47.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:12:47.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:12:47.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:12:47.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:12:47.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:12:47.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:12:47.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:12:47.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:12:47.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:12:47.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:12:47.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:12:47.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:12:47.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:12:47.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:12:47.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:12:47.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:12:47.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.092 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.092 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:12:47.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:12:47.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.092 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.092 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:12:47.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:12:47.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.092 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.092 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:12:47.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:12:47.092 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.093 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.093 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.093 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.093 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:12:47.093 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4851079880000002, 2024-09-13T09:12:47.093 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:12:47.093 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.093 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.093 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.093 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.093 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.093 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:12:47.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.094 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:12:47.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:12:47.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:12:47.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:12:47.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:12:47.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:12:47.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:12:47.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:12:47.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:12:47.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:12:47.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:12:47.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:12:47.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:12:47.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:12:47.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:12:47.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:12:47.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:12:47.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:12:47.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:12:47.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.098 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.098 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:12:47.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:12:47.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.098 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.098 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:12:47.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:12:47.098 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.098 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.098 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.099 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.099 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:12:47.099 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4850564080000002, 2024-09-13T09:12:47.099 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:12:47.099 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.099 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.099 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.099 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.099 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.099 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:12:47.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.100 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.100 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:12:47.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.100 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:12:47.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:12:47.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:12:47.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:12:47.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:12:47.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:12:47.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:12:47.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:12:47.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:12:47.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:12:47.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.103 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:12:47.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:12:47.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.103 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:12:47.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:12:47.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.104 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.104 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:12:47.104 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:12:47.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.104 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.104 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:12:47.104 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:12:47.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.104 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.104 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:12:47.104 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:12:47.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.105 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.105 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:12:47.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:12:47.105 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.105 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.105 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.105 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.105 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:12:47.105 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.485040132, 2024-09-13T09:12:47.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:12:47.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:12:47.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:12:47.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:12:47.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:12:47.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:12:47.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:12:47.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:12:47.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:12:47.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:12:47.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.109 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:12:47.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:12:47.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.109 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:12:47.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:12:47.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.109 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:12:47.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:12:47.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.110 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.110 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:12:47.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:12:47.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.110 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.110 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:12:47.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:12:47.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.110 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.110 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:12:47.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:12:47.111 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.111 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.111 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.111 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.111 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:12:47.111 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4850243580000004, 2024-09-13T09:12:47.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:12:47.111 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.111 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.111 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.111 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.112 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.112 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:12:47.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.112 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.112 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:12:47.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.112 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:12:47.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:12:47.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:12:47.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:12:47.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:12:47.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:12:47.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:12:47.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:12:47.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:12:47.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:12:47.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:12:47.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:12:47.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:12:47.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:12:47.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:12:47.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:12:47.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.116 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:12:47.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:12:47.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.116 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.116 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:12:47.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:12:47.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.116 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.116 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:12:47.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:12:47.116 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.116 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.117 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.117 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.117 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:12:47.117 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4850094829999998, 2024-09-13T09:12:47.117 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:12:47.117 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.117 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.117 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:12:47.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.118 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:12:47.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:12:47.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:12:47.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:12:47.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:12:47.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:12:47.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:12:47.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:12:47.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:12:47.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:12:47.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:12:47.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:12:47.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:12:47.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:12:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:12:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:12:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:12:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:12:47.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:12:47.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:12:47.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:12:47.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:12:47.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:12:47.122 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.122 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.122 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:12:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4849920210000001, 2024-09-13T09:12:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:12:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:12:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.123 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:12:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:12:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.124 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:12:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:12:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:12:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:12:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:12:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:12:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:12:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:12:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:12:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:12:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:12:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:12:47.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:12:47.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:12:47.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:12:47.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:12:47.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:12:47.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:12:47.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:12:47.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:12:47.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:12:47.128 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.128 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.128 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.129 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.129 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:12:47.129 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4833913890000003, 2024-09-13T09:12:47.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:12:47.129 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.129 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.129 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.129 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.129 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.129 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:12:47.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.129 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.130 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:12:47.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.130 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:12:47.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.130 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:12:47.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:12:47.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:12:47.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:12:47.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:12:47.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:12:47.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:12:47.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:12:47.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:12:47.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:12:47.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:12:47.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:12:47.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:12:47.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:12:47.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:12:47.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:12:47.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:12:47.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.134 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:12:47.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:12:47.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.134 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:12:47.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:12:47.134 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.134 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.134 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.134 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.135 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:12:47.135 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4833513380000003, 2024-09-13T09:12:47.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:12:47.135 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.135 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.135 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.135 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.135 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.135 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:12:47.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.135 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:12:47.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:12:47.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:12:47.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:12:47.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:12:47.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:12:47.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:12:47.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:12:47.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:12:47.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.138 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.138 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:12:47.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:12:47.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.138 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.138 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:12:47.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:12:47.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.138 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.138 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:12:47.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:12:47.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:12:47.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:12:47.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:12:47.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:12:47.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:12:47.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:12:47.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:12:47.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:12:47.140 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.140 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.140 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.140 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.140 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:12:47.141 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4833075749999995, 2024-09-13T09:12:47.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:12:47.141 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.141 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.141 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.141 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.141 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.141 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:12:47.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.141 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.141 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.141 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:12:47.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:12:47.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:12:47.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:12:47.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:12:47.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:12:47.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:12:47.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:12:47.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:12:47.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:12:47.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:12:47.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:12:47.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:12:47.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:12:47.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:12:47.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:47.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:12:47.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:12:47.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:12:47.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:12:47.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:12:47.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:12:47.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:12:47.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:12:47.146 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.146 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.146 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.146 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.147 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.147 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:47.148 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:47.149 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:47.147+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:47.247 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-09-13T09:12:47.248 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:47.247+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:47.298 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:47.298 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:47.307 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:47.326 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone13 from 0xd957a00 (overlap 0x15a800) to 0x67e00 2024-09-13T09:12:47.350 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:47.349+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:47.449 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:47.449+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:47.457 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:47.457 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:47.457 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:47.457 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:47.457 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:47.466 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:47.548 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-13T09:12:47.548 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T09:12:47.551 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:47.549+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:47.577 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:47.577 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:47.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:47.577 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:47.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.577 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.577 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:12:47.577 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0710064800000003, 2024-09-13T09:12:47.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:12:47.577 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.577 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.578 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.578 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.578 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.578 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:12:47.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.578 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:12:47.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:12:47.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:12:47.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:12:47.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:12:47.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:12:47.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:12:47.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:12:47.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:12:47.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:12:47.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:12:47.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:12:47.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:12:47.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:12:47.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:12:47.581 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.581 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.581 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.582 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.582 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:12:47.582 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0709787960000003, 2024-09-13T09:12:47.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:12:47.582 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.582 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.582 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.582 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.582 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.582 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:12:47.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.582 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:12:47.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:12:47.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:12:47.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:12:47.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:12:47.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:12:47.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:12:47.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:12:47.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:12:47.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:12:47.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:12:47.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:12:47.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:12:47.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:12:47.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:12:47.586 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.586 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.586 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.586 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.586 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:12:47.586 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0709544549999999, 2024-09-13T09:12:47.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:12:47.586 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.586 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.586 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.586 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.587 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.587 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:12:47.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.587 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:12:47.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:12:47.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:12:47.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:12:47.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:12:47.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:12:47.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:12:47.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:12:47.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:12:47.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:12:47.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:12:47.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:12:47.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:12:47.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:12:47.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:12:47.590 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.590 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.590 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.590 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.590 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:12:47.590 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0709381349999996, 2024-09-13T09:12:47.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:12:47.591 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.591 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.591 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.591 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.591 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.591 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:12:47.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.591 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:12:47.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:12:47.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:12:47.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:12:47.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:12:47.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:12:47.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:12:47.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:12:47.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:12:47.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:12:47.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:12:47.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:12:47.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:12:47.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:12:47.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:12:47.594 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.594 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:12:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0709217200000003, 2024-09-13T09:12:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:12:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:12:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:12:47.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:12:47.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:12:47.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:12:47.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:12:47.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:12:47.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:12:47.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:12:47.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:12:47.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:12:47.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:12:47.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:12:47.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:12:47.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:12:47.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:12:47.599 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.599 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.599 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.599 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.599 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:12:47.599 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0709056119999998, 2024-09-13T09:12:47.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:12:47.599 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.599 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.599 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.600 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.600 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.600 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:12:47.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.600 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:12:47.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:12:47.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:12:47.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:12:47.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:12:47.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:12:47.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:12:47.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.602 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:12:47.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:12:47.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.602 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:12:47.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:12:47.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.602 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:12:47.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:12:47.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:12:47.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:12:47.603 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.603 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.603 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.603 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.603 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:12:47.603 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0708900510000001, 2024-09-13T09:12:47.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:12:47.604 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.604 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.604 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.604 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.604 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.604 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:12:47.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.604 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:12:47.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:12:47.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:12:47.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:12:47.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:12:47.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:12:47.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:12:47.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:12:47.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:12:47.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:12:47.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:12:47.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:12:47.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:12:47.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:12:47.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:12:47.607 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.607 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.608 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.608 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.608 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:12:47.608 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0708732129999996, 2024-09-13T09:12:47.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:12:47.608 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.608 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.608 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.608 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.608 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.608 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:12:47.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.609 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:12:47.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:12:47.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:12:47.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:12:47.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:12:47.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:12:47.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:12:47.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:12:47.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:12:47.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:12:47.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:12:47.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:12:47.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:12:47.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:12:47.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:12:47.612 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.612 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.612 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.612 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.612 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:12:47.612 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0708572209999998, 2024-09-13T09:12:47.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:12:47.612 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.612 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.613 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.613 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.613 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.613 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:12:47.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.613 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:12:47.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:12:47.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:12:47.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:12:47.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:12:47.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:12:47.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:12:47.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:12:47.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:12:47.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:12:47.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:12:47.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:12:47.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:12:47.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:12:47.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:12:47.617 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.617 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.617 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.617 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.617 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:12:47.617 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0708297230000001, 2024-09-13T09:12:47.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:12:47.617 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.617 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.617 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.618 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.618 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.618 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:12:47.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.618 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:12:47.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:12:47.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:12:47.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:12:47.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:12:47.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:12:47.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:12:47.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:12:47.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:12:47.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:12:47.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:12:47.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:12:47.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:12:47.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:12:47.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:12:47.621 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.621 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.621 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.621 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.622 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:12:47.622 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0708038640000002, 2024-09-13T09:12:47.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:12:47.622 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.622 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.622 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.622 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.622 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.622 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:12:47.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.622 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:12:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:12:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:12:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:12:47.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:12:47.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:12:47.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:12:47.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:12:47.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:12:47.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:12:47.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:12:47.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:12:47.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:12:47.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:12:47.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:12:47.625 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.626 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.626 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.626 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.626 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:12:47.626 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0707623740000001, 2024-09-13T09:12:47.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:12:47.626 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.626 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.627 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.627 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.627 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.627 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:12:47.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.627 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:12:47.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:12:47.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:12:47.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:12:47.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:12:47.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:12:47.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:12:47.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:12:47.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:12:47.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:12:47.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:12:47.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:12:47.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:12:47.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:12:47.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:12:47.630 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.630 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.630 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.630 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:12:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0707411359999996, 2024-09-13T09:12:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:12:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:12:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:12:47.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:12:47.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:12:47.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:12:47.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:12:47.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:12:47.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:12:47.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:12:47.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:12:47.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:12:47.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:12:47.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:12:47.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:12:47.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:12:47.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:12:47.635 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.635 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.635 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.635 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.635 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:12:47.635 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0707183359999997, 2024-09-13T09:12:47.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:12:47.635 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.635 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.635 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.635 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.635 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.635 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:12:47.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.636 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:12:47.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:12:47.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:12:47.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:12:47.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:12:47.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:12:47.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:12:47.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:12:47.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:12:47.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:12:47.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:12:47.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:12:47.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:12:47.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:12:47.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:12:47.639 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.639 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.639 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.639 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.639 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:12:47.639 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0707018799999997, 2024-09-13T09:12:47.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:12:47.639 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.640 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.640 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.640 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.640 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.640 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:12:47.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.640 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:12:47.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:12:47.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:12:47.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:12:47.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:12:47.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:12:47.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:12:47.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:12:47.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:12:47.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:12:47.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:12:47.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:12:47.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:12:47.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:12:47.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:12:47.643 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.643 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.643 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.644 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.644 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:12:47.644 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0704567430000003, 2024-09-13T09:12:47.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:12:47.644 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.644 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.644 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.644 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.644 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.644 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:12:47.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.644 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:12:47.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:12:47.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:12:47.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:12:47.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:12:47.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:12:47.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:12:47.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:12:47.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:12:47.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:12:47.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:12:47.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:12:47.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:12:47.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:12:47.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:12:47.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:12:47.648 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.648 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.648 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.648 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20408 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5582848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.648 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.505992+0000", 2024-09-13T09:12:47.648 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0685921629999999, 2024-09-13T09:12:47.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050463591000000002, 2024-09-13T09:12:47.648 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.648 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.648 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.648 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.648 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20408 2024-09-13T09:12:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505992+0000", 2024-09-13T09:12:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505992+0000", 2024-09-13T09:12:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505991+0000", 2024-09-13T09:12:47.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505994+0000", 2024-09-13T09:12:47.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6259999999999999e-06 2024-09-13T09:12:47.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506002+0000", 2024-09-13T09:12:47.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1550000000000007e-06 2024-09-13T09:12:47.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506008+0000", 2024-09-13T09:12:47.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.553e-06 2024-09-13T09:12:47.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556375+0000", 2024-09-13T09:12:47.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050366850999999997 2024-09-13T09:12:47.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556412+0000", 2024-09-13T09:12:47.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7138999999999998e-05 2024-09-13T09:12:47.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556455+0000", 2024-09-13T09:12:47.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3733999999999998e-05 2024-09-13T09:12:47.652 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.652 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.652 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.652 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.652 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:12:47.652 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0685711839999996, 2024-09-13T09:12:47.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:12:47.653 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.653 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.653 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.653 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.653 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.653 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:12:47.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.653 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:12:47.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:12:47.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:12:47.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:12:47.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:12:47.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:12:47.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:12:47.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:12:47.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:12:47.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:12:47.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:12:47.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:12:47.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:12:47.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:12:47.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:12:47.656 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.656 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.656 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.657 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.657 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:12:47.657 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0685498180000002, 2024-09-13T09:12:47.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:12:47.657 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.657 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.657 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.657 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.657 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.657 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:12:47.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.657 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:12:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:12:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:12:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:12:47.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:12:47.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:12:47.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:12:47.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:12:47.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:12:47.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:12:47.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:12:47.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:12:47.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:12:47.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:12:47.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:12:47.661 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.661 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.661 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.661 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:47.661 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:12:47.661 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.0683444499999997, 2024-09-13T09:12:47.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:12:47.661 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:47.661 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:47.661 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:47.662 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:47.662 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:47.662 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:12:47.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.662 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:47.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:47.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:12:47.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:47.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:12:47.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:47.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:47.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:12:47.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:12:47.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:47.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:12:47.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:12:47.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:47.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:12:47.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:12:47.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:47.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:12:47.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:12:47.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:47.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:12:47.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:12:47.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:12:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:12:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:12:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:12:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:47.665 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:47.666 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:47.667 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:47.667 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:47.666+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:47.696 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-13T09:12:47.754 INFO:teuthology.orchestra.run.smithi157.stdout:2176 write 0xc9ab000 thru 0xc9b25f3 (0x75f4 bytes) 2024-09-13T09:12:47.754 INFO:teuthology.orchestra.run.smithi157.stdout:2179 write 0x72da400 thru 0x72e75a5 (0xd1a6 bytes) 2024-09-13T09:12:47.755 INFO:teuthology.orchestra.run.smithi157.stdout:2181 write 0x2250400 thru 0x2259d77 (0x9978 bytes) 2024-09-13T09:12:47.755 INFO:teuthology.orchestra.run.smithi157.stdout:2182 punch from 0x167b000 to 0x1685e00, (0xae00 bytes) 2024-09-13T09:12:47.768 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:47.767+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:47.844 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:47.844 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:47.844 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:47.844 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:47.848 INFO:teuthology.orchestra.run.smithi157.stdout:2184 write 0x51cba00 thru 0x51d4c7b (0x927c bytes) 2024-09-13T09:12:47.848 INFO:teuthology.orchestra.run.smithi157.stdout:2185 punch from 0xc565c00 to 0xc566400, (0x800 bytes) 2024-09-13T09:12:47.855 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:47.863 INFO:teuthology.orchestra.run.smithi157.stdout:2187 write 0x4a72e00 thru 0x4a77342 (0x4543 bytes) 2024-09-13T09:12:47.867 INFO:teuthology.orchestra.run.smithi157.stdout:2188 trunc from 0xd957a00 to 0xa047200 2024-09-13T09:12:47.899 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:47.898+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:47.936 INFO:teuthology.orchestra.run.smithi157.stdout:2189 write 0xc2d4a00 thru 0xc2dd92d (0x8f2e bytes) 2024-09-13T09:12:47.936 INFO:teuthology.orchestra.run.smithi157.stdout:2190 punch from 0x289b200 to 0x289e000, (0x2e00 bytes) 2024-09-13T09:12:47.955 INFO:teuthology.orchestra.run.smithi157.stdout:2191 write 0xd64f400 thru 0xd654806 (0x5407 bytes) 2024-09-13T09:12:47.955 INFO:teuthology.orchestra.run.smithi157.stdout:2193 write 0x2b67a00 thru 0x2b6f39e (0x799f bytes) 2024-09-13T09:12:47.956 INFO:teuthology.orchestra.run.smithi157.stdout:2196 punch from 0x8ad0c00 to 0x8adbe00, (0xb200 bytes) 2024-09-13T09:12:47.965 INFO:teuthology.orchestra.run.smithi157.stdout:2197 write 0x51e2800 thru 0x51e49b5 (0x21b6 bytes) 2024-09-13T09:12:47.969 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:47.969+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:47.974 INFO:teuthology.orchestra.run.smithi157.stdout:2201 trunc from 0xd654a00 to 0x63fac00 2024-09-13T09:12:47.976 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:47.976 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:47.976 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:47.977 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:47.977 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:47.986 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:48.019 INFO:teuthology.orchestra.run.smithi157.stdout:2202 write 0x2dee400 thru 0x2df863b (0xa23c bytes) 2024-09-13T09:12:48.037 INFO:teuthology.orchestra.run.smithi157.stdout:2203 write 0x64b1600 thru 0x64bcfde (0xb9df bytes) 2024-09-13T09:12:48.042 INFO:teuthology.orchestra.run.smithi157.stdout:2204 read 0x12c6400 thru 0x12cb125 (0x4d26 bytes) 2024-09-13T09:12:48.043 INFO:teuthology.orchestra.run.smithi157.stdout:2206 punch from 0x3da1e00 to 0x3dacc00, (0xae00 bytes) 2024-09-13T09:12:48.055 INFO:teuthology.orchestra.run.smithi157.stdout:2209 punch from 0x2e7f000 to 0x2e80600, (0x1600 bytes) 2024-09-13T09:12:48.055 INFO:teuthology.orchestra.run.smithi157.stdout:2213 read 0x15b4a00 thru 0x15c2931 (0xdf32 bytes) 2024-09-13T09:12:48.063 INFO:teuthology.orchestra.run.smithi157.stdout:2214 read 0x12bb200 thru 0x12c4245 (0x9046 bytes) 2024-09-13T09:12:48.063 INFO:teuthology.orchestra.run.smithi157.stdout:2215 write 0x38eaa00 thru 0x38f559e (0xab9f bytes) 2024-09-13T09:12:48.065 INFO:teuthology.orchestra.run.smithi157.stdout:2217 write 0x582c400 thru 0x583516f (0x8d70 bytes) 2024-09-13T09:12:48.065 INFO:teuthology.orchestra.run.smithi157.stdout:2220 write 0x476a200 thru 0x476e95a (0x475b bytes) 2024-09-13T09:12:48.070 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:48.069+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:48.100 INFO:teuthology.orchestra.run.smithi157.stdout:2221 write 0xd8afe00 thru 0xd8b7628 (0x7829 bytes) 2024-09-13T09:12:48.101 INFO:teuthology.orchestra.run.smithi157.stdout:2223 read 0x44d1a00 thru 0x44d845d (0x6a5e bytes) 2024-09-13T09:12:48.115 INFO:teuthology.orchestra.run.smithi157.stdout:2224 write 0xe7c4000 thru 0xe7c4819 (0x81a bytes) 2024-09-13T09:12:48.122 INFO:teuthology.orchestra.run.smithi157.stdout:2227 write 0x95e5c00 thru 0x95e884e (0x2c4f bytes) 2024-09-13T09:12:48.122 INFO:teuthology.orchestra.run.smithi157.stdout:2229 write 0x9719e00 thru 0x97221cb (0x83cc bytes) 2024-09-13T09:12:48.122 INFO:teuthology.orchestra.run.smithi157.stdout:2230 write 0xaab3800 thru 0xaac19ba (0xe1bb bytes) 2024-09-13T09:12:48.122 INFO:teuthology.orchestra.run.smithi157.stdout:2231 punch from 0x9c12a00 to 0x9c1c400, (0x9a00 bytes) 2024-09-13T09:12:48.122 INFO:teuthology.orchestra.run.smithi157.stdout:2232 read 0x39cc00 thru 0x39cc0d (0xe bytes) 2024-09-13T09:12:48.124 INFO:teuthology.orchestra.run.smithi157.stdout:2234 read 0xb9f1e00 thru 0xb9fade7 (0x8fe8 bytes) 2024-09-13T09:12:48.125 INFO:teuthology.orchestra.run.smithi157.stdout:2236 write 0xe1fca00 thru 0xe20a066 (0xd667 bytes) 2024-09-13T09:12:48.125 INFO:teuthology.orchestra.run.smithi157.stdout:2238 write 0x6569c00 thru 0x65750d0 (0xb4d1 bytes) 2024-09-13T09:12:48.126 INFO:teuthology.orchestra.run.smithi157.stdout:2239 write 0x2207600 thru 0x2215edb (0xe8dc bytes) 2024-09-13T09:12:48.127 INFO:teuthology.orchestra.run.smithi157.stdout:2240 punch from 0xb658600 to 0xb661000, (0x8a00 bytes) 2024-09-13T09:12:48.132 INFO:teuthology.orchestra.run.smithi157.stdout:2243 read 0xa572400 thru 0xa57d027 (0xac28 bytes) 2024-09-13T09:12:48.137 INFO:teuthology.orchestra.run.smithi157.stdout:2245 read 0x41eb000 thru 0x41ef24e (0x424f bytes) 2024-09-13T09:12:48.137 INFO:teuthology.orchestra.run.smithi157.stdout:2246 write 0x5b39400 thru 0x5b3a6b5 (0x12b6 bytes) 2024-09-13T09:12:48.138 INFO:teuthology.orchestra.run.smithi157.stdout:2247 read 0x1c17000 thru 0x1c1a6c4 (0x36c5 bytes) 2024-09-13T09:12:48.138 INFO:teuthology.orchestra.run.smithi157.stdout:2248 write 0x3f1b000 thru 0x3f22c7b (0x7c7c bytes) 2024-09-13T09:12:48.138 INFO:teuthology.orchestra.run.smithi157.stdout:2249 read 0x5615a00 thru 0x5622d95 (0xd396 bytes) 2024-09-13T09:12:48.139 INFO:teuthology.orchestra.run.smithi157.stdout:2251 read 0x17b7a00 thru 0x17c1628 (0x9c29 bytes) 2024-09-13T09:12:48.139 INFO:teuthology.orchestra.run.smithi157.stdout:2252 write 0x45b0600 thru 0x45b6dd5 (0x67d6 bytes) 2024-09-13T09:12:48.140 INFO:teuthology.orchestra.run.smithi157.stdout:2253 read 0x2260a00 thru 0x2260f49 (0x54a bytes) 2024-09-13T09:12:48.140 INFO:teuthology.orchestra.run.smithi157.stdout:2254 read 0xcce6e00 thru 0xccec1ea (0x53eb bytes) 2024-09-13T09:12:48.140 INFO:teuthology.orchestra.run.smithi157.stdout:2257 read 0xad8ec00 thru 0xad91e40 (0x3241 bytes) 2024-09-13T09:12:48.145 INFO:teuthology.orchestra.run.smithi157.stdout:2258 punch from 0x47c4200 to 0x47ce600, (0xa400 bytes) 2024-09-13T09:12:48.149 INFO:teuthology.orchestra.run.smithi157.stdout:2260 write 0xe0f6000 thru 0xe100de0 (0xade1 bytes) 2024-09-13T09:12:48.149 INFO:teuthology.orchestra.run.smithi157.stdout:2262 punch from 0x9115c00 to 0x911b800, (0x5c00 bytes) 2024-09-13T09:12:48.149 INFO:teuthology.orchestra.run.smithi157.stdout:2263 write 0xb65a400 thru 0xb66a048 (0xfc49 bytes) 2024-09-13T09:12:48.150 INFO:teuthology.orchestra.run.smithi157.stdout:2264 punch from 0xa5cde00 to 0xa5dac00, (0xce00 bytes) 2024-09-13T09:12:48.163 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:48.163 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:48.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:48.164 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:48.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.164 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:6867 3.13 3:cd293326:::rbd_header.112326fb43d3:head [watch reconnect cookie 140383087120544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T09:12:48.164 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:12:48.164 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.6324955640000001, 2024-09-13T09:12:48.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0277181360000001, 2024-09-13T09:12:48.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.164 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.164 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.165 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:12:48.165 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:12:48.165 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6867 2024-09-13T09:12:48.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.165 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:12:48.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:12:48.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529067+0000", 2024-09-13T09:12:48.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-09-13T09:12:48.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529073+0000", 2024-09-13T09:12:48.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:12:48.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529087+0000", 2024-09-13T09:12:48.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3825000000000001e-05 2024-09-13T09:12:48.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529097+0000", 2024-09-13T09:12:48.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9559999999999992e-06 2024-09-13T09:12:48.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529149+0000", 2024-09-13T09:12:48.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1668000000000001e-05 2024-09-13T09:12:48.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529161+0000", 2024-09-13T09:12:48.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1831000000000001e-05 2024-09-13T09:12:48.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.554941+0000", 2024-09-13T09:12:48.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0257799729999999 2024-09-13T09:12:48.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.555239+0000", 2024-09-13T09:12:48.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00029829100000000001 2024-09-13T09:12:48.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.556787+0000", 2024-09-13T09:12:48.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001547617 2024-09-13T09:12:48.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.170 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.170 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20898 3.13 3:cd293326:::rbd_header.112326fb43d3:head [watch reconnect cookie 140642395970896 gen 16] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T09:12:48.170 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:12:48.170 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.6278327460000002, 2024-09-13T09:12:48.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0236151950000001, 2024-09-13T09:12:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20898 2024-09-13T09:12:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:12:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:12:48.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533729+0000", 2024-09-13T09:12:48.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-09-13T09:12:48.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533736+0000", 2024-09-13T09:12:48.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9670000000000004e-06 2024-09-13T09:12:48.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533749+0000", 2024-09-13T09:12:48.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2449000000000001e-05 2024-09-13T09:12:48.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533757+0000", 2024-09-13T09:12:48.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:12:48.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533813+0000", 2024-09-13T09:12:48.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5683000000000002e-05 2024-09-13T09:12:48.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533823+0000", 2024-09-13T09:12:48.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0168e-05 2024-09-13T09:12:48.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.555940+0000", 2024-09-13T09:12:48.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0221170559999999 2024-09-13T09:12:48.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.556047+0000", 2024-09-13T09:12:48.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000106881 2024-09-13T09:12:48.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.557347+0000", 2024-09-13T09:12:48.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001299323 2024-09-13T09:12:48.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.176 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.176 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1852 3.7 3:e217568f:::rbd_data.112326fb43d3.0000000000000002:11 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.176 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798803+0000", 2024-09-13T09:12:48.176 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.362760623, 2024-09-13T09:12:48.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012724142000000001, 2024-09-13T09:12:48.176 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.177 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.177 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.177 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.177 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.177 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1852 2024-09-13T09:12:48.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.177 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798803+0000", 2024-09-13T09:12:48.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798803+0000", 2024-09-13T09:12:48.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798803+0000", 2024-09-13T09:12:48.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798805+0000", 2024-09-13T09:12:48.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9209999999999999e-06 2024-09-13T09:12:48.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798817+0000", 2024-09-13T09:12:48.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1627000000000001e-05 2024-09-13T09:12:48.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798821+0000", 2024-09-13T09:12:48.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.831e-06 2024-09-13T09:12:48.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.811381+0000", 2024-09-13T09:12:48.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012559572 2024-09-13T09:12:48.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.811463+0000", 2024-09-13T09:12:48.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2366000000000004e-05 2024-09-13T09:12:48.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.181 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.181 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.811528+0000", 2024-09-13T09:12:48.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4207000000000004e-05 2024-09-13T09:12:48.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.181 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.181 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1853 3.7 3:e217568f:::rbd_data.112326fb43d3.0000000000000002:11 [read 247808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.181 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798828+0000", 2024-09-13T09:12:48.182 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.36273575400000002, 2024-09-13T09:12:48.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012860399999999999, 2024-09-13T09:12:48.182 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.182 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.182 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.182 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.182 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.182 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1853 2024-09-13T09:12:48.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.182 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798828+0000", 2024-09-13T09:12:48.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798828+0000", 2024-09-13T09:12:48.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798828+0000", 2024-09-13T09:12:48.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798830+0000", 2024-09-13T09:12:48.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:12:48.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798837+0000", 2024-09-13T09:12:48.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8199999999999999e-06 2024-09-13T09:12:48.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798842+0000", 2024-09-13T09:12:48.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6580000000000001e-06 2024-09-13T09:12:48.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.811553+0000", 2024-09-13T09:12:48.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012711635000000001 2024-09-13T09:12:48.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.811626+0000", 2024-09-13T09:12:48.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2860000000000004e-05 2024-09-13T09:12:48.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.811689+0000", 2024-09-13T09:12:48.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2395000000000001e-05 2024-09-13T09:12:48.186 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.186 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.186 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.186 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1896 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.187 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798847+0000", 2024-09-13T09:12:48.187 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.36271746500000002, 2024-09-13T09:12:48.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01303903, 2024-09-13T09:12:48.187 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.187 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.187 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.187 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.187 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.187 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1896 2024-09-13T09:12:48.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.187 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798847+0000", 2024-09-13T09:12:48.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798847+0000", 2024-09-13T09:12:48.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798846+0000", 2024-09-13T09:12:48.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798848+0000", 2024-09-13T09:12:48.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.17e-06 2024-09-13T09:12:48.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798855+0000", 2024-09-13T09:12:48.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3330000000000002e-06 2024-09-13T09:12:48.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798859+0000", 2024-09-13T09:12:48.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3900000000000003e-06 2024-09-13T09:12:48.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.811715+0000", 2024-09-13T09:12:48.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012855775 2024-09-13T09:12:48.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.811792+0000", 2024-09-13T09:12:48.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6908000000000003e-05 2024-09-13T09:12:48.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.811886+0000", 2024-09-13T09:12:48.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3879e-05 2024-09-13T09:12:48.191 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.191 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.191 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.191 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1897 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.192 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798863+0000", 2024-09-13T09:12:48.192 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.36270078, 2024-09-13T09:12:48.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013203021000000001, 2024-09-13T09:12:48.192 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.192 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.192 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.192 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.192 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.192 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1897 2024-09-13T09:12:48.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.192 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798863+0000", 2024-09-13T09:12:48.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798863+0000", 2024-09-13T09:12:48.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798863+0000", 2024-09-13T09:12:48.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798865+0000", 2024-09-13T09:12:48.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9410000000000001e-06 2024-09-13T09:12:48.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798871+0000", 2024-09-13T09:12:48.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0589999999999999e-06 2024-09-13T09:12:48.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798875+0000", 2024-09-13T09:12:48.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.206e-06 2024-09-13T09:12:48.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.811910+0000", 2024-09-13T09:12:48.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013035147 2024-09-13T09:12:48.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.811983+0000", 2024-09-13T09:12:48.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2554999999999999e-05 2024-09-13T09:12:48.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.812066+0000", 2024-09-13T09:12:48.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3440000000000001e-05 2024-09-13T09:12:48.196 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.196 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.196 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.197 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1898 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.197 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798886+0000", 2024-09-13T09:12:48.197 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.36267788099999998, 2024-09-13T09:12:48.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014226477, 2024-09-13T09:12:48.197 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.197 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.197 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.197 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.197 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.198 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1898 2024-09-13T09:12:48.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.198 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798886+0000", 2024-09-13T09:12:48.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798886+0000", 2024-09-13T09:12:48.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798886+0000", 2024-09-13T09:12:48.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798895+0000", 2024-09-13T09:12:48.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7429999999999997e-06 2024-09-13T09:12:48.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798903+0000", 2024-09-13T09:12:48.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3180000000000001e-06 2024-09-13T09:12:48.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798908+0000", 2024-09-13T09:12:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2680000000000002e-06 2024-09-13T09:12:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.812091+0000", 2024-09-13T09:12:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013183122 2024-09-13T09:12:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.812162+0000", 2024-09-13T09:12:48.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0995999999999996e-05 2024-09-13T09:12:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813113+0000", 2024-09-13T09:12:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00095045500000000001 2024-09-13T09:12:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1899 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798913+0000", 2024-09-13T09:12:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.36265069500000002, 2024-09-13T09:12:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014390494, 2024-09-13T09:12:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1899 2024-09-13T09:12:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798913+0000", 2024-09-13T09:12:48.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798913+0000", 2024-09-13T09:12:48.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798913+0000", 2024-09-13T09:12:48.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798915+0000", 2024-09-13T09:12:48.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1890000000000001e-06 2024-09-13T09:12:48.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798921+0000", 2024-09-13T09:12:48.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2910000000000004e-06 2024-09-13T09:12:48.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798926+0000", 2024-09-13T09:12:48.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8740000000000003e-06 2024-09-13T09:12:48.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813139+0000", 2024-09-13T09:12:48.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014213046 2024-09-13T09:12:48.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813216+0000", 2024-09-13T09:12:48.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6464000000000005e-05 2024-09-13T09:12:48.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.207 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813304+0000", 2024-09-13T09:12:48.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8059999999999994e-05 2024-09-13T09:12:48.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.207 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.207 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1900 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.208 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798931+0000", 2024-09-13T09:12:48.208 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.362633343, 2024-09-13T09:12:48.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014961771, 2024-09-13T09:12:48.208 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.208 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.208 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.208 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.208 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.208 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1900 2024-09-13T09:12:48.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.208 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798931+0000", 2024-09-13T09:12:48.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798931+0000", 2024-09-13T09:12:48.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798930+0000", 2024-09-13T09:12:48.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798932+0000", 2024-09-13T09:12:48.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1110000000000002e-06 2024-09-13T09:12:48.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798938+0000", 2024-09-13T09:12:48.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0140000000000004e-06 2024-09-13T09:12:48.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.211 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.211 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798943+0000", 2024-09-13T09:12:48.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:12:48.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.211 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.211 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813330+0000", 2024-09-13T09:12:48.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014387525999999999 2024-09-13T09:12:48.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.211 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813407+0000", 2024-09-13T09:12:48.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6341999999999995e-05 2024-09-13T09:12:48.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.212 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813892+0000", 2024-09-13T09:12:48.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00048584499999999999 2024-09-13T09:12:48.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.212 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.213 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1901 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 139264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.213 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798947+0000", 2024-09-13T09:12:48.213 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.36261711400000002, 2024-09-13T09:12:48.213 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015125339, 2024-09-13T09:12:48.213 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.213 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.213 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.213 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.213 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.213 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1901 2024-09-13T09:12:48.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.214 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798947+0000", 2024-09-13T09:12:48.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798947+0000", 2024-09-13T09:12:48.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.215 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.215 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798947+0000", 2024-09-13T09:12:48.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.215 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.215 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798949+0000", 2024-09-13T09:12:48.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0420000000000001e-06 2024-09-13T09:12:48.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.215 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.215 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798955+0000", 2024-09-13T09:12:48.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.879e-06 2024-09-13T09:12:48.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798959+0000", 2024-09-13T09:12:48.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4299999999999999e-06 2024-09-13T09:12:48.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813920+0000", 2024-09-13T09:12:48.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014961502 2024-09-13T09:12:48.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.217 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.217 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814007+0000", 2024-09-13T09:12:48.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6643000000000004e-05 2024-09-13T09:12:48.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.217 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.217 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814072+0000", 2024-09-13T09:12:48.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5183000000000003e-05 2024-09-13T09:12:48.217 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.217 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.218 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.218 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1902 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 157696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.218 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798963+0000", 2024-09-13T09:12:48.218 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.36260084199999998, 2024-09-13T09:12:48.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01528532, 2024-09-13T09:12:48.218 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.218 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.218 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.218 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.218 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.219 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1902 2024-09-13T09:12:48.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.219 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798963+0000", 2024-09-13T09:12:48.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798963+0000", 2024-09-13T09:12:48.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798963+0000", 2024-09-13T09:12:48.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798965+0000", 2024-09-13T09:12:48.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9750000000000001e-06 2024-09-13T09:12:48.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798971+0000", 2024-09-13T09:12:48.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:12:48.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798975+0000", 2024-09-13T09:12:48.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3479999999999997e-06 2024-09-13T09:12:48.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814104+0000", 2024-09-13T09:12:48.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015128697 2024-09-13T09:12:48.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.222 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814181+0000", 2024-09-13T09:12:48.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7319999999999998e-05 2024-09-13T09:12:48.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.222 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814249+0000", 2024-09-13T09:12:48.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7436999999999998e-05 2024-09-13T09:12:48.222 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.223 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.223 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.223 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1903 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 192512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.223 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798979+0000", 2024-09-13T09:12:48.223 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.36258477500000003, 2024-09-13T09:12:48.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015442285, 2024-09-13T09:12:48.223 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.223 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.223 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.223 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.224 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.224 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1903 2024-09-13T09:12:48.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.224 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798979+0000", 2024-09-13T09:12:48.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798979+0000", 2024-09-13T09:12:48.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798979+0000", 2024-09-13T09:12:48.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798981+0000", 2024-09-13T09:12:48.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9479999999999998e-06 2024-09-13T09:12:48.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798987+0000", 2024-09-13T09:12:48.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8159999999999999e-06 2024-09-13T09:12:48.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798991+0000", 2024-09-13T09:12:48.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4950000000000002e-06 2024-09-13T09:12:48.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.227 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814278+0000", 2024-09-13T09:12:48.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015286978999999999 2024-09-13T09:12:48.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.227 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.227 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814358+0000", 2024-09-13T09:12:48.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.941e-05 2024-09-13T09:12:48.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.227 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.227 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814422+0000", 2024-09-13T09:12:48.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3979000000000005e-05 2024-09-13T09:12:48.228 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.228 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.228 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.228 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1904 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 210944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.228 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798995+0000", 2024-09-13T09:12:48.228 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.36256867199999998, 2024-09-13T09:12:48.228 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015597725999999999, 2024-09-13T09:12:48.228 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.228 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.229 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.229 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.229 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.229 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1904 2024-09-13T09:12:48.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.229 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.229 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.229 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798995+0000", 2024-09-13T09:12:48.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.230 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.230 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798995+0000", 2024-09-13T09:12:48.230 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.230 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.232 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798995+0000", 2024-09-13T09:12:48.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.232 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.232 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798997+0000", 2024-09-13T09:12:48.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.84e-06 2024-09-13T09:12:48.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.232 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.232 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799002+0000", 2024-09-13T09:12:48.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3809999999999998e-06 2024-09-13T09:12:48.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.233 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.233 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799007+0000", 2024-09-13T09:12:48.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3630000000000004e-06 2024-09-13T09:12:48.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.233 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.233 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814448+0000", 2024-09-13T09:12:48.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015441041000000001 2024-09-13T09:12:48.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814525+0000", 2024-09-13T09:12:48.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7539000000000004e-05 2024-09-13T09:12:48.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814593+0000", 2024-09-13T09:12:48.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8009000000000006e-05 2024-09-13T09:12:48.234 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.234 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.235 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.235 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1905 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.235 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.799011+0000", 2024-09-13T09:12:48.235 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.36255353899999998, 2024-09-13T09:12:48.235 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015754799, 2024-09-13T09:12:48.235 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.235 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.235 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.235 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.236 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.236 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1905 2024-09-13T09:12:48.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.236 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.236 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799011+0000", 2024-09-13T09:12:48.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.236 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799011+0000", 2024-09-13T09:12:48.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799010+0000", 2024-09-13T09:12:48.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799012+0000", 2024-09-13T09:12:48.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9570000000000001e-06 2024-09-13T09:12:48.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799018+0000", 2024-09-13T09:12:48.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9689999999999999e-06 2024-09-13T09:12:48.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799022+0000", 2024-09-13T09:12:48.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1899999999999997e-06 2024-09-13T09:12:48.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814620+0000", 2024-09-13T09:12:48.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015597313 2024-09-13T09:12:48.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814701+0000", 2024-09-13T09:12:48.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1403999999999995e-05 2024-09-13T09:12:48.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814765+0000", 2024-09-13T09:12:48.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4302999999999997e-05 2024-09-13T09:12:48.240 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.240 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.240 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.240 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:2310 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 51200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.240 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.805919+0000", 2024-09-13T09:12:48.240 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.35564470599999998, 2024-09-13T09:12:48.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012641813, 2024-09-13T09:12:48.240 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.240 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.241 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2310 2024-09-13T09:12:48.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.241 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.241 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.805919+0000", 2024-09-13T09:12:48.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.805919+0000", 2024-09-13T09:12:48.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.805919+0000", 2024-09-13T09:12:48.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.805921+0000", 2024-09-13T09:12:48.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1749999999999999e-06 2024-09-13T09:12:48.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.243 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.805927+0000", 2024-09-13T09:12:48.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8030000000000002e-06 2024-09-13T09:12:48.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.243 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.805932+0000", 2024-09-13T09:12:48.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7759999999999997e-06 2024-09-13T09:12:48.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.817902+0000", 2024-09-13T09:12:48.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011969802 2024-09-13T09:12:48.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.817974+0000", 2024-09-13T09:12:48.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2745999999999998e-05 2024-09-13T09:12:48.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.245 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.818561+0000", 2024-09-13T09:12:48.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00058688299999999996 2024-09-13T09:12:48.245 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.245 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.245 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.245 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:2312 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.245 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.806607+0000", 2024-09-13T09:12:48.245 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.35495735, 2024-09-13T09:12:48.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012718515, 2024-09-13T09:12:48.246 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.246 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.246 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.246 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.246 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.246 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2312 2024-09-13T09:12:48.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.246 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806607+0000", 2024-09-13T09:12:48.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806607+0000", 2024-09-13T09:12:48.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806606+0000", 2024-09-13T09:12:48.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.248 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806609+0000", 2024-09-13T09:12:48.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8009999999999999e-06 2024-09-13T09:12:48.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.248 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806616+0000", 2024-09-13T09:12:48.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2089999999999999e-06 2024-09-13T09:12:48.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806621+0000", 2024-09-13T09:12:48.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7960000000000003e-06 2024-09-13T09:12:48.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.819059+0000", 2024-09-13T09:12:48.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012437643 2024-09-13T09:12:48.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.819140+0000", 2024-09-13T09:12:48.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1809000000000002e-05 2024-09-13T09:12:48.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.250 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.250 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.819325+0000", 2024-09-13T09:12:48.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018486199999999999 2024-09-13T09:12:48.250 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.250 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.250 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.250 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:2313 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 122880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.250 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.806626+0000", 2024-09-13T09:12:48.251 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.35493835699999998, 2024-09-13T09:12:48.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012906908, 2024-09-13T09:12:48.251 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.251 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.251 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.251 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.251 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.251 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2313 2024-09-13T09:12:48.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.251 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806626+0000", 2024-09-13T09:12:48.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806626+0000", 2024-09-13T09:12:48.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806625+0000", 2024-09-13T09:12:48.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806628+0000", 2024-09-13T09:12:48.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.689e-06 2024-09-13T09:12:48.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806634+0000", 2024-09-13T09:12:48.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:12:48.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806639+0000", 2024-09-13T09:12:48.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8300000000000003e-06 2024-09-13T09:12:48.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.819353+0000", 2024-09-13T09:12:48.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012714052999999999 2024-09-13T09:12:48.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.819434+0000", 2024-09-13T09:12:48.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0605999999999994e-05 2024-09-13T09:12:48.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.819533+0000", 2024-09-13T09:12:48.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8628000000000005e-05 2024-09-13T09:12:48.255 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.255 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.255 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.255 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:2314 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 141312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.255 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.806644+0000", 2024-09-13T09:12:48.256 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.35492025999999999, 2024-09-13T09:12:48.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013076213, 2024-09-13T09:12:48.256 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.256 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.256 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.256 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.256 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.256 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2314 2024-09-13T09:12:48.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.256 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806644+0000", 2024-09-13T09:12:48.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806644+0000", 2024-09-13T09:12:48.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806643+0000", 2024-09-13T09:12:48.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806646+0000", 2024-09-13T09:12:48.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2409999999999998e-06 2024-09-13T09:12:48.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806652+0000", 2024-09-13T09:12:48.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2759999999999997e-06 2024-09-13T09:12:48.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806657+0000", 2024-09-13T09:12:48.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.853e-06 2024-09-13T09:12:48.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.819559+0000", 2024-09-13T09:12:48.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012902142 2024-09-13T09:12:48.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.819633+0000", 2024-09-13T09:12:48.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3936000000000005e-05 2024-09-13T09:12:48.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.819720+0000", 2024-09-13T09:12:48.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7140999999999998e-05 2024-09-13T09:12:48.260 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.260 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.261 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.261 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:2315 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 176128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.261 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.806661+0000", 2024-09-13T09:12:48.261 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.35490260099999998, 2024-09-13T09:12:48.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013275609000000001, 2024-09-13T09:12:48.261 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.261 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.261 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.261 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.262 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.262 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2315 2024-09-13T09:12:48.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.262 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806661+0000", 2024-09-13T09:12:48.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806661+0000", 2024-09-13T09:12:48.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806661+0000", 2024-09-13T09:12:48.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806663+0000", 2024-09-13T09:12:48.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.119e-06 2024-09-13T09:12:48.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806669+0000", 2024-09-13T09:12:48.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:12:48.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806674+0000", 2024-09-13T09:12:48.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9030000000000003e-06 2024-09-13T09:12:48.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.819750+0000", 2024-09-13T09:12:48.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013075612 2024-09-13T09:12:48.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.819845+0000", 2024-09-13T09:12:48.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5416999999999998e-05 2024-09-13T09:12:48.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.819937+0000", 2024-09-13T09:12:48.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1897000000000005e-05 2024-09-13T09:12:48.266 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.266 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.266 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.266 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:2316 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:48.266 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.806678+0000", 2024-09-13T09:12:48.266 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.35488556700000001, 2024-09-13T09:12:48.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013463803, 2024-09-13T09:12:48.266 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:48.267 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:48.267 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:48.267 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:48.267 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:48.267 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2316 2024-09-13T09:12:48.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.267 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:48.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:48.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806678+0000", 2024-09-13T09:12:48.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.268 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:48.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806678+0000", 2024-09-13T09:12:48.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:48.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.268 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:48.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806678+0000", 2024-09-13T09:12:48.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:48.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806680+0000", 2024-09-13T09:12:48.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:12:48.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:48.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806686+0000", 2024-09-13T09:12:48.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1079999999999998e-06 2024-09-13T09:12:48.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.806691+0000", 2024-09-13T09:12:48.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5390000000000001e-06 2024-09-13T09:12:48.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:48.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.819964+0000", 2024-09-13T09:12:48.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013272994 2024-09-13T09:12:48.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:48.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.820056+0000", 2024-09-13T09:12:48.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2343999999999995e-05 2024-09-13T09:12:48.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:48.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:48.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:48.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.820142+0000", 2024-09-13T09:12:48.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6046999999999998e-05 2024-09-13T09:12:48.271 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.271 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.271 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.271 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:48.271 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:48.271 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:48.272 INFO:teuthology.orchestra.run.smithi157.stdout:2266 trunc from 0xe7c4a00 to 0x954a00 2024-09-13T09:12:48.273 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:48.272+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:48.274 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:48.274 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T09:12:48.277 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T09:12:48.325 INFO:teuthology.orchestra.run.smithi157.stdout:2268 read 0x547c00 thru 0x55766c (0xfa6d bytes) 2024-09-13T09:12:48.332 INFO:teuthology.orchestra.run.smithi157.stdout:2272 punch from 0x1fc200 to 0x20aa00, (0xe800 bytes) 2024-09-13T09:12:48.332 INFO:teuthology.orchestra.run.smithi157.stdout:2274 read 0x916200 thru 0x91d4a2 (0x72a3 bytes) 2024-09-13T09:12:48.348 INFO:teuthology.orchestra.run.smithi157.stdout:2275 write 0x46d9000 thru 0x46e3fce (0xafcf bytes) 2024-09-13T09:12:48.354 INFO:teuthology.orchestra.run.smithi157.stdout:2276 punch from 0x138ba00 to 0x1399400, (0xda00 bytes) 2024-09-13T09:12:48.371 INFO:teuthology.orchestra.run.smithi157.stdout:2278 write 0xa91fc00 thru 0xa921726 (0x1b27 bytes) 2024-09-13T09:12:48.373 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:48.373+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:48.384 INFO:teuthology.orchestra.run.smithi157.stdout:2280 trunc from 0xa921800 to 0x4c51000 2024-09-13T09:12:48.385 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:48.386 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:48.396 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:48.430 INFO:teuthology.orchestra.run.smithi157.stdout:2281 write 0xa2faa00 thru 0xa301d26 (0x7327 bytes) 2024-09-13T09:12:48.431 INFO:teuthology.orchestra.run.smithi157.stdout:2283 read 0x9ef4200 thru 0x9efaa8f (0x6890 bytes) 2024-09-13T09:12:48.431 INFO:teuthology.orchestra.run.smithi157.stdout:2285 punch from 0x247d400 to 0x2485800, (0x8400 bytes) 2024-09-13T09:12:48.431 INFO:teuthology.orchestra.run.smithi157.stdout:2286 punch from 0x9262c00 to 0x9267c00, (0x5000 bytes) 2024-09-13T09:12:48.431 INFO:teuthology.orchestra.run.smithi157.stdout:2287 read 0x51e7600 thru 0x51f7430 (0xfe31 bytes) 2024-09-13T09:12:48.431 INFO:teuthology.orchestra.run.smithi157.stdout:2288 write 0x22abe00 thru 0x22bb343 (0xf544 bytes) 2024-09-13T09:12:48.431 INFO:teuthology.orchestra.run.smithi157.stdout:2289 punch from 0x2166a00 to 0x2171200, (0xa800 bytes) 2024-09-13T09:12:48.442 INFO:teuthology.orchestra.run.smithi157.stdout:2290 trunc from 0xa301e00 to 0xc34a00 2024-09-13T09:12:48.470 INFO:teuthology.orchestra.run.smithi157.stdout:2291 read 0x6a8e00 thru 0x6ae56b (0x576c bytes) 2024-09-13T09:12:48.471 INFO:teuthology.orchestra.run.smithi157.stdout:2294 punch from 0x56ca00 to 0x574200, (0x7800 bytes) 2024-09-13T09:12:48.471 INFO:teuthology.orchestra.run.smithi157.stdout:2295 read 0x81400 thru 0x8aedd (0x9ade bytes) 2024-09-13T09:12:48.471 INFO:teuthology.orchestra.run.smithi157.stdout:2296 read 0x72e000 thru 0x73da31 (0xfa32 bytes) 2024-09-13T09:12:48.472 INFO:teuthology.orchestra.run.smithi157.stdout:2297 read 0x83a200 thru 0x83c963 (0x2764 bytes) 2024-09-13T09:12:48.473 INFO:teuthology.orchestra.run.smithi157.stdout:2298 punch from 0xa0aa00 to 0xa0bc00, (0x1200 bytes) 2024-09-13T09:12:48.473 INFO:teuthology.orchestra.run.smithi157.stdout:2300 read 0x360a00 thru 0x360d5a (0x35b bytes) 2024-09-13T09:12:48.473 INFO:teuthology.orchestra.run.smithi157.stdout:2302 read 0x5dc800 thru 0x5eafb3 (0xe7b4 bytes) 2024-09-13T09:12:48.474 INFO:teuthology.orchestra.run.smithi157.stdout:2303 read 0xadaa00 thru 0xae8548 (0xdb49 bytes) 2024-09-13T09:12:48.475 INFO:teuthology.orchestra.run.smithi157.stdout:2308 punch from 0xb89400 to 0xb97000, (0xdc00 bytes) 2024-09-13T09:12:48.475 INFO:teuthology.orchestra.run.smithi157.stdout:2310 read 0xa9d600 thru 0xaaa844 (0xd245 bytes) 2024-09-13T09:12:48.477 INFO:teuthology.orchestra.run.smithi157.stdout:2311 write 0x349000 thru 0x355e36 (0xce37 bytes) 2024-09-13T09:12:48.477 INFO:teuthology.orchestra.run.smithi157.stdout:2314 read 0x284800 thru 0x29000f (0xb810 bytes) 2024-09-13T09:12:48.478 INFO:teuthology.orchestra.run.smithi157.stdout:2315 read 0x46bc00 thru 0x47973f (0xdb40 bytes) 2024-09-13T09:12:48.482 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:48.482+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:48.488 INFO:teuthology.orchestra.run.smithi157.stdout:2316 write 0x163f600 thru 0x164820a (0x8c0b bytes) 2024-09-13T09:12:48.502 INFO:teuthology.orchestra.run.smithi157.stdout:2317 write 0x3299800 thru 0x32a0299 (0x6a9a bytes) 2024-09-13T09:12:48.503 INFO:teuthology.orchestra.run.smithi157.stdout:2324 read 0x2667400 thru 0x266f2e3 (0x7ee4 bytes) 2024-09-13T09:12:48.503 INFO:teuthology.orchestra.run.smithi157.stdout:2326 write 0x2dd0600 thru 0x2dd0adb (0x4dc bytes) 2024-09-13T09:12:48.503 INFO:teuthology.orchestra.run.smithi157.stdout:2328 read 0x2ff1400 thru 0x2ffe363 (0xcf64 bytes) 2024-09-13T09:12:48.506 INFO:teuthology.orchestra.run.smithi157.stdout:2333 read 0xc0a800 thru 0xc16247 (0xba48 bytes) 2024-09-13T09:12:48.508 INFO:teuthology.orchestra.run.smithi157.stdout:2335 trunc from 0x32a0400 to 0x4a21600 2024-09-13T09:12:48.544 INFO:teuthology.orchestra.run.smithi157.stdout:2336 write 0xd20ea00 thru 0xd211241 (0x2842 bytes) 2024-09-13T09:12:48.545 INFO:teuthology.orchestra.run.smithi157.stdout:2338 write 0x2e9be00 thru 0x2ea1555 (0x5756 bytes) 2024-09-13T09:12:48.545 INFO:teuthology.orchestra.run.smithi157.stdout:2339 write 0xa722a00 thru 0xa72890b (0x5f0c bytes) 2024-09-13T09:12:48.545 INFO:teuthology.orchestra.run.smithi157.stdout:2341 punch from 0x503aa00 to 0x5041400, (0x6a00 bytes) 2024-09-13T09:12:48.545 INFO:teuthology.orchestra.run.smithi157.stdout:2342 write 0x4157c00 thru 0x41672d4 (0xf6d5 bytes) 2024-09-13T09:12:48.546 INFO:teuthology.orchestra.run.smithi157.stdout:2344 read 0x55f4a00 thru 0x5602e10 (0xe411 bytes) 2024-09-13T09:12:48.546 INFO:teuthology.orchestra.run.smithi157.stdout:2345 punch from 0x87b7200 to 0x87be000, (0x6e00 bytes) 2024-09-13T09:12:48.546 INFO:teuthology.orchestra.run.smithi157.stdout:2346 write 0x79d2000 thru 0x79dfe81 (0xde82 bytes) 2024-09-13T09:12:48.547 INFO:teuthology.orchestra.run.smithi157.stdout:2347 read 0x14e9200 thru 0x14ef62b (0x642c bytes) 2024-09-13T09:12:48.547 INFO:teuthology.orchestra.run.smithi157.stdout:2349 read 0x44fac00 thru 0x4508ee4 (0xe2e5 bytes) 2024-09-13T09:12:48.547 INFO:teuthology.orchestra.run.smithi157.stdout:2352 punch from 0x7b59000 to 0x7b67600, (0xe600 bytes) 2024-09-13T09:12:48.547 INFO:teuthology.orchestra.run.smithi157.stdout:2354 read 0xfb2e00 thru 0xfc133f (0xe540 bytes) 2024-09-13T09:12:48.564 INFO:teuthology.orchestra.run.smithi157.stdout:2355 write 0x1683000 thru 0x168d4fb (0xa4fc bytes) 2024-09-13T09:12:48.564 INFO:teuthology.orchestra.run.smithi157.stdout:2356 read 0x8c77400 thru 0x8c7fc85 (0x8886 bytes) 2024-09-13T09:12:48.564 INFO:teuthology.orchestra.run.smithi157.stdout:2358 read 0xb86400 thru 0xb910b2 (0xacb3 bytes) 2024-09-13T09:12:48.566 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:48.566 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:48.566 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:48.566 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:48.566 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:48.574 INFO:teuthology.orchestra.run.smithi157.stdout:2360 trunc from 0xd211400 to 0x5be1000 2024-09-13T09:12:48.575 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:48.574+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:48.576 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:48.592 INFO:teuthology.orchestra.run.smithi157.stdout:2362 read 0x4341400 thru 0x4345086 (0x3c87 bytes) 2024-09-13T09:12:48.593 INFO:teuthology.orchestra.run.smithi157.stdout:2363 punch from 0x4e56a00 to 0x4e5ac00, (0x4200 bytes) 2024-09-13T09:12:48.593 INFO:teuthology.orchestra.run.smithi157.stdout:2365 read 0xd73400 thru 0xd79442 (0x6043 bytes) 2024-09-13T09:12:48.593 INFO:teuthology.orchestra.run.smithi157.stdout:2368 read 0x374d000 thru 0x375ba43 (0xea44 bytes) 2024-09-13T09:12:48.594 INFO:teuthology.orchestra.run.smithi157.stdout:2372 punch from 0x6f0a00 to 0x6f8c00, (0x8200 bytes) 2024-09-13T09:12:48.613 INFO:teuthology.orchestra.run.smithi157.stdout:2374 write 0xd8e4a00 thru 0xd8eef13 (0xa514 bytes) 2024-09-13T09:12:48.614 INFO:teuthology.orchestra.run.smithi157.stdout:2375 write 0xbf77e00 thru 0xbf7833b (0x53c bytes) 2024-09-13T09:12:48.614 INFO:teuthology.orchestra.run.smithi157.stdout:2377 write 0x72c7400 thru 0x72d14df (0xa0e0 bytes) 2024-09-13T09:12:48.614 INFO:teuthology.orchestra.run.smithi157.stdout:2378 read 0x89e6c00 thru 0x89ee068 (0x7469 bytes) 2024-09-13T09:12:48.614 INFO:teuthology.orchestra.run.smithi157.stdout:2379 read 0x7658e00 thru 0x765f704 (0x6905 bytes) 2024-09-13T09:12:48.615 INFO:teuthology.orchestra.run.smithi157.stdout:2380 read 0x79b3000 thru 0x79ba4a6 (0x74a7 bytes) 2024-09-13T09:12:48.615 INFO:teuthology.orchestra.run.smithi157.stdout:2381 write 0x7552600 thru 0x7558b1e (0x651f bytes) 2024-09-13T09:12:48.615 INFO:teuthology.orchestra.run.smithi157.stdout:2383 write 0xc816800 thru 0xc816814 (0x15 bytes) 2024-09-13T09:12:48.615 INFO:teuthology.orchestra.run.smithi157.stdout:2384 write 0x7aba800 thru 0x7ac7e75 (0xd676 bytes) 2024-09-13T09:12:48.624 INFO:teuthology.orchestra.run.smithi157.stdout:2386 trunc from 0xd8ef000 to 0x5e3b800 2024-09-13T09:12:48.670 INFO:teuthology.orchestra.run.smithi157.stdout:2388 trunc from 0x5e3b800 to 0xa01e800 2024-09-13T09:12:48.675 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:48.675+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:48.684 INFO:teuthology.orchestra.run.smithi157.stdout:2389 write 0x711aa00 thru 0x71204ed (0x5aee bytes) 2024-09-13T09:12:48.686 INFO:teuthology.orchestra.run.smithi157.stdout:2390 trunc from 0xa01e800 to 0xb465000 2024-09-13T09:12:48.722 INFO:teuthology.orchestra.run.smithi157.stdout:2393 write 0xe117400 thru 0xe118d25 (0x1926 bytes) 2024-09-13T09:12:48.729 INFO:teuthology.orchestra.run.smithi157.stdout:2394 read 0x439c200 thru 0x43ab606 (0xf407 bytes) 2024-09-13T09:12:48.730 INFO:teuthology.orchestra.run.smithi157.stdout:2398 read 0x8fe4800 thru 0x8ff2e9f (0xe6a0 bytes) 2024-09-13T09:12:48.730 INFO:teuthology.orchestra.run.smithi157.stdout:2400 write 0x1477c00 thru 0x1484708 (0xcb09 bytes) 2024-09-13T09:12:48.730 INFO:teuthology.orchestra.run.smithi157.stdout:2402 read 0x15de200 thru 0x15e555a (0x735b bytes) 2024-09-13T09:12:48.731 INFO:teuthology.orchestra.run.smithi157.stdout:2403 punch from 0xd98ba00 to 0xd996800, (0xae00 bytes) 2024-09-13T09:12:48.731 INFO:teuthology.orchestra.run.smithi157.stdout:2404 read 0x2061e00 thru 0x206c821 (0xaa22 bytes) 2024-09-13T09:12:48.732 INFO:teuthology.orchestra.run.smithi157.stdout:2405 punch from 0xcdeca00 to 0xcdf3e00, (0x7400 bytes) 2024-09-13T09:12:48.732 INFO:teuthology.orchestra.run.smithi157.stdout:2408 write 0xc69f200 thru 0xc6a25b5 (0x33b6 bytes) 2024-09-13T09:12:48.732 INFO:teuthology.orchestra.run.smithi157.stdout:2409 trunc from 0xe118e00 to 0xe22cc00 2024-09-13T09:12:48.740 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:48.740 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:48.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:48.741 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:48.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.741 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.741 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:12:48.741 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2159618459999999, 2024-09-13T09:12:48.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:12:48.741 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.741 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.741 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.741 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.741 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.741 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:12:48.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.742 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:12:48.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:12:48.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:12:48.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:12:48.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:12:48.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:12:48.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:12:48.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:12:48.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:12:48.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:12:48.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:12:48.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:12:48.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:12:48.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:12:48.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:12:48.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:12:48.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:12:48.745 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.745 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.745 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.746 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.746 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:12:48.746 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.21591279, 2024-09-13T09:12:48.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:12:48.746 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.746 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.746 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.747 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.747 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.747 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:12:48.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.748 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:12:48.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:12:48.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:12:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:12:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:12:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:12:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:12:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:12:48.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:12:48.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:12:48.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:12:48.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:12:48.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:12:48.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:12:48.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:12:48.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:12:48.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:12:48.752 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.752 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.752 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.753 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.753 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:12:48.753 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2158727149999997, 2024-09-13T09:12:48.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:12:48.753 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.753 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.753 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:12:48.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.753 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:12:48.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:12:48.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:12:48.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:12:48.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:12:48.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:12:48.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:12:48.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:12:48.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:12:48.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:12:48.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:12:48.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:12:48.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:12:48.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:12:48.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:12:48.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:12:48.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:12:48.757 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.757 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.757 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.757 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.757 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:12:48.758 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2158429000000002, 2024-09-13T09:12:48.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:12:48.758 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.758 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.758 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:12:48.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.758 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:12:48.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:12:48.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:12:48.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:12:48.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:12:48.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:12:48.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:12:48.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:12:48.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:12:48.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:12:48.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:12:48.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:12:48.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:12:48.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:12:48.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:12:48.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:12:48.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:12:48.762 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.763 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.763 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.763 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.763 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:12:48.763 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.215813239, 2024-09-13T09:12:48.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:12:48.763 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.764 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.764 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.764 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.764 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.764 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:12:48.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.764 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:12:48.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:12:48.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:12:48.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:12:48.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:12:48.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:12:48.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:12:48.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:12:48.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:12:48.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:12:48.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:12:48.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:12:48.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:12:48.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:12:48.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:12:48.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:12:48.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:12:48.768 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.768 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.768 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.768 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.768 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:12:48.768 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2157840320000002, 2024-09-13T09:12:48.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:12:48.768 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.768 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.769 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.769 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.769 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.769 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:12:48.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.769 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:12:48.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:12:48.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:12:48.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:12:48.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:12:48.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:12:48.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:12:48.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:12:48.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:12:48.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:12:48.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:12:48.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:12:48.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:12:48.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:12:48.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:12:48.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:12:48.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:12:48.776 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.776 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.776 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.776 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.776 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:12:48.776 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2157278509999996, 2024-09-13T09:12:48.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:12:48.776 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.776 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.776 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.776 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.776 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.777 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:12:48.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.777 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:12:48.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:12:48.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:12:48.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:12:48.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:12:48.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:12:48.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:12:48.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:12:48.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:12:48.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:12:48.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:12:48.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:12:48.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:12:48.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:12:48.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:12:48.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:12:48.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:12:48.780 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.780 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.780 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.781 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.781 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:12:48.781 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2156743900000002, 2024-09-13T09:12:48.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:12:48.781 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.781 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.781 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.781 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.781 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.781 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:12:48.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.781 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:12:48.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:12:48.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:12:48.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:12:48.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:12:48.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:12:48.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:12:48.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:12:48.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:12:48.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:12:48.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:12:48.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:12:48.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:12:48.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:12:48.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:12:48.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:12:48.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:12:48.785 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.785 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.785 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.785 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.785 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:12:48.786 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2156175600000001, 2024-09-13T09:12:48.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:12:48.786 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.786 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.786 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.786 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.786 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.786 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:12:48.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.786 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:12:48.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:12:48.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:12:48.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:12:48.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:12:48.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:12:48.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:12:48.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:12:48.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:12:48.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:12:48.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:12:48.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:12:48.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:12:48.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:12:48.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:12:48.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:12:48.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:12:48.790 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.790 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.790 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.790 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.790 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:12:48.790 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2155665329999996, 2024-09-13T09:12:48.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:12:48.790 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.791 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.791 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.791 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.791 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.791 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:12:48.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.791 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:12:48.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:12:48.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:12:48.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:12:48.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:12:48.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:12:48.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:12:48.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:12:48.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:12:48.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:12:48.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:12:48.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:12:48.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:12:48.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:12:48.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:12:48.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:12:48.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:12:48.795 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.795 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.795 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.795 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.795 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:12:48.796 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2155253430000004, 2024-09-13T09:12:48.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:12:48.796 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.796 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.796 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.796 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.796 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.796 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:12:48.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.796 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:12:48.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:12:48.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:12:48.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:12:48.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:12:48.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:12:48.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:12:48.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:12:48.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:12:48.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:12:48.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:12:48.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:12:48.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:12:48.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:12:48.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:12:48.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:12:48.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:12:48.800 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.800 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.800 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.800 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.800 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:12:48.800 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.215483227, 2024-09-13T09:12:48.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:12:48.801 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.801 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.801 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.801 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.801 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.801 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:12:48.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.801 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:12:48.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:12:48.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:12:48.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:48.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:12:48.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:12:48.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:12:48.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:12:48.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:12:48.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:12:48.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:12:48.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:12:48.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:12:48.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:12:48.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:12:48.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:12:48.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:12:48.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:12:48.806 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.806 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.806 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.806 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.806 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:12:48.806 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2154393480000003, 2024-09-13T09:12:48.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:12:48.806 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.806 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.806 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.807 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.807 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.807 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:12:48.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.807 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:12:48.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:12:48.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:12:48.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:48.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:12:48.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:12:48.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:12:48.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:12:48.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:12:48.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:12:48.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:12:48.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:12:48.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:12:48.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:12:48.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:12:48.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:12:48.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:12:48.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:12:48.811 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.811 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.811 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.811 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.811 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:12:48.811 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2153934309999999, 2024-09-13T09:12:48.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:12:48.811 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.811 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.811 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.812 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.812 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.812 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:12:48.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.812 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:12:48.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:12:48.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:12:48.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:12:48.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:12:48.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:12:48.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:12:48.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:12:48.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:12:48.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:12:48.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:12:48.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:12:48.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:12:48.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:12:48.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:12:48.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:12:48.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:12:48.815 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.815 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.816 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.816 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.816 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:12:48.816 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2153436600000003, 2024-09-13T09:12:48.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:12:48.816 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.816 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.816 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:12:48.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.817 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:12:48.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:12:48.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:12:48.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:48.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:12:48.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:12:48.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:12:48.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:12:48.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:12:48.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:12:48.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:12:48.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:12:48.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:12:48.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:12:48.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:12:48.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:12:48.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:12:48.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:12:48.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.820 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.820 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.820 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:12:48.821 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2152982559999996, 2024-09-13T09:12:48.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:12:48.821 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.821 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.821 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:12:48.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.821 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:12:48.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:12:48.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:12:48.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:48.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:12:48.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:12:48.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:12:48.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:12:48.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:12:48.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:12:48.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:12:48.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:12:48.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:12:48.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:12:48.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:12:48.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:12:48.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:12:48.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:12:48.825 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.825 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.825 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.825 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.825 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:12:48.825 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2152476180000003, 2024-09-13T09:12:48.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:12:48.825 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.826 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.826 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.826 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.826 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.826 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:12:48.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.826 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:12:48.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:12:48.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:12:48.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:12:48.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:12:48.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:12:48.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:12:48.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:12:48.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:12:48.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:12:48.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:12:48.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:12:48.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:12:48.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:12:48.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:12:48.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:12:48.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:12:48.830 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.830 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.830 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.830 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.830 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:12:48.830 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2151917750000001, 2024-09-13T09:12:48.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:12:48.830 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.830 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.831 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.831 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:12:48.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.831 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:12:48.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:12:48.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:12:48.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:12:48.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:12:48.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:12:48.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:12:48.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:12:48.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:12:48.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:12:48.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:12:48.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:12:48.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:12:48.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:12:48.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:12:48.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:12:48.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:12:48.834 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.834 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.834 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:12:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2151348530000003, 2024-09-13T09:12:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:12:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:12:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:12:48.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:12:48.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:12:48.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:48.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:12:48.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:12:48.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:12:48.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:12:48.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:12:48.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:12:48.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:12:48.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:12:48.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:12:48.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:12:48.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:12:48.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:12:48.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:12:48.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:12:48.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.839 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.839 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:48.839 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:12:48.840 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.2150812899999996, 2024-09-13T09:12:48.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:12:48.840 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:48.840 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:48.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:48.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:48.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:48.840 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:12:48.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.840 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:48.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:48.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:12:48.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:48.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:12:48.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:48.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:48.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:12:48.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:48.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:48.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:12:48.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:12:48.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:48.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:12:48.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:12:48.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:48.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:12:48.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:12:48.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:12:48.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:12:48.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:48.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:12:48.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:12:48.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:48.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:12:48.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:12:48.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:48.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:48.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:48.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:12:48.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:12:48.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.844 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:48.844 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:48.844 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:48.846 INFO:teuthology.orchestra.run.smithi157.stdout:2410 write 0x10cd200 thru 0x10d88fd (0xb6fe bytes) 2024-09-13T09:12:48.848 INFO:teuthology.orchestra.run.smithi157.stdout:2418 trunc from 0xe22cc00 to 0xc96000 2024-09-13T09:12:48.848 INFO:teuthology.orchestra.run.smithi157.stdout:2424 write 0xb268000 thru 0xb26fd04 (0x7d05 bytes) 2024-09-13T09:12:48.848 INFO:teuthology.orchestra.run.smithi157.stdout:2425 trunc from 0xb26fe00 to 0x2c2f800 2024-09-13T09:12:48.848 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:48.849 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:48.844+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:48.851 INFO:teuthology.orchestra.run.smithi157.stdout:2426 punch from 0xde9a00 to 0xdf5a00, (0xc000 bytes) 2024-09-13T09:12:48.852 INFO:teuthology.orchestra.run.smithi157.stdout:2428 punch from 0x24f3200 to 0x24f6600, (0x3400 bytes) 2024-09-13T09:12:48.874 INFO:teuthology.orchestra.run.smithi157.stdout:2432 write 0xe52ec00 thru 0xe53cdd6 (0xe1d7 bytes) 2024-09-13T09:12:48.875 INFO:teuthology.orchestra.run.smithi157.stdout:2434 punch from 0x4fd2800 to 0x4fd8400, (0x5c00 bytes) 2024-09-13T09:12:48.894 INFO:teuthology.orchestra.run.smithi157.stdout:2435 write 0xed2dc00 thru 0xed3a481 (0xc882 bytes) 2024-09-13T09:12:48.894 INFO:teuthology.orchestra.run.smithi157.stdout:2438 read 0x3e19e00 thru 0x3e27d4b (0xdf4c bytes) 2024-09-13T09:12:48.895 INFO:teuthology.orchestra.run.smithi157.stdout:2440 write 0x9fb7400 thru 0x9fc46bd (0xd2be bytes) 2024-09-13T09:12:48.903 INFO:teuthology.orchestra.run.smithi157.stdout:2444 trunc from 0xed3a600 to 0x7aa2200 2024-09-13T09:12:48.933 INFO:teuthology.orchestra.run.smithi157.stdout:2446 clone 15 order 23 su 2097152 sc 11 2024-09-13T09:12:48.946 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:48.945+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:48.955 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:12:48.965 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:48.965 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:48.965 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:48.965 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:48.974 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:49.048 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:49.046+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:49.076 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:49.076 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:49.077 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:49.077 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:49.077 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:49.077 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:49.085 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:49.093 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:49.093 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:49.109 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:49.109 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:49.127 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:49.127 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:49.143 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:49.144 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:49.149 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:49.148+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:49.161 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:49.161 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:49.189 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:49.189 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:49.198 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:49.262 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:12:49.270 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T09:12:49.285 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:49.284+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:49.286 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T09:12:49.348 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:49.348 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:49.350 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:49.350+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:49.357 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:49.366 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone14 from 0x7aa2200 (overlap 0x954a00) to 0x0 2024-09-13T09:12:49.452 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:49.451+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:49.503 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:49.503 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:49.513 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:49.556 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:49.556+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:49.618 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:49.618 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:49.628 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:49.669 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:49.668+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:49.737 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:49.737 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:49.746 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:49.754 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:49.754+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:49.763 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-13T09:12:49.834 INFO:teuthology.orchestra.run.smithi157.stdout:2448 trunc from 0x7aa2200 to 0x605a00 2024-09-13T09:12:49.856 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:49.854+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:49.862 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:49.862 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:49.872 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:49.896 INFO:teuthology.orchestra.run.smithi157.stdout:2449 write 0xe5f5200 thru 0xe5f6d64 (0x1b65 bytes) 2024-09-13T09:12:49.896 INFO:teuthology.orchestra.run.smithi157.stdout:2450 clone 16 order 20 su 16384 sc 2 2024-09-13T09:12:49.956 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:49.955+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:49.979 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:49.979 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:49.987 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:50.057 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:50.056+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:50.134 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:50.135 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:50.144 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:50.157 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:50.157+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:50.251 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:50.251 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:50.258 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:50.258+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:50.259 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:50.286 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T09:12:50.286 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-09-13T09:12:50.302 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T09:12:50.375 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:50.375 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:50.375 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:50.374+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:50.384 INFO:teuthology.orchestra.run.smithi157.stdout:leaving image image_client.0-clone15 intact 2024-09-13T09:12:50.385 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:50.460 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:50.459+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:50.533 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:50.534 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:50.543 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:50.561 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:50.560+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:50.650 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:50.650 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:50.659 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:50.662 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:50.661+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:50.762 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:50.762+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:50.768 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:50.768 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:50.776 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:50.850 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-13T09:12:50.863 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:50.862+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:50.887 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:50.887 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:50.896 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:50.916 INFO:teuthology.orchestra.run.smithi157.stdout:2451 trunc from 0xe5f6e00 to 0x8de8400 2024-09-13T09:12:50.934 INFO:teuthology.orchestra.run.smithi157.stdout:2452 read 0x35f9800 thru 0x36076f0 (0xdef1 bytes) 2024-09-13T09:12:50.935 INFO:teuthology.orchestra.run.smithi157.stdout:2453 write 0x8a5aa00 thru 0x8a61402 (0x6a03 bytes) 2024-09-13T09:12:50.935 INFO:teuthology.orchestra.run.smithi157.stdout:2455 write 0x85b4a00 thru 0x85ba3cb (0x59cc bytes) 2024-09-13T09:12:50.935 INFO:teuthology.orchestra.run.smithi157.stdout:2456 write 0x7ca8e00 thru 0x7cb5855 (0xca56 bytes) 2024-09-13T09:12:50.936 INFO:teuthology.orchestra.run.smithi157.stdout:2457 trunc from 0x8de8400 to 0x9fbc200 2024-09-13T09:12:50.957 INFO:teuthology.orchestra.run.smithi157.stdout:2459 punch from 0x1413800 to 0x141a600, (0x6e00 bytes) 2024-09-13T09:12:50.963 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:50.962+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:50.971 INFO:teuthology.orchestra.run.smithi157.stdout:2460 write 0xcec2200 thru 0xcecf511 (0xd312 bytes) 2024-09-13T09:12:50.971 INFO:teuthology.orchestra.run.smithi157.stdout:2462 read 0x9c5f800 thru 0x9c6ae81 (0xb682 bytes) 2024-09-13T09:12:50.983 INFO:teuthology.orchestra.run.smithi157.stdout:2463 trunc from 0xcecf600 to 0x3a4b400 2024-09-13T09:12:51.007 INFO:teuthology.orchestra.run.smithi157.stdout:2465 read 0x2780000 thru 0x278567c (0x567d bytes) 2024-09-13T09:12:51.008 INFO:teuthology.orchestra.run.smithi157.stdout:2466 read 0x1436800 thru 0x143c203 (0x5a04 bytes) 2024-09-13T09:12:51.014 INFO:teuthology.orchestra.run.smithi157.stdout:2467 trunc from 0x3a4b400 to 0x7f4fa00 2024-09-13T09:12:51.023 INFO:teuthology.orchestra.run.smithi157.stdout:2468 read 0x48b5e00 thru 0x48bbcce (0x5ecf bytes) 2024-09-13T09:12:51.029 INFO:teuthology.orchestra.run.smithi157.stdout:2474 read 0x57dc400 thru 0x57dcae1 (0x6e2 bytes) 2024-09-13T09:12:51.041 INFO:teuthology.orchestra.run.smithi157.stdout:2475 write 0x8236200 thru 0x82392f9 (0x30fa bytes) 2024-09-13T09:12:51.041 INFO:teuthology.orchestra.run.smithi157.stdout:2476 punch from 0x6339200 to 0x6339a00, (0x800 bytes) 2024-09-13T09:12:51.043 INFO:teuthology.orchestra.run.smithi157.stdout:2479 write 0x63eec00 thru 0x63f9452 (0xa853 bytes) 2024-09-13T09:12:51.062 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:51.062 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:51.063 INFO:teuthology.orchestra.run.smithi157.stdout:2480 write 0xd6a6800 thru 0xd6b24c1 (0xbcc2 bytes) 2024-09-13T09:12:51.064 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:51.063+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:51.071 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:51.073 INFO:teuthology.orchestra.run.smithi157.stdout:2481 trunc from 0xd6b2600 to 0x9a7f200 2024-09-13T09:12:51.094 INFO:teuthology.orchestra.run.smithi157.stdout:2483 punch from 0x8557e00 to 0x8559600, (0x1800 bytes) 2024-09-13T09:12:51.100 INFO:teuthology.orchestra.run.smithi157.stdout:2486 write 0x8a18800 thru 0x8a1e7a6 (0x5fa7 bytes) 2024-09-13T09:12:51.101 INFO:teuthology.orchestra.run.smithi157.stdout:2489 punch from 0x5bba200 to 0x5bc1e00, (0x7c00 bytes) 2024-09-13T09:12:51.117 INFO:teuthology.orchestra.run.smithi157.stdout:2490 write 0xe9b6800 thru 0xe9bc79f (0x5fa0 bytes) 2024-09-13T09:12:51.123 INFO:teuthology.orchestra.run.smithi157.stdout:2491 read 0x780cc00 thru 0x7814182 (0x7583 bytes) 2024-09-13T09:12:51.123 INFO:teuthology.orchestra.run.smithi157.stdout:2492 write 0x8342600 thru 0x8344dd6 (0x27d7 bytes) 2024-09-13T09:12:51.124 INFO:teuthology.orchestra.run.smithi157.stdout:2493 punch from 0x4ad7200 to 0x4adda00, (0x6800 bytes) 2024-09-13T09:12:51.124 INFO:teuthology.orchestra.run.smithi157.stdout:2495 read 0x2083c00 thru 0x2086737 (0x2b38 bytes) 2024-09-13T09:12:51.130 INFO:teuthology.orchestra.run.smithi157.stdout:2497 trunc from 0xe9bc800 to 0x9ac4800 2024-09-13T09:12:51.145 INFO:teuthology.orchestra.run.smithi157.stdout:2498 write 0x4111200 thru 0x411df66 (0xcd67 bytes) 2024-09-13T09:12:51.151 INFO:teuthology.orchestra.run.smithi157.stdout:2500 trunc from 0x9ac4800 to 0xe70fc00 2024-09-13T09:12:51.163 INFO:teuthology.orchestra.run.smithi157.stdout:2502 read 0xca13000 thru 0xca1edb0 (0xbdb1 bytes) 2024-09-13T09:12:51.164 INFO:teuthology.orchestra.run.smithi157.stdout:2503 write 0x4a91c00 thru 0x4a961b1 (0x45b2 bytes) 2024-09-13T09:12:51.164 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:51.163+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:51.170 INFO:teuthology.orchestra.run.smithi157.stdout:2504 punch from 0x3dd3400 to 0x3dd9800, (0x6400 bytes) 2024-09-13T09:12:51.170 INFO:teuthology.orchestra.run.smithi157.stdout:2505 read 0xb62e800 thru 0xb63272d (0x3f2e bytes) 2024-09-13T09:12:51.170 INFO:teuthology.orchestra.run.smithi157.stdout:2510 write 0x6007c00 thru 0x6007de4 (0x1e5 bytes) 2024-09-13T09:12:51.170 INFO:teuthology.orchestra.run.smithi157.stdout:2511 write 0x3979800 thru 0x3986278 (0xca79 bytes) 2024-09-13T09:12:51.170 INFO:teuthology.orchestra.run.smithi157.stdout:2514 punch from 0x8d4ac00 to 0x8d4e200, (0x3600 bytes) 2024-09-13T09:12:51.171 INFO:teuthology.orchestra.run.smithi157.stdout:2515 read 0x9072000 thru 0x9080031 (0xe032 bytes) 2024-09-13T09:12:51.171 INFO:teuthology.orchestra.run.smithi157.stdout:2516 read 0x2a15200 thru 0x2a1701b (0x1e1c bytes) 2024-09-13T09:12:51.171 INFO:teuthology.orchestra.run.smithi157.stdout:2517 read 0x1739800 thru 0x174465e (0xae5f bytes) 2024-09-13T09:12:51.171 INFO:teuthology.orchestra.run.smithi157.stdout:2518 write 0x1b0e000 thru 0x1b0fd92 (0x1d93 bytes) 2024-09-13T09:12:51.172 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:12:51.173 INFO:teuthology.orchestra.run.smithi157.stdout:2519 write 0x792c600 thru 0x7933d36 (0x7737 bytes) 2024-09-13T09:12:51.173 INFO:teuthology.orchestra.run.smithi157.stdout:2523 punch from 0xd289c00 to 0xd290200, (0x6600 bytes) 2024-09-13T09:12:51.174 INFO:teuthology.orchestra.run.smithi157.stdout:2524 read 0x7c6dc00 thru 0x7c70178 (0x2579 bytes) 2024-09-13T09:12:51.174 INFO:teuthology.orchestra.run.smithi157.stdout:2525 read 0xe16a000 thru 0xe171758 (0x7759 bytes) 2024-09-13T09:12:51.174 INFO:teuthology.orchestra.run.smithi157.stdout:2526 write 0xd22c200 thru 0xd22d1d1 (0xfd2 bytes) 2024-09-13T09:12:51.174 INFO:teuthology.orchestra.run.smithi157.stdout:2528 read 0x7873000 thru 0x7878fd6 (0x5fd7 bytes) 2024-09-13T09:12:51.174 INFO:teuthology.orchestra.run.smithi157.stdout:2529 write 0xe0cdc00 thru 0xe0ce0a0 (0x4a1 bytes) 2024-09-13T09:12:51.174 INFO:teuthology.orchestra.run.smithi157.stdout:2530 write 0xcea8e00 thru 0xceb3d9a (0xaf9b bytes) 2024-09-13T09:12:51.174 INFO:teuthology.orchestra.run.smithi157.stdout:2531 read 0x6e14800 thru 0x6e2422d (0xfa2e bytes) 2024-09-13T09:12:51.174 INFO:teuthology.orchestra.run.smithi157.stdout:2532 read 0xde4600 thru 0xdf1a38 (0xd439 bytes) 2024-09-13T09:12:51.175 INFO:teuthology.orchestra.run.smithi157.stdout:2533 clone 17 order 25 su 524288 sc 8 2024-09-13T09:12:51.228 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:51.229 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:51.237 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:51.282 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:51.281+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:51.285 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T09:12:51.298 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T09:12:51.301 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:51.301 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:51.318 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:51.318 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:51.334 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:51.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:51.346 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:51.346 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:51.352 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:51.352 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:51.356 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:51.372 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:51.372 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:51.390 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:51.390 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:51.397 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:51.396+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:51.402 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone16 from 0xe70fc00 (overlap 0x3a4b400) to 0x1d25a00 2024-09-13T09:12:51.466 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:51.466+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:51.518 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:51.519 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:51.519 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:51.519 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:51.519 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:51.527 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:51.567 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:51.566+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:51.668 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:51.667+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:51.674 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:51.675 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:51.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:51.675 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:51.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.675 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17752 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2125824~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.675 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138424+0000", 2024-09-13T09:12:51.676 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5337813429999994, 2024-09-13T09:12:51.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045520176000000002, 2024-09-13T09:12:51.676 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.676 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.676 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.677 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17752 2024-09-13T09:12:51.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.677 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138424+0000", 2024-09-13T09:12:51.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138424+0000", 2024-09-13T09:12:51.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138423+0000", 2024-09-13T09:12:51.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:51.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138425+0000", 2024-09-13T09:12:51.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0640000000000001e-06 2024-09-13T09:12:51.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138431+0000", 2024-09-13T09:12:51.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0900000000000001e-06 2024-09-13T09:12:51.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138437+0000", 2024-09-13T09:12:51.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0329999999999997e-06 2024-09-13T09:12:51.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183854+0000", 2024-09-13T09:12:51.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045416099000000001 2024-09-13T09:12:51.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183894+0000", 2024-09-13T09:12:51.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0935e-05 2024-09-13T09:12:51.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183944+0000", 2024-09-13T09:12:51.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9329999999999997e-05 2024-09-13T09:12:51.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.681 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.681 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17753 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2148352~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.681 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138442+0000", 2024-09-13T09:12:51.681 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5337626219999994, 2024-09-13T09:12:51.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045633606, 2024-09-13T09:12:51.681 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.681 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.681 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.682 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.682 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.682 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17753 2024-09-13T09:12:51.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.682 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138442+0000", 2024-09-13T09:12:51.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138442+0000", 2024-09-13T09:12:51.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138441+0000", 2024-09-13T09:12:51.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:51.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138444+0000", 2024-09-13T09:12:51.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0120000000000001e-06 2024-09-13T09:12:51.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138450+0000", 2024-09-13T09:12:51.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6180000000000003e-06 2024-09-13T09:12:51.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138458+0000", 2024-09-13T09:12:51.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9130000000000003e-06 2024-09-13T09:12:51.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183973+0000", 2024-09-13T09:12:51.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045514741999999997 2024-09-13T09:12:51.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184022+0000", 2024-09-13T09:12:51.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9521999999999998e-05 2024-09-13T09:12:51.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184076+0000", 2024-09-13T09:12:51.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3888000000000003e-05 2024-09-13T09:12:51.685 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.685 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.685 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.686 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17754 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2170880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.686 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:12:51.686 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5337336799999992, 2024-09-13T09:12:51.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045714312999999999, 2024-09-13T09:12:51.686 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.686 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.686 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17754 2024-09-13T09:12:51.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.686 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:12:51.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:12:51.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:12:51.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:51.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138474+0000", 2024-09-13T09:12:51.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8540000000000001e-06 2024-09-13T09:12:51.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138480+0000", 2024-09-13T09:12:51.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5139999999999998e-06 2024-09-13T09:12:51.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138485+0000", 2024-09-13T09:12:51.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.989e-06 2024-09-13T09:12:51.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184095+0000", 2024-09-13T09:12:51.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045609702000000002 2024-09-13T09:12:51.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184135+0000", 2024-09-13T09:12:51.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0510999999999999e-05 2024-09-13T09:12:51.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184186+0000", 2024-09-13T09:12:51.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0262000000000001e-05 2024-09-13T09:12:51.690 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.690 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.690 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.690 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17755 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2193408~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.690 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:12:51.690 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5337144919999997, 2024-09-13T09:12:51.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045899349999999998, 2024-09-13T09:12:51.691 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.691 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.691 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.691 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.691 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.691 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17755 2024-09-13T09:12:51.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.691 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:12:51.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:12:51.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:12:51.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:51.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138492+0000", 2024-09-13T09:12:51.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4310000000000001e-06 2024-09-13T09:12:51.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138498+0000", 2024-09-13T09:12:51.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7400000000000001e-06 2024-09-13T09:12:51.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138503+0000", 2024-09-13T09:12:51.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:12:51.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184222+0000", 2024-09-13T09:12:51.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045718996999999997 2024-09-13T09:12:51.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184306+0000", 2024-09-13T09:12:51.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4293000000000001e-05 2024-09-13T09:12:51.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184390+0000", 2024-09-13T09:12:51.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3718e-05 2024-09-13T09:12:51.695 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.695 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.695 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.695 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17756 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2215936~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.695 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138507+0000", 2024-09-13T09:12:51.695 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5336979450000001, 2024-09-13T09:12:51.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046004902, 2024-09-13T09:12:51.695 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.696 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.696 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.696 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.696 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.696 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17756 2024-09-13T09:12:51.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.696 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138507+0000", 2024-09-13T09:12:51.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138507+0000", 2024-09-13T09:12:51.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138506+0000", 2024-09-13T09:12:51.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:51.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138509+0000", 2024-09-13T09:12:51.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:12:51.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138514+0000", 2024-09-13T09:12:51.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.665e-06 2024-09-13T09:12:51.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138519+0000", 2024-09-13T09:12:51.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7530000000000001e-06 2024-09-13T09:12:51.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184415+0000", 2024-09-13T09:12:51.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045895575000000001 2024-09-13T09:12:51.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184459+0000", 2024-09-13T09:12:51.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4506999999999999e-05 2024-09-13T09:12:51.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184512+0000", 2024-09-13T09:12:51.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2685999999999997e-05 2024-09-13T09:12:51.699 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.699 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.700 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.700 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17766 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2441216~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.700 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140631+0000", 2024-09-13T09:12:51.700 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5315743869999991, 2024-09-13T09:12:51.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045443425000000003, 2024-09-13T09:12:51.700 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.700 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.700 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.700 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.700 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.700 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17766 2024-09-13T09:12:51.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.701 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140631+0000", 2024-09-13T09:12:51.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140631+0000", 2024-09-13T09:12:51.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140630+0000", 2024-09-13T09:12:51.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:51.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140632+0000", 2024-09-13T09:12:51.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1770000000000001e-06 2024-09-13T09:12:51.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140638+0000", 2024-09-13T09:12:51.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5550000000000002e-06 2024-09-13T09:12:51.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140643+0000", 2024-09-13T09:12:51.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5589999999999999e-06 2024-09-13T09:12:51.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.185929+0000", 2024-09-13T09:12:51.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045286356999999999 2024-09-13T09:12:51.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.185985+0000", 2024-09-13T09:12:51.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6042999999999999e-05 2024-09-13T09:12:51.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186074+0000", 2024-09-13T09:12:51.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9083000000000001e-05 2024-09-13T09:12:51.704 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.704 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.704 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.704 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17767 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2463744~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.704 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:12:51.704 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5315585939999998, 2024-09-13T09:12:51.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045589383999999997, 2024-09-13T09:12:51.705 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.705 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.705 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.705 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.705 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.705 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17767 2024-09-13T09:12:51.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.705 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:12:51.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:12:51.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:12:51.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:51.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140648+0000", 2024-09-13T09:12:51.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.001e-06 2024-09-13T09:12:51.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140654+0000", 2024-09-13T09:12:51.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6690000000000003e-06 2024-09-13T09:12:51.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140658+0000", 2024-09-13T09:12:51.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4220000000000002e-06 2024-09-13T09:12:51.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186115+0000", 2024-09-13T09:12:51.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045456435000000003 2024-09-13T09:12:51.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186172+0000", 2024-09-13T09:12:51.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7088e-05 2024-09-13T09:12:51.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186236+0000", 2024-09-13T09:12:51.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4082000000000001e-05 2024-09-13T09:12:51.709 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.709 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.709 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.709 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17768 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2486272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.709 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:12:51.709 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5315426270000003, 2024-09-13T09:12:51.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045865601999999998, 2024-09-13T09:12:51.710 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.710 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.710 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.710 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.710 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.710 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17768 2024-09-13T09:12:51.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.710 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:12:51.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:12:51.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:12:51.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:51.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140664+0000", 2024-09-13T09:12:51.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1449999999999998e-06 2024-09-13T09:12:51.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140670+0000", 2024-09-13T09:12:51.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9460000000000003e-06 2024-09-13T09:12:51.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140675+0000", 2024-09-13T09:12:51.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5770000000000004e-06 2024-09-13T09:12:51.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186267+0000", 2024-09-13T09:12:51.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045591957000000002 2024-09-13T09:12:51.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186322+0000", 2024-09-13T09:12:51.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5049000000000001e-05 2024-09-13T09:12:51.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186528+0000", 2024-09-13T09:12:51.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020626600000000001 2024-09-13T09:12:51.713 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.713 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.714 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.714 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17769 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2508800~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.714 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:12:51.714 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5315265660000001, 2024-09-13T09:12:51.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045981877999999997, 2024-09-13T09:12:51.714 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.714 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.714 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.714 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.714 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.714 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17769 2024-09-13T09:12:51.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.715 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:12:51.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:12:51.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:12:51.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:51.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140680+0000", 2024-09-13T09:12:51.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0209999999999998e-06 2024-09-13T09:12:51.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140686+0000", 2024-09-13T09:12:51.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2840000000000003e-06 2024-09-13T09:12:51.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140691+0000", 2024-09-13T09:12:51.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.369e-06 2024-09-13T09:12:51.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186562+0000", 2024-09-13T09:12:51.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045870834999999999 2024-09-13T09:12:51.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186606+0000", 2024-09-13T09:12:51.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4631e-05 2024-09-13T09:12:51.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186660+0000", 2024-09-13T09:12:51.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4042000000000002e-05 2024-09-13T09:12:51.718 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.718 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.718 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.718 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17885 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2217984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.718 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:12:51.718 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5295534669999995, 2024-09-13T09:12:51.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045427829000000003, 2024-09-13T09:12:51.719 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.719 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.719 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.719 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.719 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.719 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17885 2024-09-13T09:12:51.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.719 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:12:51.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:12:51.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:12:51.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:51.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142653+0000", 2024-09-13T09:12:51.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2500000000000001e-06 2024-09-13T09:12:51.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142659+0000", 2024-09-13T09:12:51.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.366e-06 2024-09-13T09:12:51.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142663+0000", 2024-09-13T09:12:51.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5190000000000003e-06 2024-09-13T09:12:51.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187980+0000", 2024-09-13T09:12:51.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045316233999999997 2024-09-13T09:12:51.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188032+0000", 2024-09-13T09:12:51.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1981000000000003e-05 2024-09-13T09:12:51.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188079+0000", 2024-09-13T09:12:51.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7803000000000003e-05 2024-09-13T09:12:51.723 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.723 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.723 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.723 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17886 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2240512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.723 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142686+0000", 2024-09-13T09:12:51.723 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5295190590000001, 2024-09-13T09:12:51.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045507535000000002, 2024-09-13T09:12:51.723 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.723 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.723 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.723 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.724 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.724 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17886 2024-09-13T09:12:51.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.724 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142686+0000", 2024-09-13T09:12:51.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142686+0000", 2024-09-13T09:12:51.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142685+0000", 2024-09-13T09:12:51.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:51.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142688+0000", 2024-09-13T09:12:51.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.83e-06 2024-09-13T09:12:51.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142694+0000", 2024-09-13T09:12:51.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2910000000000004e-06 2024-09-13T09:12:51.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142699+0000", 2024-09-13T09:12:51.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9599999999999999e-06 2024-09-13T09:12:51.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188102+0000", 2024-09-13T09:12:51.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045402459999999999 2024-09-13T09:12:51.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188143+0000", 2024-09-13T09:12:51.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1607000000000003e-05 2024-09-13T09:12:51.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188193+0000", 2024-09-13T09:12:51.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9947000000000001e-05 2024-09-13T09:12:51.727 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.727 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.727 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.728 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17887 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2263040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.728 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142704+0000", 2024-09-13T09:12:51.728 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5295010680000001, 2024-09-13T09:12:51.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045652102999999999, 2024-09-13T09:12:51.728 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.728 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.728 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.728 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.728 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.728 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17887 2024-09-13T09:12:51.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.728 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142704+0000", 2024-09-13T09:12:51.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142704+0000", 2024-09-13T09:12:51.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142703+0000", 2024-09-13T09:12:51.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:51.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142706+0000", 2024-09-13T09:12:51.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.351e-06 2024-09-13T09:12:51.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142712+0000", 2024-09-13T09:12:51.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8799999999999996e-06 2024-09-13T09:12:51.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142716+0000", 2024-09-13T09:12:51.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-13T09:12:51.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188230+0000", 2024-09-13T09:12:51.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045513801 2024-09-13T09:12:51.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188303+0000", 2024-09-13T09:12:51.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3032000000000001e-05 2024-09-13T09:12:51.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188356+0000", 2024-09-13T09:12:51.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2899000000000003e-05 2024-09-13T09:12:51.732 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.732 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.732 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.732 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17893 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2398208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.732 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:12:51.732 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5284495329999999, 2024-09-13T09:12:51.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045574960999999997, 2024-09-13T09:12:51.732 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.732 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.732 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.733 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.733 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.733 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17893 2024-09-13T09:12:51.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.733 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:12:51.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:12:51.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:12:51.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:51.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143757+0000", 2024-09-13T09:12:51.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2000000000000001e-06 2024-09-13T09:12:51.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143763+0000", 2024-09-13T09:12:51.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.553e-06 2024-09-13T09:12:51.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143768+0000", 2024-09-13T09:12:51.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0019999999999997e-06 2024-09-13T09:12:51.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189160+0000", 2024-09-13T09:12:51.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045392147000000001 2024-09-13T09:12:51.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189243+0000", 2024-09-13T09:12:51.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2594000000000002e-05 2024-09-13T09:12:51.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189330+0000", 2024-09-13T09:12:51.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7764999999999997e-05 2024-09-13T09:12:51.736 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.736 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.736 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.736 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17894 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2420736~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.737 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.143886+0000", 2024-09-13T09:12:51.737 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5283190500000003, 2024-09-13T09:12:51.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045562601000000001, 2024-09-13T09:12:51.737 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.737 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.737 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.737 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.737 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.737 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17894 2024-09-13T09:12:51.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.737 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143886+0000", 2024-09-13T09:12:51.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143886+0000", 2024-09-13T09:12:51.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143771+0000", 2024-09-13T09:12:51.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998856 2024-09-13T09:12:51.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143889+0000", 2024-09-13T09:12:51.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000117314 2024-09-13T09:12:51.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143896+0000", 2024-09-13T09:12:51.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5920000000000003e-06 2024-09-13T09:12:51.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143902+0000", 2024-09-13T09:12:51.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:12:51.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189359+0000", 2024-09-13T09:12:51.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045457414000000002 2024-09-13T09:12:51.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189399+0000", 2024-09-13T09:12:51.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9653999999999997e-05 2024-09-13T09:12:51.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189449+0000", 2024-09-13T09:12:51.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9456999999999998e-05 2024-09-13T09:12:51.741 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.741 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.741 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.741 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17895 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2443264~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.741 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.143990+0000", 2024-09-13T09:12:51.741 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5282150750000003, 2024-09-13T09:12:51.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045666642, 2024-09-13T09:12:51.741 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.741 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.741 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.742 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.742 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.742 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17895 2024-09-13T09:12:51.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.742 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143990+0000", 2024-09-13T09:12:51.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143990+0000", 2024-09-13T09:12:51.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143907+0000", 2024-09-13T09:12:51.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999166 2024-09-13T09:12:51.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143993+0000", 2024-09-13T09:12:51.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6049000000000002e-05 2024-09-13T09:12:51.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144000+0000", 2024-09-13T09:12:51.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6550000000000004e-06 2024-09-13T09:12:51.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144006+0000", 2024-09-13T09:12:51.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4450000000000004e-06 2024-09-13T09:12:51.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189483+0000", 2024-09-13T09:12:51.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045477228000000001 2024-09-13T09:12:51.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189567+0000", 2024-09-13T09:12:51.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3721999999999996e-05 2024-09-13T09:12:51.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189657+0000", 2024-09-13T09:12:51.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9764000000000003e-05 2024-09-13T09:12:51.745 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.745 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.745 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.745 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17896 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2465792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.746 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144260+0000", 2024-09-13T09:12:51.746 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5279452080000002, 2024-09-13T09:12:51.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045647222000000001, 2024-09-13T09:12:51.746 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.746 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.746 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.746 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.746 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.746 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17896 2024-09-13T09:12:51.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.746 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144260+0000", 2024-09-13T09:12:51.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144260+0000", 2024-09-13T09:12:51.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144010+0000", 2024-09-13T09:12:51.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9997506 2024-09-13T09:12:51.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144263+0000", 2024-09-13T09:12:51.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025252299999999999 2024-09-13T09:12:51.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144270+0000", 2024-09-13T09:12:51.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5259999999999996e-06 2024-09-13T09:12:51.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144276+0000", 2024-09-13T09:12:51.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:12:51.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189682+0000", 2024-09-13T09:12:51.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045406482999999997 2024-09-13T09:12:51.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189723+0000", 2024-09-13T09:12:51.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1059000000000001e-05 2024-09-13T09:12:51.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189907+0000", 2024-09-13T09:12:51.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018378099999999999 2024-09-13T09:12:51.750 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.750 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.750 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.750 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17897 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2488320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.750 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144430+0000", 2024-09-13T09:12:51.750 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5277750660000002, 2024-09-13T09:12:51.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045662734000000003, 2024-09-13T09:12:51.750 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.750 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.750 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.751 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.751 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.751 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17897 2024-09-13T09:12:51.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.751 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144430+0000", 2024-09-13T09:12:51.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144430+0000", 2024-09-13T09:12:51.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144281+0000", 2024-09-13T09:12:51.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998507 2024-09-13T09:12:51.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144433+0000", 2024-09-13T09:12:51.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00015261800000000001 2024-09-13T09:12:51.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144441+0000", 2024-09-13T09:12:51.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6790000000000005e-06 2024-09-13T09:12:51.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144446+0000", 2024-09-13T09:12:51.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5280000000000003e-06 2024-09-13T09:12:51.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189943+0000", 2024-09-13T09:12:51.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045496178999999998 2024-09-13T09:12:51.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190025+0000", 2024-09-13T09:12:51.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2325999999999997e-05 2024-09-13T09:12:51.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190093+0000", 2024-09-13T09:12:51.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7780000000000005e-05 2024-09-13T09:12:51.754 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.754 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.754 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.755 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17898 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2510848~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.755 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144655+0000", 2024-09-13T09:12:51.755 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5275498909999996, 2024-09-13T09:12:51.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045552875, 2024-09-13T09:12:51.755 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.755 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.755 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.755 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.755 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.755 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17898 2024-09-13T09:12:51.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.755 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144655+0000", 2024-09-13T09:12:51.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144655+0000", 2024-09-13T09:12:51.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144451+0000", 2024-09-13T09:12:51.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9997959 2024-09-13T09:12:51.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144658+0000", 2024-09-13T09:12:51.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000207188 2024-09-13T09:12:51.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144666+0000", 2024-09-13T09:12:51.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1380000000000007e-06 2024-09-13T09:12:51.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144672+0000", 2024-09-13T09:12:51.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7980000000000002e-06 2024-09-13T09:12:51.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190116+0000", 2024-09-13T09:12:51.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045443413000000002 2024-09-13T09:12:51.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190158+0000", 2024-09-13T09:12:51.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2055000000000003e-05 2024-09-13T09:12:51.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190208+0000", 2024-09-13T09:12:51.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0387999999999999e-05 2024-09-13T09:12:51.759 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.759 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.759 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.759 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17899 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2533376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.759 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144737+0000", 2024-09-13T09:12:51.759 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5274677479999994, 2024-09-13T09:12:51.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045677290000000002, 2024-09-13T09:12:51.759 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.759 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.760 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.760 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.760 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.760 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17899 2024-09-13T09:12:51.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.760 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144737+0000", 2024-09-13T09:12:51.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144737+0000", 2024-09-13T09:12:51.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144678+0000", 2024-09-13T09:12:51.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999409 2024-09-13T09:12:51.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144740+0000", 2024-09-13T09:12:51.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2116e-05 2024-09-13T09:12:51.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144748+0000", 2024-09-13T09:12:51.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2130000000000008e-06 2024-09-13T09:12:51.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144754+0000", 2024-09-13T09:12:51.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.66e-06 2024-09-13T09:12:51.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190238+0000", 2024-09-13T09:12:51.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045483824999999999 2024-09-13T09:12:51.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.763 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190322+0000", 2024-09-13T09:12:51.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4438999999999996e-05 2024-09-13T09:12:51.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.763 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190415+0000", 2024-09-13T09:12:51.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2227999999999999e-05 2024-09-13T09:12:51.763 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.763 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.763 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.764 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17900 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2555904~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:51.764 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144938+0000", 2024-09-13T09:12:51.764 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5272668469999999, 2024-09-13T09:12:51.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045612509000000002, 2024-09-13T09:12:51.764 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:51.764 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:51.764 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:51.764 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:51.764 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:51.764 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17900 2024-09-13T09:12:51.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.765 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:51.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:51.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144938+0000", 2024-09-13T09:12:51.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:51.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144938+0000", 2024-09-13T09:12:51.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:51.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:51.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144758+0000", 2024-09-13T09:12:51.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998202 2024-09-13T09:12:51.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:51.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144941+0000", 2024-09-13T09:12:51.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000182688 2024-09-13T09:12:51.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:51.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144949+0000", 2024-09-13T09:12:51.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7300000000000005e-06 2024-09-13T09:12:51.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:51.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144954+0000", 2024-09-13T09:12:51.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4090000000000002e-06 2024-09-13T09:12:51.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:51.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190448+0000", 2024-09-13T09:12:51.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045493901000000003 2024-09-13T09:12:51.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:51.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190499+0000", 2024-09-13T09:12:51.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0677999999999998e-05 2024-09-13T09:12:51.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:51.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:51.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:51.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190551+0000", 2024-09-13T09:12:51.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1820999999999997e-05 2024-09-13T09:12:51.768 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.768 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.768 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.768 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:51.768 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:51.768 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:51.770 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:51.769+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:51.771 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:51.870 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:51.869+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:51.875 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:51.875 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:51.884 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:51.941 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-13T09:12:51.971 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:51.970+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:51.991 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:51.991 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:52.000 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:52.071 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:52.070+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:52.154 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:52.154 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16127 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 7094272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.022986+0000", 2024-09-13T09:12:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.129519482999999, 2024-09-13T09:12:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045533075999999999, 2024-09-13T09:12:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16127 2024-09-13T09:12:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.022986+0000", 2024-09-13T09:12:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.022986+0000", 2024-09-13T09:12:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.022986+0000", 2024-09-13T09:12:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.022989+0000", 2024-09-13T09:12:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8260000000000001e-06 2024-09-13T09:12:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.022994+0000", 2024-09-13T09:12:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.78e-06 2024-09-13T09:12:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.022996+0000", 2024-09-13T09:12:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8449999999999999e-06 2024-09-13T09:12:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068381+0000", 2024-09-13T09:12:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045384437 2024-09-13T09:12:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068461+0000", 2024-09-13T09:12:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0069999999999997e-05 2024-09-13T09:12:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068520+0000", 2024-09-13T09:12:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8656000000000002e-05 2024-09-13T09:12:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16134 3.c 3.c5acf96c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.023285+0000", 2024-09-13T09:12:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.129221252000001, 2024-09-13T09:12:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045544921000000002, 2024-09-13T09:12:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16134 2024-09-13T09:12:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023285+0000", 2024-09-13T09:12:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023285+0000", 2024-09-13T09:12:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023284+0000", 2024-09-13T09:12:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023286+0000", 2024-09-13T09:12:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.316e-06 2024-09-13T09:12:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023289+0000", 2024-09-13T09:12:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8169999999999999e-06 2024-09-13T09:12:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023291+0000", 2024-09-13T09:12:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.418e-06 2024-09-13T09:12:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068823+0000", 2024-09-13T09:12:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045531644000000003 2024-09-13T09:12:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068830+0000", 2024-09-13T09:12:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9450000000000004e-06 2024-09-13T09:12:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16135 3.c 3.c5acf96c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.023293+0000", 2024-09-13T09:12:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.129212976, 2024-09-13T09:12:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045568511999999999, 2024-09-13T09:12:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16135 2024-09-13T09:12:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023293+0000", 2024-09-13T09:12:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023293+0000", 2024-09-13T09:12:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023293+0000", 2024-09-13T09:12:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023294+0000", 2024-09-13T09:12:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.203e-06 2024-09-13T09:12:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023297+0000", 2024-09-13T09:12:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.768e-06 2024-09-13T09:12:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023300+0000", 2024-09-13T09:12:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2270000000000002e-06 2024-09-13T09:12:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068853+0000", 2024-09-13T09:12:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045553278000000003 2024-09-13T09:12:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068861+0000", 2024-09-13T09:12:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2579999999999995e-06 2024-09-13T09:12:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16136 3.c 3.c5acf96c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.023304+0000", 2024-09-13T09:12:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.129202171999999, 2024-09-13T09:12:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045591547000000003, 2024-09-13T09:12:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16136 2024-09-13T09:12:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023304+0000", 2024-09-13T09:12:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023304+0000", 2024-09-13T09:12:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023304+0000", 2024-09-13T09:12:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023305+0000", 2024-09-13T09:12:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.314e-06 2024-09-13T09:12:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023308+0000", 2024-09-13T09:12:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9919999999999999e-06 2024-09-13T09:12:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023310+0000", 2024-09-13T09:12:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.57e-06 2024-09-13T09:12:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068887+0000", 2024-09-13T09:12:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045576608999999997 2024-09-13T09:12:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068895+0000", 2024-09-13T09:12:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2959999999999999e-06 2024-09-13T09:12:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:16137 3.c 3.c5acf96c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.023331+0000", 2024-09-13T09:12:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.129175089, 2024-09-13T09:12:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045595084000000001, 2024-09-13T09:12:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 16137 2024-09-13T09:12:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023331+0000", 2024-09-13T09:12:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023331+0000", 2024-09-13T09:12:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023330+0000", 2024-09-13T09:12:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023333+0000", 2024-09-13T09:12:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4889999999999998e-06 2024-09-13T09:12:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023341+0000", 2024-09-13T09:12:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8779999999999998e-06 2024-09-13T09:12:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.023346+0000", 2024-09-13T09:12:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3600000000000004e-06 2024-09-13T09:12:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068919+0000", 2024-09-13T09:12:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045572806 2024-09-13T09:12:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.068926+0000", 2024-09-13T09:12:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.002e-06 2024-09-13T09:12:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17711 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2273280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140832+0000", 2024-09-13T09:12:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.011673890999999, 2024-09-13T09:12:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045543037000000001, 2024-09-13T09:12:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17711 2024-09-13T09:12:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140832+0000", 2024-09-13T09:12:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140832+0000", 2024-09-13T09:12:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140832+0000", 2024-09-13T09:12:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140834+0000", 2024-09-13T09:12:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:12:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140840+0000", 2024-09-13T09:12:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9519999999999999e-06 2024-09-13T09:12:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140844+0000", 2024-09-13T09:12:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3529999999999997e-06 2024-09-13T09:12:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186207+0000", 2024-09-13T09:12:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045362864000000003 2024-09-13T09:12:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186287+0000", 2024-09-13T09:12:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0023000000000003e-05 2024-09-13T09:12:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186375+0000", 2024-09-13T09:12:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7978000000000004e-05 2024-09-13T09:12:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17712 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2295808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140848+0000", 2024-09-13T09:12:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.01165773, 2024-09-13T09:12:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045675362999999997, 2024-09-13T09:12:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17712 2024-09-13T09:12:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140848+0000", 2024-09-13T09:12:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140848+0000", 2024-09-13T09:12:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140848+0000", 2024-09-13T09:12:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140850+0000", 2024-09-13T09:12:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0990000000000001e-06 2024-09-13T09:12:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140856+0000", 2024-09-13T09:12:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7289999999999997e-06 2024-09-13T09:12:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140860+0000", 2024-09-13T09:12:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6260000000000003e-06 2024-09-13T09:12:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186414+0000", 2024-09-13T09:12:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045553693999999999 2024-09-13T09:12:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186470+0000", 2024-09-13T09:12:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5855e-05 2024-09-13T09:12:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186524+0000", 2024-09-13T09:12:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3732e-05 2024-09-13T09:12:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17713 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2318336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140864+0000", 2024-09-13T09:12:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.011641761, 2024-09-13T09:12:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045842250000000001, 2024-09-13T09:12:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17713 2024-09-13T09:12:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140864+0000", 2024-09-13T09:12:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140864+0000", 2024-09-13T09:12:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140864+0000", 2024-09-13T09:12:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140866+0000", 2024-09-13T09:12:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0590000000000001e-06 2024-09-13T09:12:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140871+0000", 2024-09-13T09:12:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4990000000000002e-06 2024-09-13T09:12:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140876+0000", 2024-09-13T09:12:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3810000000000001e-06 2024-09-13T09:12:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186555+0000", 2024-09-13T09:12:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045679059000000001 2024-09-13T09:12:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186631+0000", 2024-09-13T09:12:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6168999999999995e-05 2024-09-13T09:12:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186706+0000", 2024-09-13T09:12:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5420999999999995e-05 2024-09-13T09:12:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17714 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2340864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140880+0000", 2024-09-13T09:12:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.011626293000001, 2024-09-13T09:12:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045950484999999999, 2024-09-13T09:12:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17714 2024-09-13T09:12:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140880+0000", 2024-09-13T09:12:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140880+0000", 2024-09-13T09:12:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140879+0000", 2024-09-13T09:12:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140881+0000", 2024-09-13T09:12:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9810000000000002e-06 2024-09-13T09:12:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140887+0000", 2024-09-13T09:12:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.874e-06 2024-09-13T09:12:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140891+0000", 2024-09-13T09:12:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3839999999999999e-06 2024-09-13T09:12:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186735+0000", 2024-09-13T09:12:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045843119000000002 2024-09-13T09:12:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186778+0000", 2024-09-13T09:12:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3618000000000001e-05 2024-09-13T09:12:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186830+0000", 2024-09-13T09:12:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1919000000000002e-05 2024-09-13T09:12:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17715 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2363392~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140896+0000", 2024-09-13T09:12:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.011610348, 2024-09-13T09:12:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046059503000000002, 2024-09-13T09:12:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17715 2024-09-13T09:12:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140896+0000", 2024-09-13T09:12:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140896+0000", 2024-09-13T09:12:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140895+0000", 2024-09-13T09:12:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140897+0000", 2024-09-13T09:12:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.029e-06 2024-09-13T09:12:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140904+0000", 2024-09-13T09:12:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9280000000000004e-06 2024-09-13T09:12:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140908+0000", 2024-09-13T09:12:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.262e-06 2024-09-13T09:12:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186856+0000", 2024-09-13T09:12:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045947057999999999 2024-09-13T09:12:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186899+0000", 2024-09-13T09:12:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3658000000000001e-05 2024-09-13T09:12:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186955+0000", 2024-09-13T09:12:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5899999999999997e-05 2024-09-13T09:12:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17716 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2385920~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140959+0000", 2024-09-13T09:12:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.011546770000001, 2024-09-13T09:12:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046114687000000001, 2024-09-13T09:12:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17716 2024-09-13T09:12:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140959+0000", 2024-09-13T09:12:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140959+0000", 2024-09-13T09:12:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140912+0000", 2024-09-13T09:12:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999533 2024-09-13T09:12:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140961+0000", 2024-09-13T09:12:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8798999999999998e-05 2024-09-13T09:12:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140966+0000", 2024-09-13T09:12:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4970000000000003e-06 2024-09-13T09:12:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140969+0000", 2024-09-13T09:12:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4170000000000001e-06 2024-09-13T09:12:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186982+0000", 2024-09-13T09:12:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046012982000000001 2024-09-13T09:12:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187024+0000", 2024-09-13T09:12:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1894999999999998e-05 2024-09-13T09:12:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187074+0000", 2024-09-13T09:12:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9956000000000001e-05 2024-09-13T09:12:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17717 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2408448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.141107+0000", 2024-09-13T09:12:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.011398627, 2024-09-13T09:12:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046093783999999999, 2024-09-13T09:12:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17717 2024-09-13T09:12:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141107+0000", 2024-09-13T09:12:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141107+0000", 2024-09-13T09:12:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140972+0000", 2024-09-13T09:12:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998646 2024-09-13T09:12:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141111+0000", 2024-09-13T09:12:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000138765 2024-09-13T09:12:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141116+0000", 2024-09-13T09:12:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:12:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141121+0000", 2024-09-13T09:12:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4120000000000003e-06 2024-09-13T09:12:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187096+0000", 2024-09-13T09:12:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045974947000000002 2024-09-13T09:12:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187145+0000", 2024-09-13T09:12:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9548000000000001e-05 2024-09-13T09:12:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187201+0000", 2024-09-13T09:12:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5708000000000003e-05 2024-09-13T09:12:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17718 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2430976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.141258+0000", 2024-09-13T09:12:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.011248077999999, 2024-09-13T09:12:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046068460999999998, 2024-09-13T09:12:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17718 2024-09-13T09:12:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141258+0000", 2024-09-13T09:12:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141258+0000", 2024-09-13T09:12:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141125+0000", 2024-09-13T09:12:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999867 2024-09-13T09:12:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141261+0000", 2024-09-13T09:12:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013606499999999999 2024-09-13T09:12:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141268+0000", 2024-09-13T09:12:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4610000000000002e-06 2024-09-13T09:12:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141275+0000", 2024-09-13T09:12:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8519999999999998e-06 2024-09-13T09:12:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187237+0000", 2024-09-13T09:12:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045961958999999997 2024-09-13T09:12:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187279+0000", 2024-09-13T09:12:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2324000000000003e-05 2024-09-13T09:12:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187326+0000", 2024-09-13T09:12:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7009999999999999e-05 2024-09-13T09:12:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17719 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2453504~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.141428+0000", 2024-09-13T09:12:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.011078269, 2024-09-13T09:12:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046028438999999997, 2024-09-13T09:12:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17719 2024-09-13T09:12:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141428+0000", 2024-09-13T09:12:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141428+0000", 2024-09-13T09:12:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141280+0000", 2024-09-13T09:12:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998522 2024-09-13T09:12:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141430+0000", 2024-09-13T09:12:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00015054599999999999 2024-09-13T09:12:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141435+0000", 2024-09-13T09:12:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:12:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141439+0000", 2024-09-13T09:12:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7699999999999999e-06 2024-09-13T09:12:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187350+0000", 2024-09-13T09:12:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045910377000000002 2024-09-13T09:12:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187398+0000", 2024-09-13T09:12:52.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8795999999999998e-05 2024-09-13T09:12:52.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187456+0000", 2024-09-13T09:12:52.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7667000000000002e-05 2024-09-13T09:12:52.218 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.218 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.218 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.218 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17720 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2476032~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.219 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.141638+0000", 2024-09-13T09:12:52.219 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.010867485, 2024-09-13T09:12:52.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045968368000000003, 2024-09-13T09:12:52.219 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.219 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.219 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.220 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.220 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.220 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17720 2024-09-13T09:12:52.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.220 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141638+0000", 2024-09-13T09:12:52.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141638+0000", 2024-09-13T09:12:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141443+0000", 2024-09-13T09:12:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999804 2024-09-13T09:12:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141641+0000", 2024-09-13T09:12:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019882100000000001 2024-09-13T09:12:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141648+0000", 2024-09-13T09:12:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.041e-06 2024-09-13T09:12:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.141653+0000", 2024-09-13T09:12:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.442e-06 2024-09-13T09:12:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187494+0000", 2024-09-13T09:12:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045841621999999999 2024-09-13T09:12:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187555+0000", 2024-09-13T09:12:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0979e-05 2024-09-13T09:12:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187607+0000", 2024-09-13T09:12:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1390000000000001e-05 2024-09-13T09:12:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17723 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2543616~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142510+0000", 2024-09-13T09:12:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.009995914999999, 2024-09-13T09:12:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045602886000000002, 2024-09-13T09:12:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17723 2024-09-13T09:12:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142510+0000", 2024-09-13T09:12:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142510+0000", 2024-09-13T09:12:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142510+0000", 2024-09-13T09:12:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142512+0000", 2024-09-13T09:12:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:12:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142518+0000", 2024-09-13T09:12:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5679999999999996e-06 2024-09-13T09:12:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142523+0000", 2024-09-13T09:12:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9409999999999998e-06 2024-09-13T09:12:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187960+0000", 2024-09-13T09:12:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045436973999999998 2024-09-13T09:12:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188032+0000", 2024-09-13T09:12:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1633999999999999e-05 2024-09-13T09:12:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188113+0000", 2024-09-13T09:12:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0866999999999997e-05 2024-09-13T09:12:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17724 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2566144~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142529+0000", 2024-09-13T09:12:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.009976441999999, 2024-09-13T09:12:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045741345000000003, 2024-09-13T09:12:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17724 2024-09-13T09:12:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142529+0000", 2024-09-13T09:12:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142529+0000", 2024-09-13T09:12:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142529+0000", 2024-09-13T09:12:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142531+0000", 2024-09-13T09:12:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4320000000000002e-06 2024-09-13T09:12:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142538+0000", 2024-09-13T09:12:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4330000000000001e-06 2024-09-13T09:12:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142543+0000", 2024-09-13T09:12:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:12:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188147+0000", 2024-09-13T09:12:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045603871999999997 2024-09-13T09:12:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188191+0000", 2024-09-13T09:12:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4388000000000003e-05 2024-09-13T09:12:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188271+0000", 2024-09-13T09:12:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9808999999999994e-05 2024-09-13T09:12:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17725 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2588672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142547+0000", 2024-09-13T09:12:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.009958644999999, 2024-09-13T09:12:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045918237000000001, 2024-09-13T09:12:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17725 2024-09-13T09:12:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142547+0000", 2024-09-13T09:12:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142547+0000", 2024-09-13T09:12:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142547+0000", 2024-09-13T09:12:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142549+0000", 2024-09-13T09:12:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1210000000000001e-06 2024-09-13T09:12:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142558+0000", 2024-09-13T09:12:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8720000000000008e-06 2024-09-13T09:12:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142563+0000", 2024-09-13T09:12:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:12:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188305+0000", 2024-09-13T09:12:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045742532000000002 2024-09-13T09:12:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188380+0000", 2024-09-13T09:12:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4277999999999996e-05 2024-09-13T09:12:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188466+0000", 2024-09-13T09:12:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5827000000000003e-05 2024-09-13T09:12:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17735 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2813952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144629+0000", 2024-09-13T09:12:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.007877176999999, 2024-09-13T09:12:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045584794999999997, 2024-09-13T09:12:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17735 2024-09-13T09:12:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144629+0000", 2024-09-13T09:12:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144629+0000", 2024-09-13T09:12:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144628+0000", 2024-09-13T09:12:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144631+0000", 2024-09-13T09:12:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3889999999999999e-06 2024-09-13T09:12:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144636+0000", 2024-09-13T09:12:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:12:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144641+0000", 2024-09-13T09:12:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4000000000000002e-06 2024-09-13T09:12:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190062+0000", 2024-09-13T09:12:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045421647000000002 2024-09-13T09:12:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190135+0000", 2024-09-13T09:12:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2191000000000001e-05 2024-09-13T09:12:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190214+0000", 2024-09-13T09:12:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8889999999999999e-05 2024-09-13T09:12:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17736 3.c 3:369f35a3:::rbd_data.1123d4249472.0000000000000003:10 [read 2836480~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144676+0000", 2024-09-13T09:12:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.007830238, 2024-09-13T09:12:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045739389999999998, 2024-09-13T09:12:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17736 2024-09-13T09:12:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144676+0000", 2024-09-13T09:12:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144676+0000", 2024-09-13T09:12:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144645+0000", 2024-09-13T09:12:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999969 2024-09-13T09:12:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144678+0000", 2024-09-13T09:12:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3572999999999999e-05 2024-09-13T09:12:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144687+0000", 2024-09-13T09:12:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4670000000000004e-06 2024-09-13T09:12:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144692+0000", 2024-09-13T09:12:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4099999999999999e-06 2024-09-13T09:12:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190247+0000", 2024-09-13T09:12:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045555224999999998 2024-09-13T09:12:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190329+0000", 2024-09-13T09:12:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1630000000000003e-05 2024-09-13T09:12:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190415+0000", 2024-09-13T09:12:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6017e-05 2024-09-13T09:12:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:52.246 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:52.248 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:52.247+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:52.248 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:52.288 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T09:12:52.303 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T09:12:52.304 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:12:52.360 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:52.360 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:52.360 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:52.360 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:52.361 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:52.359+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:52.370 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:52.449 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:52.448+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:52.523 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:52.524 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:52.524 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:52.524 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:52.524 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:52.533 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:52.551 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:52.550+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:52.642 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:52.642 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:52.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:52.642 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:52.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.642 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.642 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:12:52.642 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.110545950000001, 2024-09-13T09:12:52.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:12:52.642 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.642 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.642 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.643 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.643 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.643 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:12:52.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.643 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:12:52.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:12:52.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:12:52.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:12:52.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:12:52.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:12:52.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:12:52.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:12:52.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:12:52.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:12:52.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:12:52.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:12:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:12:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:12:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:12:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:12:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:12:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:12:52.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:12:52.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:12:52.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:12:52.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:12:52.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:12:52.648 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.648 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.648 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.648 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.648 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:12:52.648 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.110527915, 2024-09-13T09:12:52.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:12:52.648 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.648 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.648 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.648 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.649 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.649 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:12:52.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.649 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:12:52.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:12:52.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:12:52.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:12:52.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:12:52.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:12:52.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:12:52.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:12:52.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:12:52.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:12:52.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:12:52.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:12:52.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:12:52.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:12:52.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:12:52.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:12:52.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:12:52.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:12:52.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:12:52.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:12:52.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:12:52.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:12:52.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:12:52.654 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.654 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.654 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.654 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.654 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:12:52.654 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.110511245, 2024-09-13T09:12:52.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:12:52.654 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.654 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.654 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.654 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.655 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.655 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:12:52.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.655 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:12:52.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:12:52.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:12:52.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:12:52.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:12:52.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:12:52.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:12:52.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:12:52.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:12:52.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:12:52.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:12:52.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:12:52.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:12:52.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:12:52.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:12:52.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:12:52.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:12:52.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:12:52.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:12:52.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:12:52.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:12:52.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:12:52.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:12:52.659 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.659 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.660 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.660 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.660 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:12:52.660 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.110493612999999, 2024-09-13T09:12:52.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:12:52.660 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.660 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.660 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.660 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.660 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.660 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:12:52.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.661 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:12:52.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:12:52.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:12:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:12:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:12:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:12:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:12:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:12:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:12:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:12:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:12:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:12:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:12:52.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:12:52.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:12:52.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:12:52.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:12:52.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:12:52.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:12:52.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.666 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.666 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:12:52.666 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:12:52.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.666 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.666 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:12:52.666 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:12:52.666 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.666 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.666 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.667 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.667 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:12:52.667 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.110447307999999, 2024-09-13T09:12:52.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:12:52.667 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.667 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.667 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.667 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.667 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.667 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:12:52.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.667 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:12:52.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:12:52.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:12:52.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:12:52.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:12:52.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:12:52.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:12:52.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:12:52.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:12:52.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:12:52.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:12:52.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:12:52.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:12:52.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:12:52.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:12:52.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:12:52.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:12:52.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:12:52.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:12:52.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:12:52.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:12:52.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:12:52.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:12:52.672 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.673 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.673 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.673 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.673 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:12:52.673 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.110422007, 2024-09-13T09:12:52.673 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:12:52.673 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.673 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.673 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.673 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.673 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.673 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:12:52.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.674 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:12:52.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:12:52.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:12:52.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:12:52.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:12:52.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:12:52.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:12:52.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:12:52.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:12:52.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:12:52.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:12:52.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:12:52.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:12:52.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:12:52.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:12:52.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:12:52.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:12:52.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:12:52.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:12:52.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:12:52.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:12:52.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:12:52.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:12:52.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.678 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.679 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.679 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:12:52.679 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.110403778, 2024-09-13T09:12:52.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:12:52.679 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.679 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.679 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:12:52.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.679 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:12:52.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:12:52.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:12:52.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:12:52.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:12:52.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:12:52.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:12:52.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:12:52.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:12:52.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:12:52.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:12:52.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:12:52.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:12:52.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:12:52.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:12:52.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:12:52.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:12:52.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:12:52.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:12:52.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:12:52.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:12:52.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:12:52.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:12:52.684 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.684 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.684 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.684 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.685 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:12:52.685 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.110356459, 2024-09-13T09:12:52.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:12:52.685 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.685 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.685 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.685 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.685 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.685 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:12:52.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.685 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:12:52.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.686 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.686 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:12:52.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:12:52.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:12:52.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:12:52.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:12:52.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:12:52.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:12:52.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:12:52.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:12:52.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:12:52.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:12:52.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:12:52.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:12:52.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:12:52.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:12:52.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:12:52.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:12:52.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:12:52.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:12:52.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:12:52.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:12:52.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:12:52.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.690 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:12:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.109842220999999, 2024-09-13T09:12:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:12:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:12:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:12:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:12:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:12:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:12:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:12:52.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:12:52.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:12:52.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:12:52.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:12:52.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:12:52.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:12:52.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:12:52.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:12:52.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:12:52.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:12:52.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:12:52.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:12:52.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:12:52.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:12:52.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:12:52.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:12:52.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:12:52.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:12:52.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:12:52.696 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.696 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.696 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.696 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.696 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:12:52.696 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.109807691, 2024-09-13T09:12:52.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:12:52.697 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.697 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.697 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.697 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.697 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.697 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:12:52.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.697 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:12:52.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:12:52.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:12:52.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:12:52.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:12:52.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:12:52.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:12:52.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:12:52.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:12:52.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:12:52.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:12:52.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:12:52.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:12:52.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:12:52.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:12:52.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:12:52.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:12:52.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:12:52.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:12:52.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:12:52.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:12:52.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:12:52.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:12:52.702 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.702 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.702 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.702 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.702 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:12:52.702 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.109789784, 2024-09-13T09:12:52.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:12:52.703 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.703 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.703 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.703 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.703 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.703 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:12:52.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.703 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:12:52.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:12:52.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:12:52.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:12:52.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:12:52.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:12:52.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:12:52.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:12:52.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:12:52.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:12:52.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:12:52.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:12:52.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:12:52.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:12:52.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:12:52.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:12:52.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:12:52.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:12:52.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:12:52.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:12:52.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:12:52.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:12:52.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:12:52.708 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.708 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.708 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.708 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.708 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:12:52.708 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.100662936000001, 2024-09-13T09:12:52.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:12:52.708 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.708 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.708 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.708 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.708 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.709 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:12:52.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.709 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:12:52.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:12:52.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:12:52.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:12:52.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:12:52.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:12:52.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:12:52.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:12:52.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:12:52.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:12:52.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:12:52.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:12:52.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:12:52.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:12:52.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:12:52.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:12:52.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:12:52.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:12:52.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:12:52.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:12:52.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:12:52.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:12:52.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:12:52.713 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.713 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.713 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.714 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.714 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:12:52.714 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.100611356, 2024-09-13T09:12:52.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:12:52.714 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.714 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.714 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.714 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.714 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.714 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:12:52.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.714 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:12:52.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:12:52.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:12:52.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:12:52.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:12:52.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:12:52.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:12:52.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:12:52.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:12:52.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:12:52.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:12:52.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:12:52.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:12:52.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:12:52.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:12:52.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.718 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.718 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:12:52.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:12:52.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.718 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.718 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:12:52.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:12:52.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.718 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:12:52.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:12:52.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:12:52.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:12:52.719 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.719 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.719 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.719 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.719 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:12:52.720 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.10059508, 2024-09-13T09:12:52.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:12:52.720 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.720 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.720 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.720 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.720 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.720 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:12:52.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.720 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:12:52.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.721 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:12:52.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.721 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:12:52.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.721 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:12:52.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:12:52.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:12:52.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:12:52.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:12:52.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:12:52.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:12:52.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:12:52.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:12:52.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:12:52.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:12:52.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:12:52.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:12:52.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:12:52.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:12:52.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:12:52.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:12:52.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:12:52.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:12:52.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:12:52.725 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.725 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.725 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.725 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.725 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:12:52.725 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.100579306, 2024-09-13T09:12:52.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:12:52.726 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.726 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.726 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.726 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.726 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.726 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:12:52.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.726 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:12:52.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:12:52.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:12:52.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:12:52.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:12:52.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:12:52.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:12:52.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:12:52.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:12:52.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:12:52.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:12:52.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:12:52.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:12:52.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:12:52.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:12:52.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:12:52.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:12:52.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:12:52.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:12:52.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:12:52.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:12:52.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:12:52.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:12:52.731 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.731 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.731 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.731 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.731 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:12:52.732 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.100564431, 2024-09-13T09:12:52.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:12:52.732 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.732 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.732 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.732 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.732 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.732 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:12:52.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.732 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.732 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.732 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:12:52.733 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.733 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.733 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:12:52.733 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.733 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.733 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:12:52.733 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.733 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:12:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:12:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:12:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:12:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:12:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:12:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:12:52.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:12:52.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:12:52.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:12:52.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:12:52.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:12:52.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:12:52.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:12:52.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:12:52.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:12:52.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:12:52.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:12:52.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.737 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:12:52.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:12:52.737 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.737 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.737 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.737 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.737 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:12:52.737 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.100546969, 2024-09-13T09:12:52.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:12:52.737 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.737 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.738 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.738 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.738 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.738 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:12:52.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.738 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:12:52.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:12:52.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.739 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.739 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:12:52.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.739 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.739 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:12:52.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:12:52.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.739 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.739 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:12:52.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:12:52.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.740 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:12:52.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:12:52.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.740 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:12:52.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:12:52.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:12:52.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:12:52.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:12:52.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:12:52.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:12:52.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:12:52.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:12:52.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:12:52.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:12:52.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:12:52.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:12:52.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:12:52.743 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.743 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.743 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.743 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.743 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:12:52.743 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.098946336999999, 2024-09-13T09:12:52.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:12:52.743 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.743 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.743 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.743 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.744 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.744 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:12:52.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.744 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.744 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.744 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:12:52.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.744 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.744 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:12:52.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:12:52.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:12:52.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:12:52.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:12:52.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:12:52.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:12:52.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:12:52.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:12:52.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:12:52.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:12:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:12:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:12:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:12:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:12:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:12:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:12:52.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:12:52.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.748 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.748 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:12:52.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:12:52.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.748 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.748 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:12:52.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:12:52.748 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.748 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.749 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.749 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.749 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:12:52.749 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.098906286, 2024-09-13T09:12:52.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:12:52.749 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.749 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.749 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.749 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.749 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.749 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:12:52.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.750 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:12:52.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:12:52.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:12:52.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:12:52.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:12:52.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:12:52.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:12:52.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:12:52.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:12:52.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.752 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:12:52.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:12:52.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.752 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:12:52.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:12:52.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.752 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:12:52.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:12:52.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.753 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:12:52.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:12:52.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.753 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:12:52.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:12:52.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.754 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:12:52.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:12:52.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.754 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.754 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:12:52.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:12:52.754 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.754 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.754 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.754 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:52.754 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:12:52.755 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.098862522999999, 2024-09-13T09:12:52.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:12:52.755 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:52.755 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:52.755 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:52.755 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:52.755 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:52.755 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:12:52.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.755 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:52.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:52.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:12:52.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:52.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:12:52.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:52.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:52.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:12:52.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:52.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:52.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:12:52.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:12:52.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:52.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:12:52.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:12:52.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:52.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:12:52.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:12:52.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:12:52.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:12:52.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:12:52.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:12:52.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:12:52.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:12:52.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:52.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:12:52.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:12:52.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:52.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:12:52.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:12:52.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:52.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:12:52.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:12:52.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:52.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:52.760 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:52.760 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:12:52.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:12:52.760 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.760 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.760 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.760 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:52.760 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:52.760 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:52.762 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:52.763 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:52.760+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:52.764 INFO:teuthology.orchestra.run.smithi157.stdout:2534 read 0xd5f9600 thru 0xd5fad90 (0x1791 bytes) 2024-09-13T09:12:52.764 INFO:teuthology.orchestra.run.smithi157.stdout:2538 read 0x530e00 thru 0x5313b1 (0x5b2 bytes) 2024-09-13T09:12:52.764 INFO:teuthology.orchestra.run.smithi157.stdout:2540 write 0x1132400 thru 0x1135bef (0x37f0 bytes) 2024-09-13T09:12:52.764 INFO:teuthology.orchestra.run.smithi157.stdout:2541 read 0x159d000 thru 0x15abf41 (0xef42 bytes) 2024-09-13T09:12:52.764 INFO:teuthology.orchestra.run.smithi157.stdout:2542 write 0x6bef000 thru 0x6bfb9e8 (0xc9e9 bytes) 2024-09-13T09:12:52.765 INFO:teuthology.orchestra.run.smithi157.stdout:2544 write 0x77f3a00 thru 0x77f9be6 (0x61e7 bytes) 2024-09-13T09:12:52.765 INFO:teuthology.orchestra.run.smithi157.stdout:2546 read 0xc78a00 thru 0xc7d6ae (0x4caf bytes) 2024-09-13T09:12:52.765 INFO:teuthology.orchestra.run.smithi157.stdout:2548 read 0x673a800 thru 0x67477bf (0xcfc0 bytes) 2024-09-13T09:12:52.765 INFO:teuthology.orchestra.run.smithi157.stdout:2549 read 0xd689200 thru 0xd694d2b (0xbb2c bytes) 2024-09-13T09:12:52.765 INFO:teuthology.orchestra.run.smithi157.stdout:2553 read 0x9cc6400 thru 0x9cd32c4 (0xcec5 bytes) 2024-09-13T09:12:52.765 INFO:teuthology.orchestra.run.smithi157.stdout:2554 write 0x3b7c000 thru 0x3b80f22 (0x4f23 bytes) 2024-09-13T09:12:52.765 INFO:teuthology.orchestra.run.smithi157.stdout:2555 read 0x3987600 thru 0x39945b4 (0xcfb5 bytes) 2024-09-13T09:12:52.765 INFO:teuthology.orchestra.run.smithi157.stdout:2557 write 0xcd30800 thru 0xcd3a7f7 (0x9ff8 bytes) 2024-09-13T09:12:52.765 INFO:teuthology.orchestra.run.smithi157.stdout:2559 write 0x8d30800 thru 0x8d3e9ec (0xe1ed bytes) 2024-09-13T09:12:52.765 INFO:teuthology.orchestra.run.smithi157.stdout:2563 write 0x8cd8000 thru 0x8cded98 (0x6d99 bytes) 2024-09-13T09:12:52.766 INFO:teuthology.orchestra.run.smithi157.stdout:2564 write 0x48bf800 thru 0x48c4202 (0x4a03 bytes) 2024-09-13T09:12:52.766 INFO:teuthology.orchestra.run.smithi157.stdout:2566 read 0xa728800 thru 0xa736174 (0xd975 bytes) 2024-09-13T09:12:52.766 INFO:teuthology.orchestra.run.smithi157.stdout:2567 read 0x8330000 thru 0x833c99b (0xc99c bytes) 2024-09-13T09:12:52.766 INFO:teuthology.orchestra.run.smithi157.stdout:2572 trunc from 0xe70fc00 to 0x6c37400 2024-09-13T09:12:52.862 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:52.861+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:52.925 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:52.925 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:52.925 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:52.925 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:52.934 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:52.963 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:52.962+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:53.017 INFO:teuthology.orchestra.run.smithi157.stdout:2573 read 0x34a2200 thru 0x34a57fd (0x35fe bytes) 2024-09-13T09:12:53.041 INFO:teuthology.orchestra.run.smithi157.stdout:2574 write 0xd53aa00 thru 0xd54901f (0xe620 bytes) 2024-09-13T09:12:53.041 INFO:teuthology.orchestra.run.smithi157.stdout:2576 write 0x6096000 thru 0x60a1f77 (0xbf78 bytes) 2024-09-13T09:12:53.041 INFO:teuthology.orchestra.run.smithi157.stdout:2578 punch from 0x22a5400 to 0x22a9c00, (0x4800 bytes) 2024-09-13T09:12:53.042 INFO:teuthology.orchestra.run.smithi157.stdout:2579 punch from 0x4298200 to 0x429d600, (0x5400 bytes) 2024-09-13T09:12:53.060 INFO:teuthology.orchestra.run.smithi157.stdout:2581 write 0xea45600 thru 0xea49dc4 (0x47c5 bytes) 2024-09-13T09:12:53.062 INFO:teuthology.orchestra.run.smithi157.stdout:2583 read 0x48bce00 thru 0x48bf318 (0x2519 bytes) 2024-09-13T09:12:53.064 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:53.063+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:53.068 INFO:teuthology.orchestra.run.smithi157.stdout:2584 trunc from 0xea49e00 to 0x9dafa00 2024-09-13T09:12:53.089 INFO:teuthology.orchestra.run.smithi157.stdout:2587 read 0x6e7000 thru 0x6f2a02 (0xba03 bytes) 2024-09-13T09:12:53.090 INFO:teuthology.orchestra.run.smithi157.stdout:2588 read 0x3b3b200 thru 0x3b434d4 (0x82d5 bytes) 2024-09-13T09:12:53.090 INFO:teuthology.orchestra.run.smithi157.stdout:2591 write 0x421a200 thru 0x4222971 (0x8772 bytes) 2024-09-13T09:12:53.091 INFO:teuthology.orchestra.run.smithi157.stdout:2593 write 0x519c600 thru 0x51a0813 (0x4214 bytes) 2024-09-13T09:12:53.091 INFO:teuthology.orchestra.run.smithi157.stdout:2594 read 0x59c9400 thru 0x59cf841 (0x6442 bytes) 2024-09-13T09:12:53.093 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:53.093 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:53.093 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:53.093 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:53.093 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:53.093 INFO:teuthology.orchestra.run.smithi157.stdout:2595 trunc from 0x9dafa00 to 0x84d0e00 2024-09-13T09:12:53.103 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:53.112 INFO:teuthology.orchestra.run.smithi157.stdout:2596 read 0x4953400 thru 0x4953651 (0x252 bytes) 2024-09-13T09:12:53.113 INFO:teuthology.orchestra.run.smithi157.stdout:2597 read 0x7018200 thru 0x7021c40 (0x9a41 bytes) 2024-09-13T09:12:53.113 INFO:teuthology.orchestra.run.smithi157.stdout:2598 punch from 0x6723e00 to 0x672c800, (0x8a00 bytes) 2024-09-13T09:12:53.138 INFO:teuthology.orchestra.run.smithi157.stdout:2599 write 0xe5e2e00 thru 0xe5e9fb7 (0x71b8 bytes) 2024-09-13T09:12:53.138 INFO:teuthology.orchestra.run.smithi157.stdout:2600 write 0x29e5a00 thru 0x29eb0f3 (0x56f4 bytes) 2024-09-13T09:12:53.139 INFO:teuthology.orchestra.run.smithi157.stdout:2601 write 0x93d7800 thru 0x93d9398 (0x1b99 bytes) 2024-09-13T09:12:53.139 INFO:teuthology.orchestra.run.smithi157.stdout:2602 write 0x7611600 thru 0x7614022 (0x2a23 bytes) 2024-09-13T09:12:53.151 INFO:teuthology.orchestra.run.smithi157.stdout:2604 trunc from 0xe5ea000 to 0x5726a00 2024-09-13T09:12:53.177 INFO:teuthology.orchestra.run.smithi157.stdout:2606 read 0x3660200 thru 0x3665dba (0x5bbb bytes) 2024-09-13T09:12:53.178 INFO:teuthology.orchestra.run.smithi157.stdout:2607 write 0x4978a00 thru 0x497fb42 (0x7143 bytes) 2024-09-13T09:12:53.178 INFO:teuthology.orchestra.run.smithi157.stdout:2608 read 0x358a00 thru 0x363dd9 (0xb3da bytes) 2024-09-13T09:12:53.179 INFO:teuthology.orchestra.run.smithi157.stdout:2610 write 0x1e28000 thru 0x1e2ed7e (0x6d7f bytes) 2024-09-13T09:12:53.179 INFO:teuthology.orchestra.run.smithi157.stdout:2611 punch from 0x408b800 to 0x409ae00, (0xf600 bytes) 2024-09-13T09:12:53.180 INFO:teuthology.orchestra.run.smithi157.stdout:2612 write 0x312c200 thru 0x3130f99 (0x4d9a bytes) 2024-09-13T09:12:53.187 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:53.186+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:53.187 INFO:teuthology.orchestra.run.smithi157.stdout:2613 trunc from 0x5726a00 to 0x2640400 2024-09-13T09:12:53.239 INFO:teuthology.orchestra.run.smithi157.stdout:2617 write 0xb3d2200 thru 0xb3d5660 (0x3461 bytes) 2024-09-13T09:12:53.239 INFO:teuthology.orchestra.run.smithi157.stdout:2619 read 0x7195200 thru 0x71981e6 (0x2fe7 bytes) 2024-09-13T09:12:53.239 INFO:teuthology.orchestra.run.smithi157.stdout:2620 write 0x25eb600 thru 0x25f5698 (0xa099 bytes) 2024-09-13T09:12:53.239 INFO:teuthology.orchestra.run.smithi157.stdout:2621 write 0x5cbba00 thru 0x5cc46bc (0x8cbd bytes) 2024-09-13T09:12:53.240 INFO:teuthology.orchestra.run.smithi157.stdout:2622 read 0x5e76000 thru 0x5e7b8cf (0x58d0 bytes) 2024-09-13T09:12:53.250 INFO:teuthology.orchestra.run.smithi157.stdout:2623 trunc from 0xb3d5800 to 0x2e56a00 2024-09-13T09:12:53.259 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:53.260 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:53.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:53.260 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:53.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.260 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.260 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:12:53.260 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.754083632, 2024-09-13T09:12:53.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:12:53.260 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.260 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.260 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.261 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.261 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.261 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:12:53.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.261 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:12:53.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:12:53.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:12:53.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:12:53.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:12:53.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:12:53.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:12:53.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:12:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:12:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:12:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.754055948, 2024-09-13T09:12:53.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:12:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:12:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:12:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:12:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:12:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:12:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:12:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:12:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:12:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:12:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:12:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:12:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:12:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:12:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:12:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:12:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:12:53.268 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:12:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.754031607, 2024-09-13T09:12:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:12:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:12:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:12:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:12:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:12:53.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:12:53.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:12:53.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:12:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:12:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:12:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:12:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:12:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:12:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:12:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:12:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:12:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:12:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.273 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:12:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.754015287, 2024-09-13T09:12:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:12:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:12:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.274 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:12:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:12:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:12:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:12:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:12:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:12:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:12:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:12:53.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:12:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:12:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:12:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:12:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:12:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:12:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:12:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:12:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.753998872, 2024-09-13T09:12:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:12:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.278 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:12:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:12:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:12:53.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:12:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:12:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:12:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.280 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:12:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:12:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:12:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:12:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:12:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:12:53.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:12:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:12:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:12:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:12:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:12:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.753982764, 2024-09-13T09:12:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:12:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:12:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:12:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:12:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:12:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:12:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:12:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:12:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:12:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:12:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:12:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:12:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:12:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:12:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:12:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.286 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:12:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:12:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:12:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.753967203, 2024-09-13T09:12:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:12:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.287 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:12:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:12:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:12:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:12:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:12:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:12:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:12:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:12:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:12:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:12:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:12:53.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:12:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:12:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:12:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:12:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:12:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:12:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.753950365, 2024-09-13T09:12:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:12:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:12:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:12:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:12:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:12:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:12:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:12:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:12:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:12:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:12:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:12:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:12:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:12:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:12:53.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:12:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:12:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:12:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:12:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.753934373, 2024-09-13T09:12:53.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:12:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:12:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:12:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:12:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:12:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:12:53.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:12:53.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:12:53.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:12:53.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:12:53.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:12:53.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:12:53.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:12:53.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:12:53.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:12:53.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:12:53.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:12:53.301 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.301 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.301 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.301 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.301 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:12:53.301 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.753906875, 2024-09-13T09:12:53.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:12:53.301 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.301 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.301 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.301 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.302 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.302 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:12:53.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.302 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.302 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.302 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:12:53.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.302 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.302 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:12:53.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:12:53.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:12:53.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:12:53.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:12:53.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:12:53.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:12:53.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:12:53.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:12:53.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:12:53.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:12:53.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:12:53.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:12:53.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:12:53.305 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.305 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.305 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.306 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.306 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:12:53.306 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.753881015999999, 2024-09-13T09:12:53.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:12:53.306 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.306 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.306 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.306 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.306 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.306 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:12:53.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.307 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:12:53.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:12:53.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:12:53.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:12:53.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:12:53.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:12:53.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:12:53.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:12:53.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:12:53.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:12:53.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:12:53.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:12:53.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:12:53.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:12:53.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:12:53.310 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.310 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.310 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.310 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.311 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:12:53.311 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.753839526, 2024-09-13T09:12:53.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:12:53.311 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.311 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.311 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.311 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.311 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.311 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:12:53.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.311 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.311 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:12:53.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:12:53.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:12:53.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:12:53.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:12:53.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:12:53.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:12:53.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:12:53.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:12:53.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:12:53.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:12:53.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:12:53.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:12:53.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:12:53.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:12:53.315 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.315 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.315 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.315 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.315 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:12:53.315 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.753818288, 2024-09-13T09:12:53.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:12:53.315 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.316 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.316 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.316 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.316 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.316 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:12:53.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.316 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:12:53.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:12:53.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:12:53.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:12:53.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:12:53.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:12:53.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:12:53.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:12:53.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:12:53.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:12:53.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:12:53.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:12:53.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:12:53.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:12:53.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:12:53.320 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.320 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.320 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.320 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.320 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:12:53.320 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.753795488, 2024-09-13T09:12:53.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:12:53.320 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.321 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.321 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.321 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.321 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.321 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:12:53.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.321 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:12:53.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:12:53.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:12:53.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:12:53.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:12:53.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:12:53.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:12:53.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.324 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.324 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:12:53.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:12:53.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.324 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.324 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:12:53.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:12:53.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.324 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.324 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:12:53.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:12:53.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:12:53.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:12:53.325 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.325 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.325 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.325 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.325 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:12:53.326 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.753779032000001, 2024-09-13T09:12:53.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:12:53.326 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.326 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.326 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.326 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.326 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.326 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:12:53.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.326 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:12:53.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:12:53.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:12:53.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:12:53.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:12:53.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:12:53.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:12:53.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:12:53.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:12:53.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:12:53.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:12:53.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:12:53.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:12:53.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:12:53.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:12:53.330 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.330 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.330 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.330 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.330 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:12:53.331 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.753533895, 2024-09-13T09:12:53.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:12:53.331 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.331 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.331 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.331 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.331 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.331 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:12:53.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.331 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:12:53.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:12:53.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:12:53.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:12:53.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:12:53.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:12:53.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:12:53.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:12:53.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:12:53.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:12:53.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:12:53.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:12:53.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:12:53.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:12:53.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:12:53.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:12:53.335 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.335 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.335 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.335 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20408 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5582848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.335 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.505992+0000", 2024-09-13T09:12:53.335 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.751669315000001, 2024-09-13T09:12:53.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050463591000000002, 2024-09-13T09:12:53.335 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.336 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.336 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.336 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.336 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.336 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20408 2024-09-13T09:12:53.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.336 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505992+0000", 2024-09-13T09:12:53.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505992+0000", 2024-09-13T09:12:53.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505991+0000", 2024-09-13T09:12:53.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505994+0000", 2024-09-13T09:12:53.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6259999999999999e-06 2024-09-13T09:12:53.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506002+0000", 2024-09-13T09:12:53.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1550000000000007e-06 2024-09-13T09:12:53.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506008+0000", 2024-09-13T09:12:53.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.553e-06 2024-09-13T09:12:53.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556375+0000", 2024-09-13T09:12:53.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050366850999999997 2024-09-13T09:12:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556412+0000", 2024-09-13T09:12:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7138999999999998e-05 2024-09-13T09:12:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556455+0000", 2024-09-13T09:12:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3733999999999998e-05 2024-09-13T09:12:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.340 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.340 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:12:53.340 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.751648336000001, 2024-09-13T09:12:53.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:12:53.340 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.340 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.340 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.340 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.340 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.340 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:12:53.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.340 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:12:53.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:12:53.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:12:53.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:12:53.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:12:53.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:12:53.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:12:53.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:12:53.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:12:53.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:12:53.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:12:53.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:12:53.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:12:53.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:12:53.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:12:53.344 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.344 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.344 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.344 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.344 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:12:53.344 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.75162697, 2024-09-13T09:12:53.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:12:53.345 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.345 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.345 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.345 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.345 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.345 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:12:53.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.345 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:12:53.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:12:53.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:12:53.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:53.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:12:53.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:12:53.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:12:53.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:12:53.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:12:53.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:12:53.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:12:53.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:12:53.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:12:53.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:12:53.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:12:53.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:12:53.348 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.348 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.349 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.349 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:53.349 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:12:53.349 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.751421602000001, 2024-09-13T09:12:53.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:12:53.349 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:53.349 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:53.349 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:53.349 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:53.349 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:53.349 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:12:53.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.350 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:53.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:53.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:12:53.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:53.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:12:53.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:53.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:53.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:12:53.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:12:53.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:53.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:12:53.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:12:53.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:53.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:12:53.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:12:53.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:53.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:12:53.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:12:53.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:53.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:12:53.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:12:53.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:53.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:12:53.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:12:53.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:53.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:53.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:53.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:12:53.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:12:53.353 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.353 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.353 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.353 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:53.353 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:53.353 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:53.355 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:53.356 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T09:12:53.356 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-09-13T09:12:53.360 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T09:12:53.360 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:12:53.362 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:53.353+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:53.362 INFO:teuthology.orchestra.run.smithi157.stdout:2625 write 0x9b8de00 thru 0x9b8ec47 (0xe48 bytes) 2024-09-13T09:12:53.362 INFO:teuthology.orchestra.run.smithi157.stdout:2630 punch from 0x6821e00 to 0x6825200, (0x3400 bytes) 2024-09-13T09:12:53.362 INFO:teuthology.orchestra.run.smithi157.stdout:2632 write 0xcd8b800 thru 0xcd8f844 (0x4045 bytes) 2024-09-13T09:12:53.362 INFO:teuthology.orchestra.run.smithi157.stdout:2635 punch from 0x15f0000 to 0x15f8800, (0x8800 bytes) 2024-09-13T09:12:53.362 INFO:teuthology.orchestra.run.smithi157.stdout:2636 read 0x2e14400 thru 0x2e192f9 (0x4efa bytes) 2024-09-13T09:12:53.362 INFO:teuthology.orchestra.run.smithi157.stdout:2637 read 0x58a8200 thru 0x58ac64b (0x444c bytes) 2024-09-13T09:12:53.362 INFO:teuthology.orchestra.run.smithi157.stdout:2638 write 0x5eb0a00 thru 0x5eb817c (0x777d bytes) 2024-09-13T09:12:53.363 INFO:teuthology.orchestra.run.smithi157.stdout:2641 punch from 0x7b5e800 to 0x7b6e400, (0xfc00 bytes) 2024-09-13T09:12:53.363 INFO:teuthology.orchestra.run.smithi157.stdout:2644 write 0x91fb800 thru 0x92062c5 (0xaac6 bytes) 2024-09-13T09:12:53.363 INFO:teuthology.orchestra.run.smithi157.stdout:2646 write 0x57caa00 thru 0x57d5b2b (0xb12c bytes) 2024-09-13T09:12:53.363 INFO:teuthology.orchestra.run.smithi157.stdout:2647 write 0x1709e00 thru 0x1710893 (0x6a94 bytes) 2024-09-13T09:12:53.363 INFO:teuthology.orchestra.run.smithi157.stdout:2651 write 0x2495400 thru 0x2496efd (0x1afe bytes) 2024-09-13T09:12:53.363 INFO:teuthology.orchestra.run.smithi157.stdout:2653 write 0x272fc00 thru 0x272ff9a (0x39b bytes) 2024-09-13T09:12:53.363 INFO:teuthology.orchestra.run.smithi157.stdout:2656 punch from 0x2399400 to 0x23a9000, (0xfc00 bytes) 2024-09-13T09:12:53.363 INFO:teuthology.orchestra.run.smithi157.stdout:2658 read 0x6db5600 thru 0x6dba963 (0x5364 bytes) 2024-09-13T09:12:53.363 INFO:teuthology.orchestra.run.smithi157.stdout:2660 write 0x8b7ce00 thru 0x8b86fea (0xa1eb bytes) 2024-09-13T09:12:53.363 INFO:teuthology.orchestra.run.smithi157.stdout:2661 read 0xbcc00 thru 0xc14dc (0x48dd bytes) 2024-09-13T09:12:53.363 INFO:teuthology.orchestra.run.smithi157.stdout:2662 read 0xa0d8000 thru 0xa0d9114 (0x1115 bytes) 2024-09-13T09:12:53.363 INFO:teuthology.orchestra.run.smithi157.stdout:2665 write 0xdf17c00 thru 0xdf1c922 (0x4d23 bytes) 2024-09-13T09:12:53.363 INFO:teuthology.orchestra.run.smithi157.stdout:2666 read 0x7828a00 thru 0x7832eff (0xa500 bytes) 2024-09-13T09:12:53.364 INFO:teuthology.orchestra.run.smithi157.stdout:2667 trunc from 0xdf1ca00 to 0xaeeb400 2024-09-13T09:12:53.373 INFO:teuthology.orchestra.run.smithi157.stdout:2668 trunc from 0xaeeb400 to 0xe360400 2024-09-13T09:12:53.385 INFO:teuthology.orchestra.run.smithi157.stdout:2671 read 0x24de400 thru 0x24df068 (0xc69 bytes) 2024-09-13T09:12:53.393 INFO:teuthology.orchestra.run.smithi157.stdout:2674 write 0x5484600 thru 0x5487e61 (0x3862 bytes) 2024-09-13T09:12:53.394 INFO:teuthology.orchestra.run.smithi157.stdout:2675 read 0xb948600 thru 0xb94ff34 (0x7935 bytes) 2024-09-13T09:12:53.401 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:12:53.406 INFO:teuthology.orchestra.run.smithi157.stdout:2676 write 0xeded000 thru 0xedefcfb (0x2cfc bytes) 2024-09-13T09:12:53.417 INFO:teuthology.orchestra.run.smithi157.stdout:2683 trunc from 0xedefe00 to 0x6788600 2024-09-13T09:12:53.447 INFO:teuthology.orchestra.run.smithi157.stdout:2687 trunc from 0x6788600 to 0x5699400 2024-09-13T09:12:53.455 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:53.455+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:53.460 INFO:teuthology.orchestra.run.smithi157.stdout:2688 write 0x3692e00 thru 0x36a23ac (0xf5ad bytes) 2024-09-13T09:12:53.479 INFO:teuthology.orchestra.run.smithi157.stdout:2689 write 0x722e200 thru 0x7231a68 (0x3869 bytes) 2024-09-13T09:12:53.481 INFO:teuthology.orchestra.run.smithi157.stdout:2691 read 0x41ef200 thru 0x41f48f5 (0x56f6 bytes) 2024-09-13T09:12:53.503 INFO:teuthology.orchestra.run.smithi157.stdout:2692 write 0xba68a00 thru 0xba77776 (0xed77 bytes) 2024-09-13T09:12:53.504 INFO:teuthology.orchestra.run.smithi157.stdout:2693 write 0x33c7000 thru 0x33ce26e (0x726f bytes) 2024-09-13T09:12:53.505 INFO:teuthology.orchestra.run.smithi157.stdout:2694 punch from 0xa649a00 to 0xa658200, (0xe800 bytes) 2024-09-13T09:12:53.505 INFO:teuthology.orchestra.run.smithi157.stdout:2696 punch from 0x80a6a00 to 0x80b5c00, (0xf200 bytes) 2024-09-13T09:12:53.506 INFO:teuthology.orchestra.run.smithi157.stdout:2697 punch from 0xadf2000 to 0xadffa00, (0xda00 bytes) 2024-09-13T09:12:53.506 INFO:teuthology.orchestra.run.smithi157.stdout:2698 write 0x6d64000 thru 0x6d6a571 (0x6572 bytes) 2024-09-13T09:12:53.506 INFO:teuthology.orchestra.run.smithi157.stdout:2699 punch from 0x88ae400 to 0x88b9a00, (0xb600 bytes) 2024-09-13T09:12:53.506 INFO:teuthology.orchestra.run.smithi157.stdout:2701 read 0x9bd0400 thru 0x9bd834a (0x7f4b bytes) 2024-09-13T09:12:53.507 INFO:teuthology.orchestra.run.smithi157.stdout:2703 read 0x4499200 thru 0x44a7de7 (0xebe8 bytes) 2024-09-13T09:12:53.508 INFO:teuthology.orchestra.run.smithi157.stdout:2704 write 0x3c15c00 thru 0x3c1de5c (0x825d bytes) 2024-09-13T09:12:53.509 INFO:teuthology.orchestra.run.smithi157.stdout:2706 punch from 0x485d000 to 0x4860400, (0x3400 bytes) 2024-09-13T09:12:53.510 INFO:teuthology.orchestra.run.smithi157.stdout:2710 read 0x1b45c00 thru 0x1b518b0 (0xbcb1 bytes) 2024-09-13T09:12:53.510 INFO:teuthology.orchestra.run.smithi157.stdout:2713 read 0x686200 thru 0x690135 (0x9f36 bytes) 2024-09-13T09:12:53.511 INFO:teuthology.orchestra.run.smithi157.stdout:2716 read 0x254a000 thru 0x2553b53 (0x9b54 bytes) 2024-09-13T09:12:53.512 INFO:teuthology.orchestra.run.smithi157.stdout:2717 read 0xb0e5600 thru 0xb0f4eda (0xf8db bytes) 2024-09-13T09:12:53.512 INFO:teuthology.orchestra.run.smithi157.stdout:2722 write 0xa5a3600 thru 0xa5b15c0 (0xdfc1 bytes) 2024-09-13T09:12:53.513 INFO:teuthology.orchestra.run.smithi157.stdout:2724 read 0x8054a00 thru 0x80637fa (0xedfb bytes) 2024-09-13T09:12:53.523 INFO:teuthology.orchestra.run.smithi157.stdout:2726 trunc from 0xba77800 to 0x3456c00 2024-09-13T09:12:53.526 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:53.526 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:53.526 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:53.526 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:53.538 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:53.556 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:53.555+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:53.570 INFO:teuthology.orchestra.run.smithi157.stdout:2728 write 0xbfcb800 thru 0xbfd44af (0x8cb0 bytes) 2024-09-13T09:12:53.575 INFO:teuthology.orchestra.run.smithi157.stdout:2729 write 0x38ad600 thru 0x38af46e (0x1e6f bytes) 2024-09-13T09:12:53.576 INFO:teuthology.orchestra.run.smithi157.stdout:2735 punch from 0xa238c00 to 0xa23ee00, (0x6200 bytes) 2024-09-13T09:12:53.576 INFO:teuthology.orchestra.run.smithi157.stdout:2740 read 0x9173000 thru 0x917a3dc (0x73dd bytes) 2024-09-13T09:12:53.576 INFO:teuthology.orchestra.run.smithi157.stdout:2741 write 0x6d9fa00 thru 0x6da4e2c (0x542d bytes) 2024-09-13T09:12:53.577 INFO:teuthology.orchestra.run.smithi157.stdout:2742 read 0x559e600 thru 0x55ac65c (0xe05d bytes) 2024-09-13T09:12:53.577 INFO:teuthology.orchestra.run.smithi157.stdout:2744 read 0x1db6400 thru 0x1dc31ba (0xcdbb bytes) 2024-09-13T09:12:53.578 INFO:teuthology.orchestra.run.smithi157.stdout:2745 read 0x6737c00 thru 0x673e017 (0x6418 bytes) 2024-09-13T09:12:53.578 INFO:teuthology.orchestra.run.smithi157.stdout:2746 punch from 0x6b55800 to 0x6b5c800, (0x7000 bytes) 2024-09-13T09:12:53.580 INFO:teuthology.orchestra.run.smithi157.stdout:2747 trunc from 0xbfd4600 to 0xab4aa00 2024-09-13T09:12:53.585 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:53.585 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:53.592 INFO:teuthology.orchestra.run.smithi157.stdout:2750 write 0x11ec600 thru 0x11f09f1 (0x43f2 bytes) 2024-09-13T09:12:53.599 INFO:teuthology.orchestra.run.smithi157.stdout:2751 read 0xc32400 thru 0xc34983 (0x2584 bytes) 2024-09-13T09:12:53.599 INFO:teuthology.orchestra.run.smithi157.stdout:2752 read 0x1a30e00 thru 0x1a32f46 (0x2147 bytes) 2024-09-13T09:12:53.600 INFO:teuthology.orchestra.run.smithi157.stdout:2753 punch from 0xa3e6a00 to 0xa3e7800, (0xe00 bytes) 2024-09-13T09:12:53.600 INFO:teuthology.orchestra.run.smithi157.stdout:2754 punch from 0x8920e00 to 0x892f200, (0xe400 bytes) 2024-09-13T09:12:53.600 INFO:teuthology.orchestra.run.smithi157.stdout:2756 read 0x9589400 thru 0x9595de3 (0xc9e4 bytes) 2024-09-13T09:12:53.601 INFO:teuthology.orchestra.run.smithi157.stdout:2758 read 0x2124400 thru 0x2126a1f (0x2620 bytes) 2024-09-13T09:12:53.605 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:53.606 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:53.616 INFO:teuthology.orchestra.run.smithi157.stdout:2761 write 0xb859c00 thru 0xb86763f (0xda40 bytes) 2024-09-13T09:12:53.623 INFO:teuthology.orchestra.run.smithi157.stdout:2762 read 0x2819800 thru 0x2820ca2 (0x74a3 bytes) 2024-09-13T09:12:53.624 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:53.624 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:53.644 INFO:teuthology.orchestra.run.smithi157.stdout:2763 write 0xcbbfe00 thru 0xcbc3e95 (0x4096 bytes) 2024-09-13T09:12:53.644 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:53.644 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:53.645 INFO:teuthology.orchestra.run.smithi157.stdout:2767 read 0x103200 thru 0x1052f9 (0x20fa bytes) 2024-09-13T09:12:53.646 INFO:teuthology.orchestra.run.smithi157.stdout:2769 clone 18 order 22 su 16384 sc 14 2024-09-13T09:12:53.657 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:53.656+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:53.663 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:53.664 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:53.664 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:53.664 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:53.674 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:53.681 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:53.681 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:53.736 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:12:53.737 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:12:53.748 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-13T09:12:53.525567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633340482,"num_objects":575,"num_object_clones":187,"num_object_copies":1150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":575,"num_whiteouts":170,"num_read":106284,"num_read_kb":423243,"num_write":8607,"num_write_kb":66364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":9920327,"num_keys_recovered":33,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7280,"ondisk_log_size":7280,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":274608,"kb_used_data":51060,"kb_used_omap":9,"kb_used_meta":223478,"kb_avail":565956432,"statfs":{"total":579820584960,"available":579539386368,"internally_reserved":0,"allocated":52285440,"data_stored":54332610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9563,"internal_metadata":228842149},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50085837,"num_objects":241,"num_object_clones":119,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":134,"num_read":22792,"num_read_kb":49871,"num_write":1907,"num_write_kb":9905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000569"},"pg_stats":[{"pgid":"3.1f","version":"55'128","reported_seq":1367,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.410299+0000","last_change":"2024-09-13T09:12:43.692870+0000","last_active":"2024-09-13T09:12:51.410299+0000","last_peered":"2024-09-13T09:12:51.410299+0000","last_clean":"2024-09-13T09:12:51.410299+0000","last_became_active":"2024-09-13T09:12:43.534990+0000","last_became_peered":"2024-09-13T09:12:43.534990+0000","last_unstale":"2024-09-13T09:12:51.410299+0000","last_undegraded":"2024-09-13T09:12:51.410299+0000","last_fullsized":"2024-09-13T09:12:51.410299+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:45.810610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058630900000000003,"stat_sum":{"num_bytes":5467152,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":1740,"num_read_kb":3462,"num_write":108,"num_write_kb":1463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":848912,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"55'123","reported_seq":1144,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.426111+0000","last_change":"2024-09-13T09:12:43.652049+0000","last_active":"2024-09-13T09:12:51.426111+0000","last_peered":"2024-09-13T09:12:51.426111+0000","last_clean":"2024-09-13T09:12:51.426111+0000","last_became_active":"2024-09-13T09:12:43.532920+0000","last_became_peered":"2024-09-13T09:12:43.532920+0000","last_unstale":"2024-09-13T09:12:51.426111+0000","last_undegraded":"2024-09-13T09:12:51.426111+0000","last_fullsized":"2024-09-13T09:12:51.426111+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:10:38.824649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088145900000000002,"stat_sum":{"num_bytes":22193680,"num_objects":22,"num_object_clones":9,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":1265,"num_read_kb":8461,"num_write":114,"num_write_kb":1256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":311808,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"55'667","reported_seq":3067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.376286+0000","last_change":"2024-09-13T09:12:16.388369+0000","last_active":"2024-09-13T09:12:51.376286+0000","last_peered":"2024-09-13T09:12:51.376286+0000","last_clean":"2024-09-13T09:12:51.376286+0000","last_became_active":"2024-09-13T09:12:13.362666+0000","last_became_peered":"2024-09-13T09:12:13.362666+0000","last_unstale":"2024-09-13T09:12:51.376286+0000","last_undegraded":"2024-09-13T09:12:51.376286+0000","last_fullsized":"2024-09-13T09:12:51.376286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:04.594826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039115800000000001,"stat_sum":{"num_bytes":5156001,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":5170,"num_read_kb":5491,"num_write":1022,"num_write_kb":1942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"55'236","reported_seq":2254,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.501746+0000","last_change":"2024-09-13T09:12:16.388972+0000","last_active":"2024-09-13T09:12:51.501746+0000","last_peered":"2024-09-13T09:12:51.501746+0000","last_clean":"2024-09-13T09:12:51.501746+0000","last_became_active":"2024-09-13T09:12:13.364622+0000","last_became_peered":"2024-09-13T09:12:13.364622+0000","last_unstale":"2024-09-13T09:12:51.501746+0000","last_undegraded":"2024-09-13T09:12:51.501746+0000","last_fullsized":"2024-09-13T09:12:51.501746+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:34:20.301837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088725000000000002,"stat_sum":{"num_bytes":26517606,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":3525,"num_read_kb":9339,"num_write":193,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"53'219","reported_seq":4733,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.292669+0000","last_change":"2024-09-13T09:12:16.380908+0000","last_active":"2024-09-13T09:12:51.292669+0000","last_peered":"2024-09-13T09:12:51.292669+0000","last_clean":"2024-09-13T09:12:51.292669+0000","last_became_active":"2024-09-13T09:12:13.368138+0000","last_became_peered":"2024-09-13T09:12:13.368138+0000","last_unstale":"2024-09-13T09:12:51.292669+0000","last_undegraded":"2024-09-13T09:12:51.292669+0000","last_fullsized":"2024-09-13T09:12:51.292669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:10.836108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082447100000000001,"stat_sum":{"num_bytes":35081288,"num_objects":29,"num_object_clones":10,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":4815,"num_read_kb":36300,"num_write":282,"num_write_kb":5612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"54'199","reported_seq":5493,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.472303+0000","last_change":"2024-09-13T09:12:16.380731+0000","last_active":"2024-09-13T09:12:51.472303+0000","last_peered":"2024-09-13T09:12:51.472303+0000","last_clean":"2024-09-13T09:12:51.472303+0000","last_became_active":"2024-09-13T09:12:13.362533+0000","last_became_peered":"2024-09-13T09:12:13.362533+0000","last_unstale":"2024-09-13T09:12:51.472303+0000","last_undegraded":"2024-09-13T09:12:51.472303+0000","last_fullsized":"2024-09-13T09:12:51.472303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:50.399315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072061100000000002,"stat_sum":{"num_bytes":23726627,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":6211,"num_read_kb":21824,"num_write":156,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"55'154","reported_seq":4819,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.430700+0000","last_change":"2024-09-13T09:12:16.376447+0000","last_active":"2024-09-13T09:12:51.430700+0000","last_peered":"2024-09-13T09:12:51.430700+0000","last_clean":"2024-09-13T09:12:51.430700+0000","last_became_active":"2024-09-13T09:12:13.368221+0000","last_became_peered":"2024-09-13T09:12:13.368221+0000","last_unstale":"2024-09-13T09:12:51.430700+0000","last_undegraded":"2024-09-13T09:12:51.430700+0000","last_fullsized":"2024-09-13T09:12:51.430700+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:10.966835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000740483,"stat_sum":{"num_bytes":51515443,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":4839,"num_read_kb":9610,"num_write":176,"num_write_kb":1495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"55'146","reported_seq":1280,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.418730+0000","last_change":"2024-09-13T09:12:16.377490+0000","last_active":"2024-09-13T09:12:51.418730+0000","last_peered":"2024-09-13T09:12:51.418730+0000","last_clean":"2024-09-13T09:12:51.418730+0000","last_became_active":"2024-09-13T09:12:13.367773+0000","last_became_peered":"2024-09-13T09:12:13.367773+0000","last_unstale":"2024-09-13T09:12:51.418730+0000","last_undegraded":"2024-09-13T09:12:51.418730+0000","last_fullsized":"2024-09-13T09:12:51.418730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:34.023964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000479879,"stat_sum":{"num_bytes":3067904,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":6,"num_read":2269,"num_read_kb":3413,"num_write":111,"num_write_kb":675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"53'204","reported_seq":2410,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.483336+0000","last_change":"2024-09-13T09:12:16.381209+0000","last_active":"2024-09-13T09:12:51.483336+0000","last_peered":"2024-09-13T09:12:51.483336+0000","last_clean":"2024-09-13T09:12:51.483336+0000","last_became_active":"2024-09-13T09:12:13.365820+0000","last_became_peered":"2024-09-13T09:12:13.365820+0000","last_unstale":"2024-09-13T09:12:51.483336+0000","last_undegraded":"2024-09-13T09:12:51.483336+0000","last_fullsized":"2024-09-13T09:12:51.483336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:48.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097766600000000008,"stat_sum":{"num_bytes":11883008,"num_objects":16,"num_object_clones":6,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":3062,"num_read_kb":16784,"num_write":211,"num_write_kb":5119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"55'317","reported_seq":2341,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.468478+0000","last_change":"2024-09-13T09:12:46.251509+0000","last_active":"2024-09-13T09:12:51.468478+0000","last_peered":"2024-09-13T09:12:51.468478+0000","last_clean":"2024-09-13T09:12:51.468478+0000","last_became_active":"2024-09-13T09:12:46.250870+0000","last_became_peered":"2024-09-13T09:12:46.250870+0000","last_unstale":"2024-09-13T09:12:51.468478+0000","last_undegraded":"2024-09-13T09:12:51.468478+0000","last_fullsized":"2024-09-13T09:12:51.468478+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:33.138405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040009000000000003,"stat_sum":{"num_bytes":22262387,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":2897,"num_read_kb":13342,"num_write":455,"num_write_kb":1653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":706121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"55'568","reported_seq":5808,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.410321+0000","last_change":"2024-09-13T09:12:46.255856+0000","last_active":"2024-09-13T09:12:51.410321+0000","last_peered":"2024-09-13T09:12:51.410321+0000","last_clean":"2024-09-13T09:12:51.410321+0000","last_became_active":"2024-09-13T09:12:46.250696+0000","last_became_peered":"2024-09-13T09:12:46.250696+0000","last_unstale":"2024-09-13T09:12:51.410321+0000","last_undegraded":"2024-09-13T09:12:51.410321+0000","last_fullsized":"2024-09-13T09:12:51.410321+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:43.821997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172600000000004,"stat_sum":{"num_bytes":16437285,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":7793,"num_read_kb":20827,"num_write":998,"num_write_kb":2875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1136450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"55'255","reported_seq":3209,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.411006+0000","last_change":"2024-09-13T09:12:46.254606+0000","last_active":"2024-09-13T09:12:51.411006+0000","last_peered":"2024-09-13T09:12:51.411006+0000","last_clean":"2024-09-13T09:12:51.411006+0000","last_became_active":"2024-09-13T09:12:46.254016+0000","last_became_peered":"2024-09-13T09:12:46.254016+0000","last_unstale":"2024-09-13T09:12:51.411006+0000","last_undegraded":"2024-09-13T09:12:51.411006+0000","last_fullsized":"2024-09-13T09:12:51.411006+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:50.193567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074689299999999999,"stat_sum":{"num_bytes":45383291,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":8,"num_read":3516,"num_read_kb":17232,"num_write":210,"num_write_kb":2007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1110608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.288095+0000","last_change":"2024-09-13T09:12:43.531552+0000","last_active":"2024-09-13T09:12:50.288095+0000","last_peered":"2024-09-13T09:12:50.288095+0000","last_clean":"2024-09-13T09:12:50.288095+0000","last_became_active":"2024-09-13T09:12:43.531274+0000","last_became_peered":"2024-09-13T09:12:43.531274+0000","last_unstale":"2024-09-13T09:12:50.288095+0000","last_undegraded":"2024-09-13T09:12:50.288095+0000","last_fullsized":"2024-09-13T09:12:50.288095+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:41:06.992512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"55'412","reported_seq":3597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.502909+0000","last_change":"2024-09-13T09:12:16.380593+0000","last_active":"2024-09-13T09:12:51.502909+0000","last_peered":"2024-09-13T09:12:51.502909+0000","last_clean":"2024-09-13T09:12:51.502909+0000","last_became_active":"2024-09-13T09:12:13.365496+0000","last_became_peered":"2024-09-13T09:12:13.365496+0000","last_unstale":"2024-09-13T09:12:51.502909+0000","last_undegraded":"2024-09-13T09:12:51.502909+0000","last_fullsized":"2024-09-13T09:12:51.502909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:29:49.115977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061775199999999995,"stat_sum":{"num_bytes":18079886,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":1,"num_read":4513,"num_read_kb":35473,"num_write":636,"num_write_kb":3106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.284411+0000","last_change":"2024-09-13T09:12:43.531266+0000","last_active":"2024-09-13T09:12:50.284411+0000","last_peered":"2024-09-13T09:12:50.284411+0000","last_clean":"2024-09-13T09:12:50.284411+0000","last_became_active":"2024-09-13T09:12:43.530941+0000","last_became_peered":"2024-09-13T09:12:43.530941+0000","last_unstale":"2024-09-13T09:12:50.284411+0000","last_undegraded":"2024-09-13T09:12:50.284411+0000","last_fullsized":"2024-09-13T09:12:50.284411+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:27.261467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"55'397","reported_seq":5388,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.420650+0000","last_change":"2024-09-13T09:12:16.378483+0000","last_active":"2024-09-13T09:12:51.420650+0000","last_peered":"2024-09-13T09:12:51.420650+0000","last_clean":"2024-09-13T09:12:51.420650+0000","last_became_active":"2024-09-13T09:12:13.367146+0000","last_became_peered":"2024-09-13T09:12:13.367146+0000","last_unstale":"2024-09-13T09:12:51.420650+0000","last_undegraded":"2024-09-13T09:12:51.420650+0000","last_fullsized":"2024-09-13T09:12:51.420650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:49.370653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032368300000000001,"stat_sum":{"num_bytes":11395072,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":8164,"num_read_kb":23386,"num_write":300,"num_write_kb":3250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.287934+0000","last_change":"2024-09-13T09:12:43.533829+0000","last_active":"2024-09-13T09:12:50.287934+0000","last_peered":"2024-09-13T09:12:50.287934+0000","last_clean":"2024-09-13T09:12:50.287934+0000","last_became_active":"2024-09-13T09:12:43.533557+0000","last_became_peered":"2024-09-13T09:12:43.533557+0000","last_unstale":"2024-09-13T09:12:50.287934+0000","last_undegraded":"2024-09-13T09:12:50.287934+0000","last_fullsized":"2024-09-13T09:12:50.287934+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:51.939164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"55'146","reported_seq":2517,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.419981+0000","last_change":"2024-09-13T09:12:16.376528+0000","last_active":"2024-09-13T09:12:51.419981+0000","last_peered":"2024-09-13T09:12:51.419981+0000","last_clean":"2024-09-13T09:12:51.419981+0000","last_became_active":"2024-09-13T09:12:13.367848+0000","last_became_peered":"2024-09-13T09:12:13.367848+0000","last_unstale":"2024-09-13T09:12:51.419981+0000","last_undegraded":"2024-09-13T09:12:51.419981+0000","last_fullsized":"2024-09-13T09:12:51.419981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:21.943266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067051299999999997,"stat_sum":{"num_bytes":39939584,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":3045,"num_read_kb":11648,"num_write":127,"num_write_kb":1170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.284658+0000","last_change":"2024-09-13T09:12:06.328091+0000","last_active":"2024-09-13T09:12:50.284658+0000","last_peered":"2024-09-13T09:12:50.284658+0000","last_clean":"2024-09-13T09:12:50.284658+0000","last_became_active":"2024-09-13T09:12:04.446508+0000","last_became_peered":"2024-09-13T09:12:04.446508+0000","last_unstale":"2024-09-13T09:12:50.284658+0000","last_undegraded":"2024-09-13T09:12:50.284658+0000","last_fullsized":"2024-09-13T09:12:50.284658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:33.410873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042514000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"55'250","reported_seq":610,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.420742+0000","last_change":"2024-09-13T09:12:16.389006+0000","last_active":"2024-09-13T09:12:51.420742+0000","last_peered":"2024-09-13T09:12:51.420742+0000","last_clean":"2024-09-13T09:12:51.420742+0000","last_became_active":"2024-09-13T09:12:13.369489+0000","last_became_peered":"2024-09-13T09:12:13.369489+0000","last_unstale":"2024-09-13T09:12:51.420742+0000","last_undegraded":"2024-09-13T09:12:51.420742+0000","last_fullsized":"2024-09-13T09:12:51.420742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:33.879162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088595999999999998,"stat_sum":{"num_bytes":3100815,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":1210,"num_read_kb":2374,"num_write":369,"num_write_kb":1172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"55'171","reported_seq":4758,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.511548+0000","last_change":"2024-09-13T09:12:16.380981+0000","last_active":"2024-09-13T09:12:51.511548+0000","last_peered":"2024-09-13T09:12:51.511548+0000","last_clean":"2024-09-13T09:12:51.511548+0000","last_became_active":"2024-09-13T09:12:13.367770+0000","last_became_peered":"2024-09-13T09:12:13.367770+0000","last_unstale":"2024-09-13T09:12:51.511548+0000","last_undegraded":"2024-09-13T09:12:51.511548+0000","last_fullsized":"2024-09-13T09:12:51.511548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:21:50.690425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083150800000000001,"stat_sum":{"num_bytes":24312100,"num_objects":32,"num_object_clones":12,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":4602,"num_read_kb":24088,"num_write":179,"num_write_kb":2391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":54,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.303125+0000","last_change":"2024-09-13T09:12:06.330649+0000","last_active":"2024-09-13T09:12:50.303125+0000","last_peered":"2024-09-13T09:12:50.303125+0000","last_clean":"2024-09-13T09:12:50.303125+0000","last_became_active":"2024-09-13T09:12:04.325470+0000","last_became_peered":"2024-09-13T09:12:04.325470+0000","last_unstale":"2024-09-13T09:12:50.303125+0000","last_undegraded":"2024-09-13T09:12:50.303125+0000","last_fullsized":"2024-09-13T09:12:50.303125+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:37.971615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093634899999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"55'144","reported_seq":1559,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.426975+0000","last_change":"2024-09-13T09:12:16.378750+0000","last_active":"2024-09-13T09:12:51.426975+0000","last_peered":"2024-09-13T09:12:51.426975+0000","last_clean":"2024-09-13T09:12:51.426975+0000","last_became_active":"2024-09-13T09:12:13.367656+0000","last_became_peered":"2024-09-13T09:12:13.367656+0000","last_unstale":"2024-09-13T09:12:51.426975+0000","last_undegraded":"2024-09-13T09:12:51.426975+0000","last_fullsized":"2024-09-13T09:12:51.426975+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:11.942294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003044836,"stat_sum":{"num_bytes":15526944,"num_objects":24,"num_object_clones":9,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":1415,"num_read_kb":14662,"num_write":136,"num_write_kb":1483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.284477+0000","last_change":"2024-09-13T09:12:06.845125+0000","last_active":"2024-09-13T09:12:50.284477+0000","last_peered":"2024-09-13T09:12:50.284477+0000","last_clean":"2024-09-13T09:12:50.284477+0000","last_became_active":"2024-09-13T09:12:04.321941+0000","last_became_peered":"2024-09-13T09:12:04.321941+0000","last_unstale":"2024-09-13T09:12:50.284477+0000","last_undegraded":"2024-09-13T09:12:50.284477+0000","last_fullsized":"2024-09-13T09:12:50.284477+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:57.706060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031569600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"55'98","reported_seq":1225,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.416394+0000","last_change":"2024-09-13T09:12:16.380222+0000","last_active":"2024-09-13T09:12:51.416394+0000","last_peered":"2024-09-13T09:12:51.416394+0000","last_clean":"2024-09-13T09:12:51.416394+0000","last_became_active":"2024-09-13T09:12:13.367633+0000","last_became_peered":"2024-09-13T09:12:13.367633+0000","last_unstale":"2024-09-13T09:12:51.416394+0000","last_undegraded":"2024-09-13T09:12:51.416394+0000","last_fullsized":"2024-09-13T09:12:51.416394+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:34.734632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438009,"stat_sum":{"num_bytes":6182912,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":1094,"num_read_kb":8707,"num_write":97,"num_write_kb":968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.287845+0000","last_change":"2024-09-13T09:12:43.532201+0000","last_active":"2024-09-13T09:12:50.287845+0000","last_peered":"2024-09-13T09:12:50.287845+0000","last_clean":"2024-09-13T09:12:50.287845+0000","last_became_active":"2024-09-13T09:12:43.531942+0000","last_became_peered":"2024-09-13T09:12:43.531942+0000","last_unstale":"2024-09-13T09:12:50.287845+0000","last_undegraded":"2024-09-13T09:12:50.287845+0000","last_fullsized":"2024-09-13T09:12:50.287845+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:26.408883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"55'232","reported_seq":1010,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.409444+0000","last_change":"2024-09-13T09:12:16.377544+0000","last_active":"2024-09-13T09:12:51.409444+0000","last_peered":"2024-09-13T09:12:51.409444+0000","last_clean":"2024-09-13T09:12:51.409444+0000","last_became_active":"2024-09-13T09:12:13.361909+0000","last_became_peered":"2024-09-13T09:12:13.361909+0000","last_unstale":"2024-09-13T09:12:51.409444+0000","last_undegraded":"2024-09-13T09:12:51.409444+0000","last_fullsized":"2024-09-13T09:12:51.409444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:34.570438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001546782,"stat_sum":{"num_bytes":3548775,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":1539,"num_read_kb":3778,"num_write":342,"num_write_kb":887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":53,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.303101+0000","last_change":"2024-09-13T09:12:06.330140+0000","last_active":"2024-09-13T09:12:50.303101+0000","last_peered":"2024-09-13T09:12:50.303101+0000","last_clean":"2024-09-13T09:12:50.303101+0000","last_became_active":"2024-09-13T09:12:04.324857+0000","last_became_peered":"2024-09-13T09:12:04.324857+0000","last_unstale":"2024-09-13T09:12:50.303101+0000","last_undegraded":"2024-09-13T09:12:50.303101+0000","last_fullsized":"2024-09-13T09:12:50.303101+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:57.400286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038911000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.288060+0000","last_change":"2024-09-13T09:12:46.248714+0000","last_active":"2024-09-13T09:12:50.288060+0000","last_peered":"2024-09-13T09:12:50.288060+0000","last_clean":"2024-09-13T09:12:50.288060+0000","last_became_active":"2024-09-13T09:12:46.248451+0000","last_became_peered":"2024-09-13T09:12:46.248451+0000","last_unstale":"2024-09-13T09:12:50.288060+0000","last_undegraded":"2024-09-13T09:12:50.288060+0000","last_fullsized":"2024-09-13T09:12:50.288060+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:27.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"55'104","reported_seq":610,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.421862+0000","last_change":"2024-09-13T09:12:16.377642+0000","last_active":"2024-09-13T09:12:51.421862+0000","last_peered":"2024-09-13T09:12:51.421862+0000","last_clean":"2024-09-13T09:12:51.421862+0000","last_became_active":"2024-09-13T09:12:13.369742+0000","last_became_peered":"2024-09-13T09:12:13.369742+0000","last_unstale":"2024-09-13T09:12:51.421862+0000","last_undegraded":"2024-09-13T09:12:51.421862+0000","last_fullsized":"2024-09-13T09:12:51.421862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:27:48.361252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060355399999999996,"stat_sum":{"num_bytes":19369518,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":586,"num_read_kb":3371,"num_write":113,"num_write_kb":2896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"55'329","reported_seq":6351,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.411749+0000","last_change":"2024-09-13T09:12:16.380989+0000","last_active":"2024-09-13T09:12:51.411749+0000","last_peered":"2024-09-13T09:12:51.411749+0000","last_clean":"2024-09-13T09:12:51.411749+0000","last_became_active":"2024-09-13T09:12:13.370096+0000","last_became_peered":"2024-09-13T09:12:13.370096+0000","last_unstale":"2024-09-13T09:12:51.411749+0000","last_undegraded":"2024-09-13T09:12:51.411749+0000","last_fullsized":"2024-09-13T09:12:51.411749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:14.224546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000903458,"stat_sum":{"num_bytes":39831040,"num_objects":22,"num_object_clones":9,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":8500,"num_read_kb":19304,"num_write":220,"num_write_kb":1089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"55'136","reported_seq":1746,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.427613+0000","last_change":"2024-09-13T09:12:46.253134+0000","last_active":"2024-09-13T09:12:51.427613+0000","last_peered":"2024-09-13T09:12:51.427613+0000","last_clean":"2024-09-13T09:12:51.427613+0000","last_became_active":"2024-09-13T09:12:46.252578+0000","last_became_peered":"2024-09-13T09:12:46.252578+0000","last_unstale":"2024-09-13T09:12:51.427613+0000","last_undegraded":"2024-09-13T09:12:51.427613+0000","last_fullsized":"2024-09-13T09:12:51.427613+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:03.931492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041459800000000002,"stat_sum":{"num_bytes":51840068,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":4,"num_read":1569,"num_read_kb":18162,"num_write":150,"num_write_kb":3781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2590736,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"55'101","reported_seq":2074,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.410222+0000","last_change":"2024-09-13T09:12:16.377864+0000","last_active":"2024-09-13T09:12:51.410222+0000","last_peered":"2024-09-13T09:12:51.410222+0000","last_clean":"2024-09-13T09:12:51.410222+0000","last_became_active":"2024-09-13T09:12:13.362409+0000","last_became_peered":"2024-09-13T09:12:13.362409+0000","last_unstale":"2024-09-13T09:12:51.410222+0000","last_undegraded":"2024-09-13T09:12:51.410222+0000","last_fullsized":"2024-09-13T09:12:51.410222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:37.023594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001713904,"stat_sum":{"num_bytes":40788512,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":1961,"num_read_kb":21606,"num_write":139,"num_write_kb":3597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"55'371","reported_seq":4130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.424863+0000","last_change":"2024-09-13T09:12:16.389470+0000","last_active":"2024-09-13T09:12:51.424863+0000","last_peered":"2024-09-13T09:12:51.424863+0000","last_clean":"2024-09-13T09:12:51.424863+0000","last_became_active":"2024-09-13T09:12:13.365758+0000","last_became_peered":"2024-09-13T09:12:13.365758+0000","last_unstale":"2024-09-13T09:12:51.424863+0000","last_undegraded":"2024-09-13T09:12:51.424863+0000","last_fullsized":"2024-09-13T09:12:51.424863+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:46.649879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013123500000000001,"stat_sum":{"num_bytes":39062602,"num_objects":25,"num_object_clones":10,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":5167,"num_read_kb":22878,"num_write":588,"num_write_kb":3406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"55'116","reported_seq":337,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.420419+0000","last_change":"2024-09-13T09:12:43.630169+0000","last_active":"2024-09-13T09:12:51.420419+0000","last_peered":"2024-09-13T09:12:51.420419+0000","last_clean":"2024-09-13T09:12:51.420419+0000","last_became_active":"2024-09-13T09:12:43.533321+0000","last_became_peered":"2024-09-13T09:12:43.533321+0000","last_unstale":"2024-09-13T09:12:51.420419+0000","last_undegraded":"2024-09-13T09:12:51.420419+0000","last_fullsized":"2024-09-13T09:12:51.420419+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:43.973948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000679308,"stat_sum":{"num_bytes":3810347,"num_objects":14,"num_object_clones":6,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":357,"num_read_kb":1959,"num_write":117,"num_write_kb":809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"55'110","reported_seq":1226,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.459723+0000","last_change":"2024-09-13T09:12:16.389243+0000","last_active":"2024-09-13T09:12:51.459723+0000","last_peered":"2024-09-13T09:12:51.459723+0000","last_clean":"2024-09-13T09:12:51.459723+0000","last_became_active":"2024-09-13T09:12:13.362547+0000","last_became_peered":"2024-09-13T09:12:13.362547+0000","last_unstale":"2024-09-13T09:12:51.459723+0000","last_undegraded":"2024-09-13T09:12:51.459723+0000","last_fullsized":"2024-09-13T09:12:51.459723+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:24:24.557400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011202860000000001,"stat_sum":{"num_bytes":2537984,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":1665,"num_read_kb":3468,"num_write":90,"num_write_kb":612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"53'112","reported_seq":827,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.303260+0000","last_change":"2024-09-13T09:12:43.623612+0000","last_active":"2024-09-13T09:12:50.303260+0000","last_peered":"2024-09-13T09:12:50.303260+0000","last_clean":"2024-09-13T09:12:50.303260+0000","last_became_active":"2024-09-13T09:12:43.533088+0000","last_became_peered":"2024-09-13T09:12:43.533088+0000","last_unstale":"2024-09-13T09:12:50.303260+0000","last_undegraded":"2024-09-13T09:12:50.303260+0000","last_fullsized":"2024-09-13T09:12:50.303260+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:43:49.678463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059255799999999997,"stat_sum":{"num_bytes":2678908,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":661,"num_read_kb":1263,"num_write":114,"num_write_kb":1329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":635004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"55'84","reported_seq":827,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.366624+0000","last_change":"2024-09-13T09:12:16.389277+0000","last_active":"2024-09-13T09:12:51.366624+0000","last_peered":"2024-09-13T09:12:51.366624+0000","last_clean":"2024-09-13T09:12:51.366624+0000","last_became_active":"2024-09-13T09:12:13.367757+0000","last_became_peered":"2024-09-13T09:12:13.367757+0000","last_unstale":"2024-09-13T09:12:51.366624+0000","last_undegraded":"2024-09-13T09:12:51.366624+0000","last_fullsized":"2024-09-13T09:12:51.366624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:33.596156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011263219999999999,"stat_sum":{"num_bytes":8565852,"num_objects":12,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":721,"num_read_kb":4898,"num_write":81,"num_write_kb":871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"53'125","reported_seq":434,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.880961+0000","last_change":"2024-09-13T09:12:43.614382+0000","last_active":"2024-09-13T09:12:50.880961+0000","last_peered":"2024-09-13T09:12:50.880961+0000","last_clean":"2024-09-13T09:12:50.880961+0000","last_became_active":"2024-09-13T09:12:43.532504+0000","last_became_peered":"2024-09-13T09:12:43.532504+0000","last_unstale":"2024-09-13T09:12:50.880961+0000","last_undegraded":"2024-09-13T09:12:50.880961+0000","last_fullsized":"2024-09-13T09:12:50.880961+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:25:04.225563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034391499999999998,"stat_sum":{"num_bytes":11360807,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":928,"num_read_kb":11657,"num_write":129,"num_write_kb":1596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":694311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"55'167","reported_seq":7571,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.428666+0000","last_change":"2024-09-13T09:12:46.248982+0000","last_active":"2024-09-13T09:12:51.428666+0000","last_peered":"2024-09-13T09:12:51.428666+0000","last_clean":"2024-09-13T09:12:51.428666+0000","last_became_active":"2024-09-13T09:12:46.248261+0000","last_became_peered":"2024-09-13T09:12:46.248261+0000","last_unstale":"2024-09-13T09:12:51.428666+0000","last_undegraded":"2024-09-13T09:12:51.428666+0000","last_fullsized":"2024-09-13T09:12:51.428666+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:44.056762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049261300000000004,"stat_sum":{"num_bytes":18003968,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":7227,"num_read_kb":19668,"num_write":166,"num_write_kb":1674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":676864,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"55'339","reported_seq":3933,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.478669+0000","last_change":"2024-09-13T09:12:46.251913+0000","last_active":"2024-09-13T09:12:51.478669+0000","last_peered":"2024-09-13T09:12:51.478669+0000","last_clean":"2024-09-13T09:12:51.478669+0000","last_became_active":"2024-09-13T09:12:46.250552+0000","last_became_peered":"2024-09-13T09:12:46.250552+0000","last_unstale":"2024-09-13T09:12:51.478669+0000","last_undegraded":"2024-09-13T09:12:51.478669+0000","last_fullsized":"2024-09-13T09:12:51.478669+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:47.142721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059370899999999999,"stat_sum":{"num_bytes":4122729,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":4142,"num_read_kb":4708,"num_write":317,"num_write_kb":945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":28777,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":632750095,"num_objects":571,"num_object_clones":187,"num_object_copies":1142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":571,"num_whiteouts":170,"num_read":106168,"num_read_kb":423143,"num_write":8446,"num_write_kb":63946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":8739591,"num_keys_recovered":33,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":47992832,"data_stored":52231198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7160,"ondisk_log_size":7160,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640270,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45904,"kb_used_data":6600,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94325936,"statfs":{"total":96636764160,"available":96589758464,"internally_reserved":0,"allocated":6758400,"data_stored":6708214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]}]},{"osd":4,"up_from":11,"seq":47244640271,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":10764,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":11022336,"data_stored":12048839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640272,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43760,"kb_used_data":8552,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94328080,"statfs":{"total":96636764160,"available":96591953920,"internally_reserved":0,"allocated":8757248,"data_stored":9241505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640270,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39780,"kb_used_data":4572,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332060,"statfs":{"total":96636764160,"available":96596029440,"internally_reserved":0,"allocated":4681728,"data_stored":4587364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]}]},{"osd":1,"up_from":11,"seq":47244640271,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42324,"kb_used_data":7116,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94329516,"statfs":{"total":96636764160,"available":96593424384,"internally_reserved":0,"allocated":7286784,"data_stored":7663640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36043205},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]}]},{"osd":0,"up_from":11,"seq":47244640270,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52768,"kb_used_data":13456,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94319072,"statfs":{"total":96636764160,"available":96582729728,"internally_reserved":0,"allocated":13778944,"data_stored":14083048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12562432,"data_stored":13246709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6717440,"data_stored":7461305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4456216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7540736,"data_stored":8405166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10620928,"data_stored":12005807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6356992,"data_stored":6655995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:12:53.751 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:12:53.758 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:53.758+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:53.782 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:53.783 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:53.792 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:53.859 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:53.858+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:53.896 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:53.897 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:53.905 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:53.990 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:53.989+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:54.036 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:12:54.037 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:12:54.047 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:54.048 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:54.048 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-13T09:12:53.525567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633340482,"num_objects":575,"num_object_clones":187,"num_object_copies":1150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":575,"num_whiteouts":170,"num_read":106284,"num_read_kb":423243,"num_write":8607,"num_write_kb":66364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":9920327,"num_keys_recovered":33,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7280,"ondisk_log_size":7280,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":274608,"kb_used_data":51060,"kb_used_omap":9,"kb_used_meta":223478,"kb_avail":565956432,"statfs":{"total":579820584960,"available":579539386368,"internally_reserved":0,"allocated":52285440,"data_stored":54332610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9563,"internal_metadata":228842149},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50085837,"num_objects":241,"num_object_clones":119,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":134,"num_read":22792,"num_read_kb":49871,"num_write":1907,"num_write_kb":9905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000569"},"pg_stats":[{"pgid":"3.1f","version":"55'128","reported_seq":1367,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.410299+0000","last_change":"2024-09-13T09:12:43.692870+0000","last_active":"2024-09-13T09:12:51.410299+0000","last_peered":"2024-09-13T09:12:51.410299+0000","last_clean":"2024-09-13T09:12:51.410299+0000","last_became_active":"2024-09-13T09:12:43.534990+0000","last_became_peered":"2024-09-13T09:12:43.534990+0000","last_unstale":"2024-09-13T09:12:51.410299+0000","last_undegraded":"2024-09-13T09:12:51.410299+0000","last_fullsized":"2024-09-13T09:12:51.410299+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:45.810610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058630900000000003,"stat_sum":{"num_bytes":5467152,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":1740,"num_read_kb":3462,"num_write":108,"num_write_kb":1463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":848912,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"55'123","reported_seq":1144,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.426111+0000","last_change":"2024-09-13T09:12:43.652049+0000","last_active":"2024-09-13T09:12:51.426111+0000","last_peered":"2024-09-13T09:12:51.426111+0000","last_clean":"2024-09-13T09:12:51.426111+0000","last_became_active":"2024-09-13T09:12:43.532920+0000","last_became_peered":"2024-09-13T09:12:43.532920+0000","last_unstale":"2024-09-13T09:12:51.426111+0000","last_undegraded":"2024-09-13T09:12:51.426111+0000","last_fullsized":"2024-09-13T09:12:51.426111+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:10:38.824649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088145900000000002,"stat_sum":{"num_bytes":22193680,"num_objects":22,"num_object_clones":9,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":1265,"num_read_kb":8461,"num_write":114,"num_write_kb":1256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":311808,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"55'667","reported_seq":3067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.376286+0000","last_change":"2024-09-13T09:12:16.388369+0000","last_active":"2024-09-13T09:12:51.376286+0000","last_peered":"2024-09-13T09:12:51.376286+0000","last_clean":"2024-09-13T09:12:51.376286+0000","last_became_active":"2024-09-13T09:12:13.362666+0000","last_became_peered":"2024-09-13T09:12:13.362666+0000","last_unstale":"2024-09-13T09:12:51.376286+0000","last_undegraded":"2024-09-13T09:12:51.376286+0000","last_fullsized":"2024-09-13T09:12:51.376286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:04.594826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039115800000000001,"stat_sum":{"num_bytes":5156001,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":5170,"num_read_kb":5491,"num_write":1022,"num_write_kb":1942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"55'236","reported_seq":2254,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.501746+0000","last_change":"2024-09-13T09:12:16.388972+0000","last_active":"2024-09-13T09:12:51.501746+0000","last_peered":"2024-09-13T09:12:51.501746+0000","last_clean":"2024-09-13T09:12:51.501746+0000","last_became_active":"2024-09-13T09:12:13.364622+0000","last_became_peered":"2024-09-13T09:12:13.364622+0000","last_unstale":"2024-09-13T09:12:51.501746+0000","last_undegraded":"2024-09-13T09:12:51.501746+0000","last_fullsized":"2024-09-13T09:12:51.501746+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:34:20.301837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088725000000000002,"stat_sum":{"num_bytes":26517606,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":3525,"num_read_kb":9339,"num_write":193,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"53'219","reported_seq":4733,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.292669+0000","last_change":"2024-09-13T09:12:16.380908+0000","last_active":"2024-09-13T09:12:51.292669+0000","last_peered":"2024-09-13T09:12:51.292669+0000","last_clean":"2024-09-13T09:12:51.292669+0000","last_became_active":"2024-09-13T09:12:13.368138+0000","last_became_peered":"2024-09-13T09:12:13.368138+0000","last_unstale":"2024-09-13T09:12:51.292669+0000","last_undegraded":"2024-09-13T09:12:51.292669+0000","last_fullsized":"2024-09-13T09:12:51.292669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:10.836108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082447100000000001,"stat_sum":{"num_bytes":35081288,"num_objects":29,"num_object_clones":10,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":4815,"num_read_kb":36300,"num_write":282,"num_write_kb":5612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"54'199","reported_seq":5493,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.472303+0000","last_change":"2024-09-13T09:12:16.380731+0000","last_active":"2024-09-13T09:12:51.472303+0000","last_peered":"2024-09-13T09:12:51.472303+0000","last_clean":"2024-09-13T09:12:51.472303+0000","last_became_active":"2024-09-13T09:12:13.362533+0000","last_became_peered":"2024-09-13T09:12:13.362533+0000","last_unstale":"2024-09-13T09:12:51.472303+0000","last_undegraded":"2024-09-13T09:12:51.472303+0000","last_fullsized":"2024-09-13T09:12:51.472303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:50.399315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072061100000000002,"stat_sum":{"num_bytes":23726627,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":6211,"num_read_kb":21824,"num_write":156,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"55'154","reported_seq":4819,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.430700+0000","last_change":"2024-09-13T09:12:16.376447+0000","last_active":"2024-09-13T09:12:51.430700+0000","last_peered":"2024-09-13T09:12:51.430700+0000","last_clean":"2024-09-13T09:12:51.430700+0000","last_became_active":"2024-09-13T09:12:13.368221+0000","last_became_peered":"2024-09-13T09:12:13.368221+0000","last_unstale":"2024-09-13T09:12:51.430700+0000","last_undegraded":"2024-09-13T09:12:51.430700+0000","last_fullsized":"2024-09-13T09:12:51.430700+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:10.966835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000740483,"stat_sum":{"num_bytes":51515443,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":4839,"num_read_kb":9610,"num_write":176,"num_write_kb":1495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"55'146","reported_seq":1280,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.418730+0000","last_change":"2024-09-13T09:12:16.377490+0000","last_active":"2024-09-13T09:12:51.418730+0000","last_peered":"2024-09-13T09:12:51.418730+0000","last_clean":"2024-09-13T09:12:51.418730+0000","last_became_active":"2024-09-13T09:12:13.367773+0000","last_became_peered":"2024-09-13T09:12:13.367773+0000","last_unstale":"2024-09-13T09:12:51.418730+0000","last_undegraded":"2024-09-13T09:12:51.418730+0000","last_fullsized":"2024-09-13T09:12:51.418730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:34.023964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000479879,"stat_sum":{"num_bytes":3067904,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":6,"num_read":2269,"num_read_kb":3413,"num_write":111,"num_write_kb":675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"53'204","reported_seq":2410,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.483336+0000","last_change":"2024-09-13T09:12:16.381209+0000","last_active":"2024-09-13T09:12:51.483336+0000","last_peered":"2024-09-13T09:12:51.483336+0000","last_clean":"2024-09-13T09:12:51.483336+0000","last_became_active":"2024-09-13T09:12:13.365820+0000","last_became_peered":"2024-09-13T09:12:13.365820+0000","last_unstale":"2024-09-13T09:12:51.483336+0000","last_undegraded":"2024-09-13T09:12:51.483336+0000","last_fullsized":"2024-09-13T09:12:51.483336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:48.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097766600000000008,"stat_sum":{"num_bytes":11883008,"num_objects":16,"num_object_clones":6,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":3062,"num_read_kb":16784,"num_write":211,"num_write_kb":5119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"55'317","reported_seq":2341,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.468478+0000","last_change":"2024-09-13T09:12:46.251509+0000","last_active":"2024-09-13T09:12:51.468478+0000","last_peered":"2024-09-13T09:12:51.468478+0000","last_clean":"2024-09-13T09:12:51.468478+0000","last_became_active":"2024-09-13T09:12:46.250870+0000","last_became_peered":"2024-09-13T09:12:46.250870+0000","last_unstale":"2024-09-13T09:12:51.468478+0000","last_undegraded":"2024-09-13T09:12:51.468478+0000","last_fullsized":"2024-09-13T09:12:51.468478+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:33.138405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040009000000000003,"stat_sum":{"num_bytes":22262387,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":2897,"num_read_kb":13342,"num_write":455,"num_write_kb":1653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":706121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"55'568","reported_seq":5808,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.410321+0000","last_change":"2024-09-13T09:12:46.255856+0000","last_active":"2024-09-13T09:12:51.410321+0000","last_peered":"2024-09-13T09:12:51.410321+0000","last_clean":"2024-09-13T09:12:51.410321+0000","last_became_active":"2024-09-13T09:12:46.250696+0000","last_became_peered":"2024-09-13T09:12:46.250696+0000","last_unstale":"2024-09-13T09:12:51.410321+0000","last_undegraded":"2024-09-13T09:12:51.410321+0000","last_fullsized":"2024-09-13T09:12:51.410321+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:43.821997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172600000000004,"stat_sum":{"num_bytes":16437285,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":7793,"num_read_kb":20827,"num_write":998,"num_write_kb":2875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1136450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"55'255","reported_seq":3209,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.411006+0000","last_change":"2024-09-13T09:12:46.254606+0000","last_active":"2024-09-13T09:12:51.411006+0000","last_peered":"2024-09-13T09:12:51.411006+0000","last_clean":"2024-09-13T09:12:51.411006+0000","last_became_active":"2024-09-13T09:12:46.254016+0000","last_became_peered":"2024-09-13T09:12:46.254016+0000","last_unstale":"2024-09-13T09:12:51.411006+0000","last_undegraded":"2024-09-13T09:12:51.411006+0000","last_fullsized":"2024-09-13T09:12:51.411006+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:50.193567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074689299999999999,"stat_sum":{"num_bytes":45383291,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":8,"num_read":3516,"num_read_kb":17232,"num_write":210,"num_write_kb":2007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1110608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.288095+0000","last_change":"2024-09-13T09:12:43.531552+0000","last_active":"2024-09-13T09:12:50.288095+0000","last_peered":"2024-09-13T09:12:50.288095+0000","last_clean":"2024-09-13T09:12:50.288095+0000","last_became_active":"2024-09-13T09:12:43.531274+0000","last_became_peered":"2024-09-13T09:12:43.531274+0000","last_unstale":"2024-09-13T09:12:50.288095+0000","last_undegraded":"2024-09-13T09:12:50.288095+0000","last_fullsized":"2024-09-13T09:12:50.288095+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:41:06.992512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"55'412","reported_seq":3597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.502909+0000","last_change":"2024-09-13T09:12:16.380593+0000","last_active":"2024-09-13T09:12:51.502909+0000","last_peered":"2024-09-13T09:12:51.502909+0000","last_clean":"2024-09-13T09:12:51.502909+0000","last_became_active":"2024-09-13T09:12:13.365496+0000","last_became_peered":"2024-09-13T09:12:13.365496+0000","last_unstale":"2024-09-13T09:12:51.502909+0000","last_undegraded":"2024-09-13T09:12:51.502909+0000","last_fullsized":"2024-09-13T09:12:51.502909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:29:49.115977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061775199999999995,"stat_sum":{"num_bytes":18079886,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":1,"num_read":4513,"num_read_kb":35473,"num_write":636,"num_write_kb":3106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.284411+0000","last_change":"2024-09-13T09:12:43.531266+0000","last_active":"2024-09-13T09:12:50.284411+0000","last_peered":"2024-09-13T09:12:50.284411+0000","last_clean":"2024-09-13T09:12:50.284411+0000","last_became_active":"2024-09-13T09:12:43.530941+0000","last_became_peered":"2024-09-13T09:12:43.530941+0000","last_unstale":"2024-09-13T09:12:50.284411+0000","last_undegraded":"2024-09-13T09:12:50.284411+0000","last_fullsized":"2024-09-13T09:12:50.284411+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:27.261467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"55'397","reported_seq":5388,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.420650+0000","last_change":"2024-09-13T09:12:16.378483+0000","last_active":"2024-09-13T09:12:51.420650+0000","last_peered":"2024-09-13T09:12:51.420650+0000","last_clean":"2024-09-13T09:12:51.420650+0000","last_became_active":"2024-09-13T09:12:13.367146+0000","last_became_peered":"2024-09-13T09:12:13.367146+0000","last_unstale":"2024-09-13T09:12:51.420650+0000","last_undegraded":"2024-09-13T09:12:51.420650+0000","last_fullsized":"2024-09-13T09:12:51.420650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:49.370653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032368300000000001,"stat_sum":{"num_bytes":11395072,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":8164,"num_read_kb":23386,"num_write":300,"num_write_kb":3250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.287934+0000","last_change":"2024-09-13T09:12:43.533829+0000","last_active":"2024-09-13T09:12:50.287934+0000","last_peered":"2024-09-13T09:12:50.287934+0000","last_clean":"2024-09-13T09:12:50.287934+0000","last_became_active":"2024-09-13T09:12:43.533557+0000","last_became_peered":"2024-09-13T09:12:43.533557+0000","last_unstale":"2024-09-13T09:12:50.287934+0000","last_undegraded":"2024-09-13T09:12:50.287934+0000","last_fullsized":"2024-09-13T09:12:50.287934+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:51.939164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"55'146","reported_seq":2517,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.419981+0000","last_change":"2024-09-13T09:12:16.376528+0000","last_active":"2024-09-13T09:12:51.419981+0000","last_peered":"2024-09-13T09:12:51.419981+0000","last_clean":"2024-09-13T09:12:51.419981+0000","last_became_active":"2024-09-13T09:12:13.367848+0000","last_became_peered":"2024-09-13T09:12:13.367848+0000","last_unstale":"2024-09-13T09:12:51.419981+0000","last_undegraded":"2024-09-13T09:12:51.419981+0000","last_fullsized":"2024-09-13T09:12:51.419981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:21.943266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067051299999999997,"stat_sum":{"num_bytes":39939584,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":3045,"num_read_kb":11648,"num_write":127,"num_write_kb":1170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.284658+0000","last_change":"2024-09-13T09:12:06.328091+0000","last_active":"2024-09-13T09:12:50.284658+0000","last_peered":"2024-09-13T09:12:50.284658+0000","last_clean":"2024-09-13T09:12:50.284658+0000","last_became_active":"2024-09-13T09:12:04.446508+0000","last_became_peered":"2024-09-13T09:12:04.446508+0000","last_unstale":"2024-09-13T09:12:50.284658+0000","last_undegraded":"2024-09-13T09:12:50.284658+0000","last_fullsized":"2024-09-13T09:12:50.284658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:33.410873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042514000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"55'250","reported_seq":610,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.420742+0000","last_change":"2024-09-13T09:12:16.389006+0000","last_active":"2024-09-13T09:12:51.420742+0000","last_peered":"2024-09-13T09:12:51.420742+0000","last_clean":"2024-09-13T09:12:51.420742+0000","last_became_active":"2024-09-13T09:12:13.369489+0000","last_became_peered":"2024-09-13T09:12:13.369489+0000","last_unstale":"2024-09-13T09:12:51.420742+0000","last_undegraded":"2024-09-13T09:12:51.420742+0000","last_fullsized":"2024-09-13T09:12:51.420742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:33.879162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088595999999999998,"stat_sum":{"num_bytes":3100815,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":1210,"num_read_kb":2374,"num_write":369,"num_write_kb":1172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"55'171","reported_seq":4758,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.511548+0000","last_change":"2024-09-13T09:12:16.380981+0000","last_active":"2024-09-13T09:12:51.511548+0000","last_peered":"2024-09-13T09:12:51.511548+0000","last_clean":"2024-09-13T09:12:51.511548+0000","last_became_active":"2024-09-13T09:12:13.367770+0000","last_became_peered":"2024-09-13T09:12:13.367770+0000","last_unstale":"2024-09-13T09:12:51.511548+0000","last_undegraded":"2024-09-13T09:12:51.511548+0000","last_fullsized":"2024-09-13T09:12:51.511548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:21:50.690425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083150800000000001,"stat_sum":{"num_bytes":24312100,"num_objects":32,"num_object_clones":12,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":4602,"num_read_kb":24088,"num_write":179,"num_write_kb":2391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":54,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.303125+0000","last_change":"2024-09-13T09:12:06.330649+0000","last_active":"2024-09-13T09:12:50.303125+0000","last_peered":"2024-09-13T09:12:50.303125+0000","last_clean":"2024-09-13T09:12:50.303125+0000","last_became_active":"2024-09-13T09:12:04.325470+0000","last_became_peered":"2024-09-13T09:12:04.325470+0000","last_unstale":"2024-09-13T09:12:50.303125+0000","last_undegraded":"2024-09-13T09:12:50.303125+0000","last_fullsized":"2024-09-13T09:12:50.303125+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:37.971615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093634899999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"55'144","reported_seq":1559,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.426975+0000","last_change":"2024-09-13T09:12:16.378750+0000","last_active":"2024-09-13T09:12:51.426975+0000","last_peered":"2024-09-13T09:12:51.426975+0000","last_clean":"2024-09-13T09:12:51.426975+0000","last_became_active":"2024-09-13T09:12:13.367656+0000","last_became_peered":"2024-09-13T09:12:13.367656+0000","last_unstale":"2024-09-13T09:12:51.426975+0000","last_undegraded":"2024-09-13T09:12:51.426975+0000","last_fullsized":"2024-09-13T09:12:51.426975+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:11.942294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003044836,"stat_sum":{"num_bytes":15526944,"num_objects":24,"num_object_clones":9,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":1415,"num_read_kb":14662,"num_write":136,"num_write_kb":1483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.284477+0000","last_change":"2024-09-13T09:12:06.845125+0000","last_active":"2024-09-13T09:12:50.284477+0000","last_peered":"2024-09-13T09:12:50.284477+0000","last_clean":"2024-09-13T09:12:50.284477+0000","last_became_active":"2024-09-13T09:12:04.321941+0000","last_became_peered":"2024-09-13T09:12:04.321941+0000","last_unstale":"2024-09-13T09:12:50.284477+0000","last_undegraded":"2024-09-13T09:12:50.284477+0000","last_fullsized":"2024-09-13T09:12:50.284477+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:57.706060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031569600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"55'98","reported_seq":1225,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.416394+0000","last_change":"2024-09-13T09:12:16.380222+0000","last_active":"2024-09-13T09:12:51.416394+0000","last_peered":"2024-09-13T09:12:51.416394+0000","last_clean":"2024-09-13T09:12:51.416394+0000","last_became_active":"2024-09-13T09:12:13.367633+0000","last_became_peered":"2024-09-13T09:12:13.367633+0000","last_unstale":"2024-09-13T09:12:51.416394+0000","last_undegraded":"2024-09-13T09:12:51.416394+0000","last_fullsized":"2024-09-13T09:12:51.416394+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:34.734632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438009,"stat_sum":{"num_bytes":6182912,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":1094,"num_read_kb":8707,"num_write":97,"num_write_kb":968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.287845+0000","last_change":"2024-09-13T09:12:43.532201+0000","last_active":"2024-09-13T09:12:50.287845+0000","last_peered":"2024-09-13T09:12:50.287845+0000","last_clean":"2024-09-13T09:12:50.287845+0000","last_became_active":"2024-09-13T09:12:43.531942+0000","last_became_peered":"2024-09-13T09:12:43.531942+0000","last_unstale":"2024-09-13T09:12:50.287845+0000","last_undegraded":"2024-09-13T09:12:50.287845+0000","last_fullsized":"2024-09-13T09:12:50.287845+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:26.408883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"55'232","reported_seq":1010,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.409444+0000","last_change":"2024-09-13T09:12:16.377544+0000","last_active":"2024-09-13T09:12:51.409444+0000","last_peered":"2024-09-13T09:12:51.409444+0000","last_clean":"2024-09-13T09:12:51.409444+0000","last_became_active":"2024-09-13T09:12:13.361909+0000","last_became_peered":"2024-09-13T09:12:13.361909+0000","last_unstale":"2024-09-13T09:12:51.409444+0000","last_undegraded":"2024-09-13T09:12:51.409444+0000","last_fullsized":"2024-09-13T09:12:51.409444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:34.570438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001546782,"stat_sum":{"num_bytes":3548775,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":1539,"num_read_kb":3778,"num_write":342,"num_write_kb":887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":53,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.303101+0000","last_change":"2024-09-13T09:12:06.330140+0000","last_active":"2024-09-13T09:12:50.303101+0000","last_peered":"2024-09-13T09:12:50.303101+0000","last_clean":"2024-09-13T09:12:50.303101+0000","last_became_active":"2024-09-13T09:12:04.324857+0000","last_became_peered":"2024-09-13T09:12:04.324857+0000","last_unstale":"2024-09-13T09:12:50.303101+0000","last_undegraded":"2024-09-13T09:12:50.303101+0000","last_fullsized":"2024-09-13T09:12:50.303101+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:57.400286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038911000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.288060+0000","last_change":"2024-09-13T09:12:46.248714+0000","last_active":"2024-09-13T09:12:50.288060+0000","last_peered":"2024-09-13T09:12:50.288060+0000","last_clean":"2024-09-13T09:12:50.288060+0000","last_became_active":"2024-09-13T09:12:46.248451+0000","last_became_peered":"2024-09-13T09:12:46.248451+0000","last_unstale":"2024-09-13T09:12:50.288060+0000","last_undegraded":"2024-09-13T09:12:50.288060+0000","last_fullsized":"2024-09-13T09:12:50.288060+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:27.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"55'104","reported_seq":610,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.421862+0000","last_change":"2024-09-13T09:12:16.377642+0000","last_active":"2024-09-13T09:12:51.421862+0000","last_peered":"2024-09-13T09:12:51.421862+0000","last_clean":"2024-09-13T09:12:51.421862+0000","last_became_active":"2024-09-13T09:12:13.369742+0000","last_became_peered":"2024-09-13T09:12:13.369742+0000","last_unstale":"2024-09-13T09:12:51.421862+0000","last_undegraded":"2024-09-13T09:12:51.421862+0000","last_fullsized":"2024-09-13T09:12:51.421862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:27:48.361252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060355399999999996,"stat_sum":{"num_bytes":19369518,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":586,"num_read_kb":3371,"num_write":113,"num_write_kb":2896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"55'329","reported_seq":6351,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.411749+0000","last_change":"2024-09-13T09:12:16.380989+0000","last_active":"2024-09-13T09:12:51.411749+0000","last_peered":"2024-09-13T09:12:51.411749+0000","last_clean":"2024-09-13T09:12:51.411749+0000","last_became_active":"2024-09-13T09:12:13.370096+0000","last_became_peered":"2024-09-13T09:12:13.370096+0000","last_unstale":"2024-09-13T09:12:51.411749+0000","last_undegraded":"2024-09-13T09:12:51.411749+0000","last_fullsized":"2024-09-13T09:12:51.411749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:14.224546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000903458,"stat_sum":{"num_bytes":39831040,"num_objects":22,"num_object_clones":9,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":8500,"num_read_kb":19304,"num_write":220,"num_write_kb":1089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"55'136","reported_seq":1746,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.427613+0000","last_change":"2024-09-13T09:12:46.253134+0000","last_active":"2024-09-13T09:12:51.427613+0000","last_peered":"2024-09-13T09:12:51.427613+0000","last_clean":"2024-09-13T09:12:51.427613+0000","last_became_active":"2024-09-13T09:12:46.252578+0000","last_became_peered":"2024-09-13T09:12:46.252578+0000","last_unstale":"2024-09-13T09:12:51.427613+0000","last_undegraded":"2024-09-13T09:12:51.427613+0000","last_fullsized":"2024-09-13T09:12:51.427613+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:03.931492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041459800000000002,"stat_sum":{"num_bytes":51840068,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":4,"num_read":1569,"num_read_kb":18162,"num_write":150,"num_write_kb":3781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2590736,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"55'101","reported_seq":2074,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.410222+0000","last_change":"2024-09-13T09:12:16.377864+0000","last_active":"2024-09-13T09:12:51.410222+0000","last_peered":"2024-09-13T09:12:51.410222+0000","last_clean":"2024-09-13T09:12:51.410222+0000","last_became_active":"2024-09-13T09:12:13.362409+0000","last_became_peered":"2024-09-13T09:12:13.362409+0000","last_unstale":"2024-09-13T09:12:51.410222+0000","last_undegraded":"2024-09-13T09:12:51.410222+0000","last_fullsized":"2024-09-13T09:12:51.410222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:37.023594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001713904,"stat_sum":{"num_bytes":40788512,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":1961,"num_read_kb":21606,"num_write":139,"num_write_kb":3597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"55'371","reported_seq":4130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.424863+0000","last_change":"2024-09-13T09:12:16.389470+0000","last_active":"2024-09-13T09:12:51.424863+0000","last_peered":"2024-09-13T09:12:51.424863+0000","last_clean":"2024-09-13T09:12:51.424863+0000","last_became_active":"2024-09-13T09:12:13.365758+0000","last_became_peered":"2024-09-13T09:12:13.365758+0000","last_unstale":"2024-09-13T09:12:51.424863+0000","last_undegraded":"2024-09-13T09:12:51.424863+0000","last_fullsized":"2024-09-13T09:12:51.424863+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:46.649879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013123500000000001,"stat_sum":{"num_bytes":39062602,"num_objects":25,"num_object_clones":10,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":5167,"num_read_kb":22878,"num_write":588,"num_write_kb":3406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"55'116","reported_seq":337,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.420419+0000","last_change":"2024-09-13T09:12:43.630169+0000","last_active":"2024-09-13T09:12:51.420419+0000","last_peered":"2024-09-13T09:12:51.420419+0000","last_clean":"2024-09-13T09:12:51.420419+0000","last_became_active":"2024-09-13T09:12:43.533321+0000","last_became_peered":"2024-09-13T09:12:43.533321+0000","last_unstale":"2024-09-13T09:12:51.420419+0000","last_undegraded":"2024-09-13T09:12:51.420419+0000","last_fullsized":"2024-09-13T09:12:51.420419+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:43.973948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000679308,"stat_sum":{"num_bytes":3810347,"num_objects":14,"num_object_clones":6,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":357,"num_read_kb":1959,"num_write":117,"num_write_kb":809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"55'110","reported_seq":1226,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.459723+0000","last_change":"2024-09-13T09:12:16.389243+0000","last_active":"2024-09-13T09:12:51.459723+0000","last_peered":"2024-09-13T09:12:51.459723+0000","last_clean":"2024-09-13T09:12:51.459723+0000","last_became_active":"2024-09-13T09:12:13.362547+0000","last_became_peered":"2024-09-13T09:12:13.362547+0000","last_unstale":"2024-09-13T09:12:51.459723+0000","last_undegraded":"2024-09-13T09:12:51.459723+0000","last_fullsized":"2024-09-13T09:12:51.459723+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:24:24.557400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011202860000000001,"stat_sum":{"num_bytes":2537984,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":1665,"num_read_kb":3468,"num_write":90,"num_write_kb":612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"53'112","reported_seq":827,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.303260+0000","last_change":"2024-09-13T09:12:43.623612+0000","last_active":"2024-09-13T09:12:50.303260+0000","last_peered":"2024-09-13T09:12:50.303260+0000","last_clean":"2024-09-13T09:12:50.303260+0000","last_became_active":"2024-09-13T09:12:43.533088+0000","last_became_peered":"2024-09-13T09:12:43.533088+0000","last_unstale":"2024-09-13T09:12:50.303260+0000","last_undegraded":"2024-09-13T09:12:50.303260+0000","last_fullsized":"2024-09-13T09:12:50.303260+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:43:49.678463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059255799999999997,"stat_sum":{"num_bytes":2678908,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":661,"num_read_kb":1263,"num_write":114,"num_write_kb":1329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":635004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"55'84","reported_seq":827,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.366624+0000","last_change":"2024-09-13T09:12:16.389277+0000","last_active":"2024-09-13T09:12:51.366624+0000","last_peered":"2024-09-13T09:12:51.366624+0000","last_clean":"2024-09-13T09:12:51.366624+0000","last_became_active":"2024-09-13T09:12:13.367757+0000","last_became_peered":"2024-09-13T09:12:13.367757+0000","last_unstale":"2024-09-13T09:12:51.366624+0000","last_undegraded":"2024-09-13T09:12:51.366624+0000","last_fullsized":"2024-09-13T09:12:51.366624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:33.596156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011263219999999999,"stat_sum":{"num_bytes":8565852,"num_objects":12,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":721,"num_read_kb":4898,"num_write":81,"num_write_kb":871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"53'125","reported_seq":434,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.880961+0000","last_change":"2024-09-13T09:12:43.614382+0000","last_active":"2024-09-13T09:12:50.880961+0000","last_peered":"2024-09-13T09:12:50.880961+0000","last_clean":"2024-09-13T09:12:50.880961+0000","last_became_active":"2024-09-13T09:12:43.532504+0000","last_became_peered":"2024-09-13T09:12:43.532504+0000","last_unstale":"2024-09-13T09:12:50.880961+0000","last_undegraded":"2024-09-13T09:12:50.880961+0000","last_fullsized":"2024-09-13T09:12:50.880961+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:25:04.225563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034391499999999998,"stat_sum":{"num_bytes":11360807,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":928,"num_read_kb":11657,"num_write":129,"num_write_kb":1596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":694311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"55'167","reported_seq":7571,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.428666+0000","last_change":"2024-09-13T09:12:46.248982+0000","last_active":"2024-09-13T09:12:51.428666+0000","last_peered":"2024-09-13T09:12:51.428666+0000","last_clean":"2024-09-13T09:12:51.428666+0000","last_became_active":"2024-09-13T09:12:46.248261+0000","last_became_peered":"2024-09-13T09:12:46.248261+0000","last_unstale":"2024-09-13T09:12:51.428666+0000","last_undegraded":"2024-09-13T09:12:51.428666+0000","last_fullsized":"2024-09-13T09:12:51.428666+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:44.056762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049261300000000004,"stat_sum":{"num_bytes":18003968,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":7227,"num_read_kb":19668,"num_write":166,"num_write_kb":1674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":676864,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"55'339","reported_seq":3933,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.478669+0000","last_change":"2024-09-13T09:12:46.251913+0000","last_active":"2024-09-13T09:12:51.478669+0000","last_peered":"2024-09-13T09:12:51.478669+0000","last_clean":"2024-09-13T09:12:51.478669+0000","last_became_active":"2024-09-13T09:12:46.250552+0000","last_became_peered":"2024-09-13T09:12:46.250552+0000","last_unstale":"2024-09-13T09:12:51.478669+0000","last_undegraded":"2024-09-13T09:12:51.478669+0000","last_fullsized":"2024-09-13T09:12:51.478669+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:47.142721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059370899999999999,"stat_sum":{"num_bytes":4122729,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":4142,"num_read_kb":4708,"num_write":317,"num_write_kb":945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":28777,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":632750095,"num_objects":571,"num_object_clones":187,"num_object_copies":1142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":571,"num_whiteouts":170,"num_read":106168,"num_read_kb":423143,"num_write":8446,"num_write_kb":63946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":8739591,"num_keys_recovered":33,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":47992832,"data_stored":52231198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7160,"ondisk_log_size":7160,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640270,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45904,"kb_used_data":6600,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94325936,"statfs":{"total":96636764160,"available":96589758464,"internally_reserved":0,"allocated":6758400,"data_stored":6708214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]}]},{"osd":4,"up_from":11,"seq":47244640271,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":10764,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":11022336,"data_stored":12048839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640272,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43760,"kb_used_data":8552,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94328080,"statfs":{"total":96636764160,"available":96591953920,"internally_reserved":0,"allocated":8757248,"data_stored":9241505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640270,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39780,"kb_used_data":4572,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332060,"statfs":{"total":96636764160,"available":96596029440,"internally_reserved":0,"allocated":4681728,"data_stored":4587364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]}]},{"osd":1,"up_from":11,"seq":47244640271,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42324,"kb_used_data":7116,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94329516,"statfs":{"total":96636764160,"available":96593424384,"internally_reserved":0,"allocated":7286784,"data_stored":7663640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36043205},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]}]},{"osd":0,"up_from":11,"seq":47244640270,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52768,"kb_used_data":13456,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94319072,"statfs":{"total":96636764160,"available":96582729728,"internally_reserved":0,"allocated":13778944,"data_stored":14083048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12562432,"data_stored":13246709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6717440,"data_stored":7461305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4456216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7540736,"data_stored":8405166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10620928,"data_stored":12005807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6356992,"data_stored":6655995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:12:54.051 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T09:12:54.051 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:12:54.051 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T09:12:54.051 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:12:54.056 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:54.097 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:54.096+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:54.161 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:54.160+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:54.199 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:54.199 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:54.207 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:54.262 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:54.261+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:54.310 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:54.310 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:54.320 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:54.338 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:12:54.338 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:54.338 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 58, 2024-09-13T09:12:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "06cda68b-dd73-4266-b5e6-93ae2b3f7bd5", 2024-09-13T09:12:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2024-09-13T09:11:45.458147+0000", 2024-09-13T09:12:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T09:12:54.303050+0000", 2024-09-13T09:12:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "last_up_change": "2024-09-13T09:11:56.271910+0000", 2024-09-13T09:12:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "last_in_change": "2024-09-13T09:12:42.513669+0000", 2024-09-13T09:12:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:12:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_num": 5806080, 2024-09-13T09:12:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_set": [ 2024-09-13T09:12:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "nodeep-scrub", 2024-09-13T09:12:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "noscrub", 2024-09-13T09:12:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "pglog_hardlimit", 2024-09-13T09:12:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "purged_snapdirs", 2024-09-13T09:12:54.339 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_deletes", 2024-09-13T09:12:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: "sortbitwise" 2024-09-13T09:12:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:12:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_version": 4, 2024-09-13T09:12:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:12:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:12:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:12:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_snapshot": "", 2024-09-13T09:12:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_max": 3, 2024-09-13T09:12:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd": 6, 2024-09-13T09:12:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:12:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: "min_compat_client": "jewel", 2024-09-13T09:12:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: "require_osd_release": "reef", 2024-09-13T09:12:54.340 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_crimson": false, 2024-09-13T09:12:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: "pools": [ 2024-09-13T09:12:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 1, 2024-09-13T09:12:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": ".mgr", 2024-09-13T09:12:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-09-13T09:11:57.562930+0000", 2024-09-13T09:12:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T09:12:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool", 2024-09-13T09:12:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-09-13T09:12:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-09-13T09:12:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-09-13T09:12:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-09-13T09:12:54.341 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:12:54.342 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:12:54.342 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:12:54.342 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:12:54.342 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-09-13T09:12:54.342 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:12:54.342 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 1, 2024-09-13T09:12:54.342 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 1, 2024-09-13T09:12:54.342 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 1, 2024-09-13T09:12:54.342 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 1, 2024-09-13T09:12:54.342 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 1, 2024-09-13T09:12:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-09-13T09:12:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-09-13T09:12:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-09-13T09:12:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-09-13T09:12:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-09-13T09:12:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-09-13T09:12:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-09-13T09:12:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "15", 2024-09-13T09:12:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-09-13T09:12:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:12:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:12:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-09-13T09:12:54.343 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:12:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 0, 2024-09-13T09:12:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 0, 2024-09-13T09:12:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-09-13T09:12:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-09-13T09:12:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-09-13T09:12:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-09-13T09:12:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-09-13T09:12:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-09-13T09:12:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-09-13T09:12:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-09-13T09:12:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-09-13T09:12:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-09-13T09:12:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-09-13T09:12:54.344 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:12:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:12:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:12:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-09-13T09:12:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-09-13T09:12:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-09-13T09:12:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-09-13T09:12:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-09-13T09:12:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-09-13T09:12:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-09-13T09:12:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-09-13T09:12:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:12:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:12:54.345 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:12:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:12:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-09-13T09:12:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-09-13T09:12:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-09-13T09:12:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-09-13T09:12:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: "options": { 2024-09-13T09:12:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_max": 32, 2024-09-13T09:12:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_min": 1 2024-09-13T09:12:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-09-13T09:12:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": {} 2024-09-13T09:12:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-09-13T09:12:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 6.059999942779541, 2024-09-13T09:12:54.347 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 6.059999942779541, 2024-09-13T09:12:54.347 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 0.33000001311302185, 2024-09-13T09:12:54.347 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 2, 2024-09-13T09:12:54.347 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 2, 2024-09-13T09:12:54.347 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:12:54.347 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 1, 2024-09-13T09:12:54.347 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:12:54.347 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.347 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 2, 2024-09-13T09:12:54.347 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": "rbd", 2024-09-13T09:12:54.347 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-09-13T09:12:03.214789+0000", 2024-09-13T09:12:54.347 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 8193, 2024-09-13T09:12:54.348 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:12:54.348 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-09-13T09:12:54.348 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-09-13T09:12:54.348 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-09-13T09:12:54.348 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-09-13T09:12:54.348 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:12:54.348 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:12:54.348 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:12:54.348 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:12:54.348 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-09-13T09:12:54.348 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:12:54.348 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 8, 2024-09-13T09:12:54.348 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 8, 2024-09-13T09:12:54.348 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 8, 2024-09-13T09:12:54.349 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 8, 2024-09-13T09:12:54.349 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 8, 2024-09-13T09:12:54.349 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-09-13T09:12:54.349 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-09-13T09:12:54.349 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-09-13T09:12:54.349 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-09-13T09:12:54.349 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-09-13T09:12:54.349 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-09-13T09:12:54.349 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-09-13T09:12:54.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.349 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "19", 2024-09-13T09:12:54.349 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-09-13T09:12:54.349 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:12:54.350 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:12:54.350 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-09-13T09:12:54.350 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:12:54.350 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 3, 2024-09-13T09:12:54.350 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 19, 2024-09-13T09:12:54.350 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-09-13T09:12:54.350 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-09-13T09:12:54.350 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-09-13T09:12:54.350 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-09-13T09:12:54.350 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-09-13T09:12:54.350 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-09-13T09:12:54.350 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-09-13T09:12:54.350 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-09-13T09:12:54.350 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-09-13T09:12:54.351 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-09-13T09:12:54.351 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-09-13T09:12:54.351 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:12:54.351 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:12:54.351 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:12:54.351 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-09-13T09:12:54.351 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-09-13T09:12:54.351 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-09-13T09:12:54.351 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-09-13T09:12:54.351 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-09-13T09:12:54.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.351 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-09-13T09:12:54.351 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-09-13T09:12:54.351 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-09-13T09:12:54.352 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:12:54.352 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:12:54.352 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:12:54.352 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:12:54.352 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-09-13T09:12:54.352 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-09-13T09:12:54.352 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-09-13T09:12:54.352 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-09-13T09:12:54.352 INFO:teuthology.orchestra.run.smithi044.stdout: "options": {}, 2024-09-13T09:12:54.352 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-09-13T09:12:54.352 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd": {} 2024-09-13T09:12:54.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.352 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-09-13T09:12:54.353 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 2.25, 2024-09-13T09:12:54.353 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 2.25, 2024-09-13T09:12:54.353 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 1, 2024-09-13T09:12:54.353 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 2.25, 2024-09-13T09:12:54.353 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 2.25, 2024-09-13T09:12:54.353 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:12:54.353 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 1, 2024-09-13T09:12:54.353 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:12:54.353 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.353 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 3, 2024-09-13T09:12:54.353 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": "pool_client.0", 2024-09-13T09:12:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-09-13T09:12:12.120436+0000", 2024-09-13T09:12:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 8193, 2024-09-13T09:12:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:12:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-09-13T09:12:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-09-13T09:12:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-09-13T09:12:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-09-13T09:12:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:12:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:12:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:12:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:12:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-09-13T09:12:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:12:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 32, 2024-09-13T09:12:54.354 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 32, 2024-09-13T09:12:54.355 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 32, 2024-09-13T09:12:54.355 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 32, 2024-09-13T09:12:54.355 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 32, 2024-09-13T09:12:54.355 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-09-13T09:12:54.355 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-09-13T09:12:54.355 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-09-13T09:12:54.355 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-09-13T09:12:54.355 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-09-13T09:12:54.355 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-09-13T09:12:54.355 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-09-13T09:12:54.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.355 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "58", 2024-09-13T09:12:54.355 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-09-13T09:12:54.355 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:12:54.356 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:12:54.356 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-09-13T09:12:54.356 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:12:54.356 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 21, 2024-09-13T09:12:54.356 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 58, 2024-09-13T09:12:54.356 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-09-13T09:12:54.356 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-09-13T09:12:54.356 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-09-13T09:12:54.356 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-09-13T09:12:54.356 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-09-13T09:12:54.356 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-09-13T09:12:54.356 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-09-13T09:12:54.356 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-09-13T09:12:54.357 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-09-13T09:12:54.357 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-09-13T09:12:54.357 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-09-13T09:12:54.357 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:12:54.357 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:12:54.357 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:12:54.357 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-09-13T09:12:54.357 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-09-13T09:12:54.357 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-09-13T09:12:54.357 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-09-13T09:12:54.357 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-09-13T09:12:54.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.357 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-09-13T09:12:54.358 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-09-13T09:12:54.358 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-09-13T09:12:54.358 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:12:54.358 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:12:54.358 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:12:54.358 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:12:54.358 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-09-13T09:12:54.358 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-09-13T09:12:54.358 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-09-13T09:12:54.358 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-09-13T09:12:54.358 INFO:teuthology.orchestra.run.smithi044.stdout: "options": {}, 2024-09-13T09:12:54.358 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-09-13T09:12:54.358 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd": {} 2024-09-13T09:12:54.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.359 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-09-13T09:12:54.359 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 1.5, 2024-09-13T09:12:54.359 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 1.5, 2024-09-13T09:12:54.359 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 1, 2024-09-13T09:12:54.359 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 1.5, 2024-09-13T09:12:54.359 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 1.5, 2024-09-13T09:12:54.359 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:12:54.359 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 1, 2024-09-13T09:12:54.359 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:12:54.359 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.359 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.360 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:12:54.360 INFO:teuthology.orchestra.run.smithi044.stdout: "osds": [ 2024-09-13T09:12:54.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.360 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 0, 2024-09-13T09:12:54.360 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "156c6730-f804-4806-bc2d-3a7d0856c2b2", 2024-09-13T09:12:54.360 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:12:54.360 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:12:54.360 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:12:54.360 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:12:54.360 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:12:54.360 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:12:54.360 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:12:54.360 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 49, 2024-09-13T09:12:54.361 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:12:54.361 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:12:54.361 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:12:54.361 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.361 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.361 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6808", 2024-09-13T09:12:54.361 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:12:54.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.361 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.361 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6810", 2024-09-13T09:12:54.361 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:12:54.362 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.362 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.362 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:12:54.362 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.362 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.362 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6813", 2024-09-13T09:12:54.362 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:12:54.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.362 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.362 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6815", 2024-09-13T09:12:54.363 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:12:54.363 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.363 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.363 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:12:54.363 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.363 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.363 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6821", 2024-09-13T09:12:54.363 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:12:54.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.363 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.364 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6823", 2024-09-13T09:12:54.364 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:12:54.364 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.364 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.364 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:12:54.364 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.364 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.365 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6817", 2024-09-13T09:12:54.365 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:12:54.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.365 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.365 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6819", 2024-09-13T09:12:54.365 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:12:54.365 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.365 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.365 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6810/2268459392", 2024-09-13T09:12:54.366 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6815/2268459392", 2024-09-13T09:12:54.366 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6823/2268459392", 2024-09-13T09:12:54.366 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6819/2268459392", 2024-09-13T09:12:54.366 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:12:54.366 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:12:54.366 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:12:54.366 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.366 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 1, 2024-09-13T09:12:54.366 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "00735025-3bf9-4dd0-8b42-f08f1bd1c421", 2024-09-13T09:12:54.366 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:12:54.367 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:12:54.367 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:12:54.367 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:12:54.367 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:12:54.367 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:12:54.367 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:12:54.367 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 48, 2024-09-13T09:12:54.367 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:12:54.367 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:12:54.367 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:12:54.367 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.367 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.367 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6800", 2024-09-13T09:12:54.368 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:12:54.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.368 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.368 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6801", 2024-09-13T09:12:54.368 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:12:54.368 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.368 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.368 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:12:54.368 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.368 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.369 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6802", 2024-09-13T09:12:54.369 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:12:54.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.369 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.369 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6803", 2024-09-13T09:12:54.369 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:12:54.369 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.369 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.369 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:12:54.369 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.370 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.370 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6806", 2024-09-13T09:12:54.370 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:12:54.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.370 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.370 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6807", 2024-09-13T09:12:54.370 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:12:54.370 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.370 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.371 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:12:54.371 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.371 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.371 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6804", 2024-09-13T09:12:54.371 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:12:54.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.371 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.371 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6805", 2024-09-13T09:12:54.371 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:12:54.371 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.371 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.372 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6801/2209612586", 2024-09-13T09:12:54.372 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6803/2209612586", 2024-09-13T09:12:54.372 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6807/2209612586", 2024-09-13T09:12:54.372 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6805/2209612586", 2024-09-13T09:12:54.372 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:12:54.372 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:12:54.372 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:12:54.372 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.372 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 2, 2024-09-13T09:12:54.372 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "395516ac-33b6-4f43-8333-dcb1f4cded32", 2024-09-13T09:12:54.372 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:12:54.373 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:12:54.373 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:12:54.373 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:12:54.373 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:12:54.373 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:12:54.373 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:12:54.373 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 48, 2024-09-13T09:12:54.373 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:12:54.373 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:12:54.373 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:12:54.373 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.373 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.373 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6809", 2024-09-13T09:12:54.374 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:12:54.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.374 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.374 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6811", 2024-09-13T09:12:54.374 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:12:54.374 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.374 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.374 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:12:54.374 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.374 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.374 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6812", 2024-09-13T09:12:54.375 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:12:54.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.375 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.375 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6814", 2024-09-13T09:12:54.375 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:12:54.375 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.375 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.375 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:12:54.375 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.375 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.375 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6820", 2024-09-13T09:12:54.376 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:12:54.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.376 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.376 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6822", 2024-09-13T09:12:54.376 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:12:54.376 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.376 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.376 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:12:54.376 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.376 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.377 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6816", 2024-09-13T09:12:54.377 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:12:54.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.377 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.377 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6818", 2024-09-13T09:12:54.377 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:12:54.377 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.377 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.377 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6811/770926265", 2024-09-13T09:12:54.377 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6814/770926265", 2024-09-13T09:12:54.377 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6822/770926265", 2024-09-13T09:12:54.377 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6818/770926265", 2024-09-13T09:12:54.378 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:12:54.378 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:12:54.378 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:12:54.378 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.378 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 3, 2024-09-13T09:12:54.378 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "429e877f-5462-402a-80e5-92a7b17a2eaa", 2024-09-13T09:12:54.378 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:12:54.378 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:12:54.378 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:12:54.378 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:12:54.378 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:12:54.378 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:12:54.379 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:12:54.379 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 49, 2024-09-13T09:12:54.379 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:12:54.379 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:12:54.379 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:12:54.379 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.379 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.379 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6808", 2024-09-13T09:12:54.379 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:12:54.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.379 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.380 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6809", 2024-09-13T09:12:54.380 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:12:54.380 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.380 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.380 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:12:54.380 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.380 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.380 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6810", 2024-09-13T09:12:54.380 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:12:54.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.380 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.381 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6811", 2024-09-13T09:12:54.381 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:12:54.381 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.381 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.381 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:12:54.381 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.381 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.381 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6814", 2024-09-13T09:12:54.381 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:12:54.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.382 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.382 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6815", 2024-09-13T09:12:54.382 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:12:54.382 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.382 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.382 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:12:54.382 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.382 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.382 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6812", 2024-09-13T09:12:54.382 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:12:54.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.383 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.383 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6813", 2024-09-13T09:12:54.383 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:12:54.383 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.383 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.383 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.129:6809/2132151186", 2024-09-13T09:12:54.383 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.129:6811/2132151186", 2024-09-13T09:12:54.383 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.129:6815/2132151186", 2024-09-13T09:12:54.383 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.129:6813/2132151186", 2024-09-13T09:12:54.383 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:12:54.383 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:12:54.383 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:12:54.384 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.384 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 4, 2024-09-13T09:12:54.384 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "9b3be757-15ee-4157-80a8-0537cd1bcf28", 2024-09-13T09:12:54.384 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:12:54.384 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:12:54.384 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:12:54.384 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:12:54.384 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:12:54.384 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:12:54.384 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:12:54.384 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 48, 2024-09-13T09:12:54.384 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:12:54.385 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:12:54.385 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:12:54.385 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.385 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.385 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6800", 2024-09-13T09:12:54.385 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:12:54.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.385 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.385 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6801", 2024-09-13T09:12:54.385 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:12:54.385 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.385 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.386 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:12:54.386 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.386 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.386 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6802", 2024-09-13T09:12:54.386 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:12:54.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.386 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.386 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6803", 2024-09-13T09:12:54.386 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:12:54.386 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.386 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.387 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:12:54.387 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.387 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.387 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6806", 2024-09-13T09:12:54.387 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:12:54.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.387 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.387 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6807", 2024-09-13T09:12:54.387 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:12:54.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.387 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.388 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:12:54.388 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.388 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.388 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6804", 2024-09-13T09:12:54.388 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:12:54.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.388 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.388 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6805", 2024-09-13T09:12:54.388 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:12:54.388 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.389 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.389 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.129:6801/3393542416", 2024-09-13T09:12:54.389 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.129:6803/3393542416", 2024-09-13T09:12:54.389 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.129:6807/3393542416", 2024-09-13T09:12:54.389 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.129:6805/3393542416", 2024-09-13T09:12:54.389 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:12:54.389 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:12:54.389 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:12:54.389 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.389 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 5, 2024-09-13T09:12:54.389 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "c7601d5e-2164-45e1-ab21-368d9e3af27e", 2024-09-13T09:12:54.390 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:12:54.390 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:12:54.390 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:12:54.390 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:12:54.390 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:12:54.390 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:12:54.390 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:12:54.390 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 49, 2024-09-13T09:12:54.390 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:12:54.390 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:12:54.390 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:12:54.390 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.391 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.391 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6816", 2024-09-13T09:12:54.391 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:12:54.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.391 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.391 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6817", 2024-09-13T09:12:54.391 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:12:54.391 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.391 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.391 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:12:54.391 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.392 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.392 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6818", 2024-09-13T09:12:54.392 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:12:54.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.392 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.392 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6819", 2024-09-13T09:12:54.392 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:12:54.392 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.392 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.392 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:12:54.392 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.393 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.393 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6822", 2024-09-13T09:12:54.393 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:12:54.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.393 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.393 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6823", 2024-09-13T09:12:54.393 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:12:54.393 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.393 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.393 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:12:54.393 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:12:54.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.394 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:12:54.394 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6820", 2024-09-13T09:12:54.394 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:12:54.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.394 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:12:54.394 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6821", 2024-09-13T09:12:54.394 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:12:54.394 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.394 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.394 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.129:6817/1803198188", 2024-09-13T09:12:54.394 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.129:6819/1803198188", 2024-09-13T09:12:54.394 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.129:6823/1803198188", 2024-09-13T09:12:54.395 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.129:6821/1803198188", 2024-09-13T09:12:54.395 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:12:54.395 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:12:54.395 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:12:54.395 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:54.395 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.395 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:12:54.395 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_xinfo": [ 2024-09-13T09:12:54.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.395 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 0, 2024-09-13T09:12:54.395 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:12:54.395 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:12:54.395 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:12:54.395 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:12:54.396 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:12:54.396 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.230474+0000", 2024-09-13T09:12:54.396 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:12:54.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.396 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 1, 2024-09-13T09:12:54.396 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:12:54.396 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:12:54.396 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:12:54.396 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:12:54.396 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:12:54.396 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.195476+0000", 2024-09-13T09:12:54.396 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:12:54.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.397 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 2, 2024-09-13T09:12:54.397 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:12:54.397 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:12:54.397 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:12:54.397 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:12:54.397 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:12:54.397 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.248729+0000", 2024-09-13T09:12:54.397 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:12:54.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.397 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 3, 2024-09-13T09:12:54.397 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:12:54.397 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:12:54.398 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:12:54.398 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:12:54.398 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:12:54.398 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.195482+0000", 2024-09-13T09:12:54.398 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:12:54.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.398 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 4, 2024-09-13T09:12:54.398 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:12:54.398 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:12:54.398 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:12:54.398 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:12:54.398 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:12:54.398 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.198746+0000", 2024-09-13T09:12:54.399 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:12:54.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:54.399 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 5, 2024-09-13T09:12:54.399 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:12:54.399 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:12:54.399 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:12:54.399 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:12:54.399 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:12:54.399 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.279957+0000", 2024-09-13T09:12:54.399 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:12:54.399 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.399 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:12:54.399 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap": [], 2024-09-13T09:12:54.400 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap_items": [], 2024-09-13T09:12:54.400 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap_primaries": [], 2024-09-13T09:12:54.400 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_temp": [], 2024-09-13T09:12:54.400 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_temp": [], 2024-09-13T09:12:54.400 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklist": {}, 2024-09-13T09:12:54.400 INFO:teuthology.orchestra.run.smithi044.stdout: "range_blocklist": {}, 2024-09-13T09:12:54.400 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profiles": { 2024-09-13T09:12:54.400 INFO:teuthology.orchestra.run.smithi044.stdout: "default": { 2024-09-13T09:12:54.400 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-failure-domain": "osd", 2024-09-13T09:12:54.400 INFO:teuthology.orchestra.run.smithi044.stdout: "k": "2", 2024-09-13T09:12:54.400 INFO:teuthology.orchestra.run.smithi044.stdout: "m": "1", 2024-09-13T09:12:54.400 INFO:teuthology.orchestra.run.smithi044.stdout: "plugin": "jerasure", 2024-09-13T09:12:54.400 INFO:teuthology.orchestra.run.smithi044.stdout: "technique": "reed_sol_van" 2024-09-13T09:12:54.400 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:54.401 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps_queue": [], 2024-09-13T09:12:54.401 INFO:teuthology.orchestra.run.smithi044.stdout: "new_removed_snaps": [], 2024-09-13T09:12:54.401 INFO:teuthology.orchestra.run.smithi044.stdout: "new_purged_snaps": [], 2024-09-13T09:12:54.401 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_node_flags": {}, 2024-09-13T09:12:54.401 INFO:teuthology.orchestra.run.smithi044.stdout: "device_class_flags": {}, 2024-09-13T09:12:54.401 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": { 2024-09-13T09:12:54.401 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode_enabled": false, 2024-09-13T09:12:54.401 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_bucket_count": 0, 2024-09-13T09:12:54.401 INFO:teuthology.orchestra.run.smithi044.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:12:54.401 INFO:teuthology.orchestra.run.smithi044.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:12:54.401 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode_bucket": 0 2024-09-13T09:12:54.401 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:54.401 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:54.403 INFO:tasks.thrashosds.thrasher:j is {'epoch': 58, 'fsid': '06cda68b-dd73-4266-b5e6-93ae2b3f7bd5', 'created': '2024-09-13T09:11:45.458147+0000', 'modified': '2024-09-13T09:12:54.303050+0000', 'last_up_change': '2024-09-13T09:11:56.271910+0000', 'last_in_change': '2024-09-13T09:12:42.513669+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:11:57.562930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:12:03.214789+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:12:12.120436+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 58, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '156c6730-f804-4806-bc2d-3a7d0856c2b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6808', 'nonce': 2268459392}, {'type': 'v1', 'addr': '172.21.15.44:6810', 'nonce': 2268459392}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6813', 'nonce': 2268459392}, {'type': 'v1', 'addr': '172.21.15.44:6815', 'nonce': 2268459392}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6821', 'nonce': 2268459392}, {'type': 'v1', 'addr': '172.21.15.44:6823', 'nonce': 2268459392}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6817', 'nonce': 2268459392}, {'type': 'v1', 'addr': '172.21.15.44:6819', 'nonce': 2268459392}]}, 'public_addr': '172.21.15.44:6810/2268459392', 'cluster_addr': '172.21.15.44:6815/2268459392', 'heartbeat_back_addr': '172.21.15.44:6823/2268459392', 'heartbeat_front_addr': '172.21.15.44:6819/2268459392', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '00735025-3bf9-4dd0-8b42-f08f1bd1c421', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6800', 'nonce': 2209612586}, {'type': 'v1', 'addr': '172.21.15.44:6801', 'nonce': 2209612586}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6802', 'nonce': 2209612586}, {'type': 'v1', 'addr': '172.21.15.44:6803', 'nonce': 2209612586}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6806', 'nonce': 2209612586}, {'type': 'v1', 'addr': '172.21.15.44:6807', 'nonce': 2209612586}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6804', 'nonce': 2209612586}, {'type': 'v1', 'addr': '172.21.15.44:6805', 'nonce': 2209612586}]}, 'public_addr': '172.21.15.44:6801/2209612586', 'cluster_addr': '172.21.15.44:6803/2209612586', 'heartbeat_back_addr': '172.21.15.44:6807/2209612586', 'heartbeat_front_addr': '172.21.15.44:6805/2209612586', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '395516ac-33b6-4f43-8333-dcb1f4cded32', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6809', 'nonce': 770926265}, {'type': 'v1', 'addr': '172.21.15.44:6811', 'nonce': 770926265}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6812', 'nonce': 770926265}, {'type': 'v1', 'addr': '172.21.15.44:6814', 'nonce': 770926265}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6820', 'nonce': 770926265}, {'type': 'v1', 'addr': '172.21.15.44:6822', 'nonce': 770926265}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6816', 'nonce': 770926265}, {'type': 'v1', 'addr': '172.21.15.44:6818', 'nonce': 770926265}]}, 'public_addr': '172.21.15.44:6811/770926265', 'cluster_addr': '172.21.15.44:6814/770926265', 'heartbeat_back_addr': '172.21.15.44:6822/770926265', 'heartbeat_front_addr': '172.21.15.44:6818/770926265', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '429e877f-5462-402a-80e5-92a7b17a2eaa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6808', 'nonce': 2132151186}, {'type': 'v1', 'addr': '172.21.15.129:6809', 'nonce': 2132151186}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6810', 'nonce': 2132151186}, {'type': 'v1', 'addr': '172.21.15.129:6811', 'nonce': 2132151186}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6814', 'nonce': 2132151186}, {'type': 'v1', 'addr': '172.21.15.129:6815', 'nonce': 2132151186}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6812', 'nonce': 2132151186}, {'type': 'v1', 'addr': '172.21.15.129:6813', 'nonce': 2132151186}]}, 'public_addr': '172.21.15.129:6809/2132151186', 'cluster_addr': '172.21.15.129:6811/2132151186', 'heartbeat_back_addr': '172.21.15.129:6815/2132151186', 'heartbeat_front_addr': '172.21.15.129:6813/2132151186', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '9b3be757-15ee-4157-80a8-0537cd1bcf28', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6800', 'nonce': 3393542416}, {'type': 'v1', 'addr': '172.21.15.129:6801', 'nonce': 3393542416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6802', 'nonce': 3393542416}, {'type': 'v1', 'addr': '172.21.15.129:6803', 'nonce': 3393542416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6806', 'nonce': 3393542416}, {'type': 'v1', 'addr': '172.21.15.129:6807', 'nonce': 3393542416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6804', 'nonce': 3393542416}, {'type': 'v1', 'addr': '172.21.15.129:6805', 'nonce': 3393542416}]}, 'public_addr': '172.21.15.129:6801/3393542416', 'cluster_addr': '172.21.15.129:6803/3393542416', 'heartbeat_back_addr': '172.21.15.129:6807/3393542416', 'heartbeat_front_addr': '172.21.15.129:6805/3393542416', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c7601d5e-2164-45e1-ab21-368d9e3af27e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6816', 'nonce': 1803198188}, {'type': 'v1', 'addr': '172.21.15.129:6817', 'nonce': 1803198188}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6818', 'nonce': 1803198188}, {'type': 'v1', 'addr': '172.21.15.129:6819', 'nonce': 1803198188}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6822', 'nonce': 1803198188}, {'type': 'v1', 'addr': '172.21.15.129:6823', 'nonce': 1803198188}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6820', 'nonce': 1803198188}, {'type': 'v1', 'addr': '172.21.15.129:6821', 'nonce': 1803198188}]}, 'public_addr': '172.21.15.129:6817/1803198188', 'cluster_addr': '172.21.15.129:6819/1803198188', 'heartbeat_back_addr': '172.21.15.129:6823/1803198188', 'heartbeat_front_addr': '172.21.15.129:6821/1803198188', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.230474+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.195476+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.248729+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.195482+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.198746+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.279957+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:12:54.403 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:12:54.404 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:54.402+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:54.426 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:54.426 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:54.429 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone17 from 0xcbc4000 (overlap 0x2640400) to 0x0 2024-09-13T09:12:54.435 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:54.521 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:54.519+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:54.581 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:54.581 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:54.591 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:54.633 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:54.632+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:54.705 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:54.704+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:54.731 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:12:54.731 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:12:54.738 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:54.738 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:54.743 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-13T09:12:54.306748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633340482,"num_objects":575,"num_object_clones":187,"num_object_copies":1150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":575,"num_whiteouts":170,"num_read":106284,"num_read_kb":423243,"num_write":8607,"num_write_kb":66364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":9920327,"num_keys_recovered":33,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7280,"ondisk_log_size":7280,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":274608,"kb_used_data":51060,"kb_used_omap":9,"kb_used_meta":223478,"kb_avail":565956432,"statfs":{"total":579820584960,"available":579539386368,"internally_reserved":0,"allocated":52285440,"data_stored":54332610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9563,"internal_metadata":228842149},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50085837,"num_objects":241,"num_object_clones":119,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":134,"num_read":22792,"num_read_kb":49871,"num_write":1907,"num_write_kb":9905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025382"},"pg_stats":[{"pgid":"3.1f","version":"55'128","reported_seq":1367,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.410299+0000","last_change":"2024-09-13T09:12:43.692870+0000","last_active":"2024-09-13T09:12:51.410299+0000","last_peered":"2024-09-13T09:12:51.410299+0000","last_clean":"2024-09-13T09:12:51.410299+0000","last_became_active":"2024-09-13T09:12:43.534990+0000","last_became_peered":"2024-09-13T09:12:43.534990+0000","last_unstale":"2024-09-13T09:12:51.410299+0000","last_undegraded":"2024-09-13T09:12:51.410299+0000","last_fullsized":"2024-09-13T09:12:51.410299+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:45.810610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058630900000000003,"stat_sum":{"num_bytes":5467152,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":1740,"num_read_kb":3462,"num_write":108,"num_write_kb":1463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":848912,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"55'123","reported_seq":1144,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.426111+0000","last_change":"2024-09-13T09:12:43.652049+0000","last_active":"2024-09-13T09:12:51.426111+0000","last_peered":"2024-09-13T09:12:51.426111+0000","last_clean":"2024-09-13T09:12:51.426111+0000","last_became_active":"2024-09-13T09:12:43.532920+0000","last_became_peered":"2024-09-13T09:12:43.532920+0000","last_unstale":"2024-09-13T09:12:51.426111+0000","last_undegraded":"2024-09-13T09:12:51.426111+0000","last_fullsized":"2024-09-13T09:12:51.426111+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:10:38.824649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088145900000000002,"stat_sum":{"num_bytes":22193680,"num_objects":22,"num_object_clones":9,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":1265,"num_read_kb":8461,"num_write":114,"num_write_kb":1256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":311808,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"55'667","reported_seq":3067,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.376286+0000","last_change":"2024-09-13T09:12:16.388369+0000","last_active":"2024-09-13T09:12:51.376286+0000","last_peered":"2024-09-13T09:12:51.376286+0000","last_clean":"2024-09-13T09:12:51.376286+0000","last_became_active":"2024-09-13T09:12:13.362666+0000","last_became_peered":"2024-09-13T09:12:13.362666+0000","last_unstale":"2024-09-13T09:12:51.376286+0000","last_undegraded":"2024-09-13T09:12:51.376286+0000","last_fullsized":"2024-09-13T09:12:51.376286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:04.594826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039115800000000001,"stat_sum":{"num_bytes":5156001,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":5170,"num_read_kb":5491,"num_write":1022,"num_write_kb":1942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"55'236","reported_seq":2254,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.501746+0000","last_change":"2024-09-13T09:12:16.388972+0000","last_active":"2024-09-13T09:12:51.501746+0000","last_peered":"2024-09-13T09:12:51.501746+0000","last_clean":"2024-09-13T09:12:51.501746+0000","last_became_active":"2024-09-13T09:12:13.364622+0000","last_became_peered":"2024-09-13T09:12:13.364622+0000","last_unstale":"2024-09-13T09:12:51.501746+0000","last_undegraded":"2024-09-13T09:12:51.501746+0000","last_fullsized":"2024-09-13T09:12:51.501746+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:34:20.301837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088725000000000002,"stat_sum":{"num_bytes":26517606,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":3525,"num_read_kb":9339,"num_write":193,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"53'219","reported_seq":4733,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.292669+0000","last_change":"2024-09-13T09:12:16.380908+0000","last_active":"2024-09-13T09:12:51.292669+0000","last_peered":"2024-09-13T09:12:51.292669+0000","last_clean":"2024-09-13T09:12:51.292669+0000","last_became_active":"2024-09-13T09:12:13.368138+0000","last_became_peered":"2024-09-13T09:12:13.368138+0000","last_unstale":"2024-09-13T09:12:51.292669+0000","last_undegraded":"2024-09-13T09:12:51.292669+0000","last_fullsized":"2024-09-13T09:12:51.292669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:10.836108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082447100000000001,"stat_sum":{"num_bytes":35081288,"num_objects":29,"num_object_clones":10,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":4815,"num_read_kb":36300,"num_write":282,"num_write_kb":5612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"54'199","reported_seq":5493,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.472303+0000","last_change":"2024-09-13T09:12:16.380731+0000","last_active":"2024-09-13T09:12:51.472303+0000","last_peered":"2024-09-13T09:12:51.472303+0000","last_clean":"2024-09-13T09:12:51.472303+0000","last_became_active":"2024-09-13T09:12:13.362533+0000","last_became_peered":"2024-09-13T09:12:13.362533+0000","last_unstale":"2024-09-13T09:12:51.472303+0000","last_undegraded":"2024-09-13T09:12:51.472303+0000","last_fullsized":"2024-09-13T09:12:51.472303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:50.399315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072061100000000002,"stat_sum":{"num_bytes":23726627,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":6211,"num_read_kb":21824,"num_write":156,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"55'154","reported_seq":4819,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.430700+0000","last_change":"2024-09-13T09:12:16.376447+0000","last_active":"2024-09-13T09:12:51.430700+0000","last_peered":"2024-09-13T09:12:51.430700+0000","last_clean":"2024-09-13T09:12:51.430700+0000","last_became_active":"2024-09-13T09:12:13.368221+0000","last_became_peered":"2024-09-13T09:12:13.368221+0000","last_unstale":"2024-09-13T09:12:51.430700+0000","last_undegraded":"2024-09-13T09:12:51.430700+0000","last_fullsized":"2024-09-13T09:12:51.430700+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:10.966835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000740483,"stat_sum":{"num_bytes":51515443,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":4839,"num_read_kb":9610,"num_write":176,"num_write_kb":1495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"55'146","reported_seq":1280,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.418730+0000","last_change":"2024-09-13T09:12:16.377490+0000","last_active":"2024-09-13T09:12:51.418730+0000","last_peered":"2024-09-13T09:12:51.418730+0000","last_clean":"2024-09-13T09:12:51.418730+0000","last_became_active":"2024-09-13T09:12:13.367773+0000","last_became_peered":"2024-09-13T09:12:13.367773+0000","last_unstale":"2024-09-13T09:12:51.418730+0000","last_undegraded":"2024-09-13T09:12:51.418730+0000","last_fullsized":"2024-09-13T09:12:51.418730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:34.023964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000479879,"stat_sum":{"num_bytes":3067904,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":6,"num_read":2269,"num_read_kb":3413,"num_write":111,"num_write_kb":675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"53'204","reported_seq":2410,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.483336+0000","last_change":"2024-09-13T09:12:16.381209+0000","last_active":"2024-09-13T09:12:51.483336+0000","last_peered":"2024-09-13T09:12:51.483336+0000","last_clean":"2024-09-13T09:12:51.483336+0000","last_became_active":"2024-09-13T09:12:13.365820+0000","last_became_peered":"2024-09-13T09:12:13.365820+0000","last_unstale":"2024-09-13T09:12:51.483336+0000","last_undegraded":"2024-09-13T09:12:51.483336+0000","last_fullsized":"2024-09-13T09:12:51.483336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:48.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097766600000000008,"stat_sum":{"num_bytes":11883008,"num_objects":16,"num_object_clones":6,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":3062,"num_read_kb":16784,"num_write":211,"num_write_kb":5119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"55'317","reported_seq":2341,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.468478+0000","last_change":"2024-09-13T09:12:46.251509+0000","last_active":"2024-09-13T09:12:51.468478+0000","last_peered":"2024-09-13T09:12:51.468478+0000","last_clean":"2024-09-13T09:12:51.468478+0000","last_became_active":"2024-09-13T09:12:46.250870+0000","last_became_peered":"2024-09-13T09:12:46.250870+0000","last_unstale":"2024-09-13T09:12:51.468478+0000","last_undegraded":"2024-09-13T09:12:51.468478+0000","last_fullsized":"2024-09-13T09:12:51.468478+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:33.138405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040009000000000003,"stat_sum":{"num_bytes":22262387,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":2897,"num_read_kb":13342,"num_write":455,"num_write_kb":1653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":706121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"55'568","reported_seq":5808,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.410321+0000","last_change":"2024-09-13T09:12:46.255856+0000","last_active":"2024-09-13T09:12:51.410321+0000","last_peered":"2024-09-13T09:12:51.410321+0000","last_clean":"2024-09-13T09:12:51.410321+0000","last_became_active":"2024-09-13T09:12:46.250696+0000","last_became_peered":"2024-09-13T09:12:46.250696+0000","last_unstale":"2024-09-13T09:12:51.410321+0000","last_undegraded":"2024-09-13T09:12:51.410321+0000","last_fullsized":"2024-09-13T09:12:51.410321+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:43.821997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172600000000004,"stat_sum":{"num_bytes":16437285,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":7793,"num_read_kb":20827,"num_write":998,"num_write_kb":2875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1136450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"55'255","reported_seq":3209,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.411006+0000","last_change":"2024-09-13T09:12:46.254606+0000","last_active":"2024-09-13T09:12:51.411006+0000","last_peered":"2024-09-13T09:12:51.411006+0000","last_clean":"2024-09-13T09:12:51.411006+0000","last_became_active":"2024-09-13T09:12:46.254016+0000","last_became_peered":"2024-09-13T09:12:46.254016+0000","last_unstale":"2024-09-13T09:12:51.411006+0000","last_undegraded":"2024-09-13T09:12:51.411006+0000","last_fullsized":"2024-09-13T09:12:51.411006+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:50.193567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074689299999999999,"stat_sum":{"num_bytes":45383291,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":8,"num_read":3516,"num_read_kb":17232,"num_write":210,"num_write_kb":2007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1110608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.288095+0000","last_change":"2024-09-13T09:12:43.531552+0000","last_active":"2024-09-13T09:12:50.288095+0000","last_peered":"2024-09-13T09:12:50.288095+0000","last_clean":"2024-09-13T09:12:50.288095+0000","last_became_active":"2024-09-13T09:12:43.531274+0000","last_became_peered":"2024-09-13T09:12:43.531274+0000","last_unstale":"2024-09-13T09:12:50.288095+0000","last_undegraded":"2024-09-13T09:12:50.288095+0000","last_fullsized":"2024-09-13T09:12:50.288095+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:41:06.992512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"55'412","reported_seq":3597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.502909+0000","last_change":"2024-09-13T09:12:16.380593+0000","last_active":"2024-09-13T09:12:51.502909+0000","last_peered":"2024-09-13T09:12:51.502909+0000","last_clean":"2024-09-13T09:12:51.502909+0000","last_became_active":"2024-09-13T09:12:13.365496+0000","last_became_peered":"2024-09-13T09:12:13.365496+0000","last_unstale":"2024-09-13T09:12:51.502909+0000","last_undegraded":"2024-09-13T09:12:51.502909+0000","last_fullsized":"2024-09-13T09:12:51.502909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:29:49.115977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061775199999999995,"stat_sum":{"num_bytes":18079886,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":1,"num_read":4513,"num_read_kb":35473,"num_write":636,"num_write_kb":3106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.284411+0000","last_change":"2024-09-13T09:12:43.531266+0000","last_active":"2024-09-13T09:12:50.284411+0000","last_peered":"2024-09-13T09:12:50.284411+0000","last_clean":"2024-09-13T09:12:50.284411+0000","last_became_active":"2024-09-13T09:12:43.530941+0000","last_became_peered":"2024-09-13T09:12:43.530941+0000","last_unstale":"2024-09-13T09:12:50.284411+0000","last_undegraded":"2024-09-13T09:12:50.284411+0000","last_fullsized":"2024-09-13T09:12:50.284411+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:27.261467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"55'397","reported_seq":5388,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.420650+0000","last_change":"2024-09-13T09:12:16.378483+0000","last_active":"2024-09-13T09:12:51.420650+0000","last_peered":"2024-09-13T09:12:51.420650+0000","last_clean":"2024-09-13T09:12:51.420650+0000","last_became_active":"2024-09-13T09:12:13.367146+0000","last_became_peered":"2024-09-13T09:12:13.367146+0000","last_unstale":"2024-09-13T09:12:51.420650+0000","last_undegraded":"2024-09-13T09:12:51.420650+0000","last_fullsized":"2024-09-13T09:12:51.420650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:49.370653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032368300000000001,"stat_sum":{"num_bytes":11395072,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":8164,"num_read_kb":23386,"num_write":300,"num_write_kb":3250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.287934+0000","last_change":"2024-09-13T09:12:43.533829+0000","last_active":"2024-09-13T09:12:50.287934+0000","last_peered":"2024-09-13T09:12:50.287934+0000","last_clean":"2024-09-13T09:12:50.287934+0000","last_became_active":"2024-09-13T09:12:43.533557+0000","last_became_peered":"2024-09-13T09:12:43.533557+0000","last_unstale":"2024-09-13T09:12:50.287934+0000","last_undegraded":"2024-09-13T09:12:50.287934+0000","last_fullsized":"2024-09-13T09:12:50.287934+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:51.939164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"55'146","reported_seq":2517,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.419981+0000","last_change":"2024-09-13T09:12:16.376528+0000","last_active":"2024-09-13T09:12:51.419981+0000","last_peered":"2024-09-13T09:12:51.419981+0000","last_clean":"2024-09-13T09:12:51.419981+0000","last_became_active":"2024-09-13T09:12:13.367848+0000","last_became_peered":"2024-09-13T09:12:13.367848+0000","last_unstale":"2024-09-13T09:12:51.419981+0000","last_undegraded":"2024-09-13T09:12:51.419981+0000","last_fullsized":"2024-09-13T09:12:51.419981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:21.943266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067051299999999997,"stat_sum":{"num_bytes":39939584,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":3045,"num_read_kb":11648,"num_write":127,"num_write_kb":1170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.284658+0000","last_change":"2024-09-13T09:12:06.328091+0000","last_active":"2024-09-13T09:12:50.284658+0000","last_peered":"2024-09-13T09:12:50.284658+0000","last_clean":"2024-09-13T09:12:50.284658+0000","last_became_active":"2024-09-13T09:12:04.446508+0000","last_became_peered":"2024-09-13T09:12:04.446508+0000","last_unstale":"2024-09-13T09:12:50.284658+0000","last_undegraded":"2024-09-13T09:12:50.284658+0000","last_fullsized":"2024-09-13T09:12:50.284658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:33.410873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042514000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"55'250","reported_seq":610,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.420742+0000","last_change":"2024-09-13T09:12:16.389006+0000","last_active":"2024-09-13T09:12:51.420742+0000","last_peered":"2024-09-13T09:12:51.420742+0000","last_clean":"2024-09-13T09:12:51.420742+0000","last_became_active":"2024-09-13T09:12:13.369489+0000","last_became_peered":"2024-09-13T09:12:13.369489+0000","last_unstale":"2024-09-13T09:12:51.420742+0000","last_undegraded":"2024-09-13T09:12:51.420742+0000","last_fullsized":"2024-09-13T09:12:51.420742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:33.879162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088595999999999998,"stat_sum":{"num_bytes":3100815,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":1210,"num_read_kb":2374,"num_write":369,"num_write_kb":1172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"55'171","reported_seq":4758,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.511548+0000","last_change":"2024-09-13T09:12:16.380981+0000","last_active":"2024-09-13T09:12:51.511548+0000","last_peered":"2024-09-13T09:12:51.511548+0000","last_clean":"2024-09-13T09:12:51.511548+0000","last_became_active":"2024-09-13T09:12:13.367770+0000","last_became_peered":"2024-09-13T09:12:13.367770+0000","last_unstale":"2024-09-13T09:12:51.511548+0000","last_undegraded":"2024-09-13T09:12:51.511548+0000","last_fullsized":"2024-09-13T09:12:51.511548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:21:50.690425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083150800000000001,"stat_sum":{"num_bytes":24312100,"num_objects":32,"num_object_clones":12,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":4602,"num_read_kb":24088,"num_write":179,"num_write_kb":2391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":54,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.303125+0000","last_change":"2024-09-13T09:12:06.330649+0000","last_active":"2024-09-13T09:12:50.303125+0000","last_peered":"2024-09-13T09:12:50.303125+0000","last_clean":"2024-09-13T09:12:50.303125+0000","last_became_active":"2024-09-13T09:12:04.325470+0000","last_became_peered":"2024-09-13T09:12:04.325470+0000","last_unstale":"2024-09-13T09:12:50.303125+0000","last_undegraded":"2024-09-13T09:12:50.303125+0000","last_fullsized":"2024-09-13T09:12:50.303125+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:37.971615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093634899999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"55'144","reported_seq":1559,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.426975+0000","last_change":"2024-09-13T09:12:16.378750+0000","last_active":"2024-09-13T09:12:51.426975+0000","last_peered":"2024-09-13T09:12:51.426975+0000","last_clean":"2024-09-13T09:12:51.426975+0000","last_became_active":"2024-09-13T09:12:13.367656+0000","last_became_peered":"2024-09-13T09:12:13.367656+0000","last_unstale":"2024-09-13T09:12:51.426975+0000","last_undegraded":"2024-09-13T09:12:51.426975+0000","last_fullsized":"2024-09-13T09:12:51.426975+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:11.942294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003044836,"stat_sum":{"num_bytes":15526944,"num_objects":24,"num_object_clones":9,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":1415,"num_read_kb":14662,"num_write":136,"num_write_kb":1483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.284477+0000","last_change":"2024-09-13T09:12:06.845125+0000","last_active":"2024-09-13T09:12:50.284477+0000","last_peered":"2024-09-13T09:12:50.284477+0000","last_clean":"2024-09-13T09:12:50.284477+0000","last_became_active":"2024-09-13T09:12:04.321941+0000","last_became_peered":"2024-09-13T09:12:04.321941+0000","last_unstale":"2024-09-13T09:12:50.284477+0000","last_undegraded":"2024-09-13T09:12:50.284477+0000","last_fullsized":"2024-09-13T09:12:50.284477+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:57.706060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031569600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"55'98","reported_seq":1225,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.416394+0000","last_change":"2024-09-13T09:12:16.380222+0000","last_active":"2024-09-13T09:12:51.416394+0000","last_peered":"2024-09-13T09:12:51.416394+0000","last_clean":"2024-09-13T09:12:51.416394+0000","last_became_active":"2024-09-13T09:12:13.367633+0000","last_became_peered":"2024-09-13T09:12:13.367633+0000","last_unstale":"2024-09-13T09:12:51.416394+0000","last_undegraded":"2024-09-13T09:12:51.416394+0000","last_fullsized":"2024-09-13T09:12:51.416394+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:34.734632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438009,"stat_sum":{"num_bytes":6182912,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":1094,"num_read_kb":8707,"num_write":97,"num_write_kb":968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.287845+0000","last_change":"2024-09-13T09:12:43.532201+0000","last_active":"2024-09-13T09:12:50.287845+0000","last_peered":"2024-09-13T09:12:50.287845+0000","last_clean":"2024-09-13T09:12:50.287845+0000","last_became_active":"2024-09-13T09:12:43.531942+0000","last_became_peered":"2024-09-13T09:12:43.531942+0000","last_unstale":"2024-09-13T09:12:50.287845+0000","last_undegraded":"2024-09-13T09:12:50.287845+0000","last_fullsized":"2024-09-13T09:12:50.287845+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:26.408883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"55'232","reported_seq":1010,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.409444+0000","last_change":"2024-09-13T09:12:16.377544+0000","last_active":"2024-09-13T09:12:51.409444+0000","last_peered":"2024-09-13T09:12:51.409444+0000","last_clean":"2024-09-13T09:12:51.409444+0000","last_became_active":"2024-09-13T09:12:13.361909+0000","last_became_peered":"2024-09-13T09:12:13.361909+0000","last_unstale":"2024-09-13T09:12:51.409444+0000","last_undegraded":"2024-09-13T09:12:51.409444+0000","last_fullsized":"2024-09-13T09:12:51.409444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:34.570438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001546782,"stat_sum":{"num_bytes":3548775,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":1539,"num_read_kb":3778,"num_write":342,"num_write_kb":887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":53,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.303101+0000","last_change":"2024-09-13T09:12:06.330140+0000","last_active":"2024-09-13T09:12:50.303101+0000","last_peered":"2024-09-13T09:12:50.303101+0000","last_clean":"2024-09-13T09:12:50.303101+0000","last_became_active":"2024-09-13T09:12:04.324857+0000","last_became_peered":"2024-09-13T09:12:04.324857+0000","last_unstale":"2024-09-13T09:12:50.303101+0000","last_undegraded":"2024-09-13T09:12:50.303101+0000","last_fullsized":"2024-09-13T09:12:50.303101+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:57.400286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038911000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.288060+0000","last_change":"2024-09-13T09:12:46.248714+0000","last_active":"2024-09-13T09:12:50.288060+0000","last_peered":"2024-09-13T09:12:50.288060+0000","last_clean":"2024-09-13T09:12:50.288060+0000","last_became_active":"2024-09-13T09:12:46.248451+0000","last_became_peered":"2024-09-13T09:12:46.248451+0000","last_unstale":"2024-09-13T09:12:50.288060+0000","last_undegraded":"2024-09-13T09:12:50.288060+0000","last_fullsized":"2024-09-13T09:12:50.288060+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:27.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"55'104","reported_seq":610,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.421862+0000","last_change":"2024-09-13T09:12:16.377642+0000","last_active":"2024-09-13T09:12:51.421862+0000","last_peered":"2024-09-13T09:12:51.421862+0000","last_clean":"2024-09-13T09:12:51.421862+0000","last_became_active":"2024-09-13T09:12:13.369742+0000","last_became_peered":"2024-09-13T09:12:13.369742+0000","last_unstale":"2024-09-13T09:12:51.421862+0000","last_undegraded":"2024-09-13T09:12:51.421862+0000","last_fullsized":"2024-09-13T09:12:51.421862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:27:48.361252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060355399999999996,"stat_sum":{"num_bytes":19369518,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":586,"num_read_kb":3371,"num_write":113,"num_write_kb":2896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"55'329","reported_seq":6351,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.411749+0000","last_change":"2024-09-13T09:12:16.380989+0000","last_active":"2024-09-13T09:12:51.411749+0000","last_peered":"2024-09-13T09:12:51.411749+0000","last_clean":"2024-09-13T09:12:51.411749+0000","last_became_active":"2024-09-13T09:12:13.370096+0000","last_became_peered":"2024-09-13T09:12:13.370096+0000","last_unstale":"2024-09-13T09:12:51.411749+0000","last_undegraded":"2024-09-13T09:12:51.411749+0000","last_fullsized":"2024-09-13T09:12:51.411749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:14.224546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000903458,"stat_sum":{"num_bytes":39831040,"num_objects":22,"num_object_clones":9,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":8500,"num_read_kb":19304,"num_write":220,"num_write_kb":1089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"55'136","reported_seq":1746,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.427613+0000","last_change":"2024-09-13T09:12:46.253134+0000","last_active":"2024-09-13T09:12:51.427613+0000","last_peered":"2024-09-13T09:12:51.427613+0000","last_clean":"2024-09-13T09:12:51.427613+0000","last_became_active":"2024-09-13T09:12:46.252578+0000","last_became_peered":"2024-09-13T09:12:46.252578+0000","last_unstale":"2024-09-13T09:12:51.427613+0000","last_undegraded":"2024-09-13T09:12:51.427613+0000","last_fullsized":"2024-09-13T09:12:51.427613+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:03.931492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041459800000000002,"stat_sum":{"num_bytes":51840068,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":4,"num_read":1569,"num_read_kb":18162,"num_write":150,"num_write_kb":3781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2590736,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"55'101","reported_seq":2074,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.410222+0000","last_change":"2024-09-13T09:12:16.377864+0000","last_active":"2024-09-13T09:12:51.410222+0000","last_peered":"2024-09-13T09:12:51.410222+0000","last_clean":"2024-09-13T09:12:51.410222+0000","last_became_active":"2024-09-13T09:12:13.362409+0000","last_became_peered":"2024-09-13T09:12:13.362409+0000","last_unstale":"2024-09-13T09:12:51.410222+0000","last_undegraded":"2024-09-13T09:12:51.410222+0000","last_fullsized":"2024-09-13T09:12:51.410222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:37.023594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001713904,"stat_sum":{"num_bytes":40788512,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":1961,"num_read_kb":21606,"num_write":139,"num_write_kb":3597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"55'371","reported_seq":4130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.424863+0000","last_change":"2024-09-13T09:12:16.389470+0000","last_active":"2024-09-13T09:12:51.424863+0000","last_peered":"2024-09-13T09:12:51.424863+0000","last_clean":"2024-09-13T09:12:51.424863+0000","last_became_active":"2024-09-13T09:12:13.365758+0000","last_became_peered":"2024-09-13T09:12:13.365758+0000","last_unstale":"2024-09-13T09:12:51.424863+0000","last_undegraded":"2024-09-13T09:12:51.424863+0000","last_fullsized":"2024-09-13T09:12:51.424863+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:46.649879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013123500000000001,"stat_sum":{"num_bytes":39062602,"num_objects":25,"num_object_clones":10,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":5167,"num_read_kb":22878,"num_write":588,"num_write_kb":3406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"55'116","reported_seq":337,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.420419+0000","last_change":"2024-09-13T09:12:43.630169+0000","last_active":"2024-09-13T09:12:51.420419+0000","last_peered":"2024-09-13T09:12:51.420419+0000","last_clean":"2024-09-13T09:12:51.420419+0000","last_became_active":"2024-09-13T09:12:43.533321+0000","last_became_peered":"2024-09-13T09:12:43.533321+0000","last_unstale":"2024-09-13T09:12:51.420419+0000","last_undegraded":"2024-09-13T09:12:51.420419+0000","last_fullsized":"2024-09-13T09:12:51.420419+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:43.973948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000679308,"stat_sum":{"num_bytes":3810347,"num_objects":14,"num_object_clones":6,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":357,"num_read_kb":1959,"num_write":117,"num_write_kb":809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"55'110","reported_seq":1226,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.459723+0000","last_change":"2024-09-13T09:12:16.389243+0000","last_active":"2024-09-13T09:12:51.459723+0000","last_peered":"2024-09-13T09:12:51.459723+0000","last_clean":"2024-09-13T09:12:51.459723+0000","last_became_active":"2024-09-13T09:12:13.362547+0000","last_became_peered":"2024-09-13T09:12:13.362547+0000","last_unstale":"2024-09-13T09:12:51.459723+0000","last_undegraded":"2024-09-13T09:12:51.459723+0000","last_fullsized":"2024-09-13T09:12:51.459723+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:24:24.557400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011202860000000001,"stat_sum":{"num_bytes":2537984,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":1665,"num_read_kb":3468,"num_write":90,"num_write_kb":612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"53'112","reported_seq":827,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.303260+0000","last_change":"2024-09-13T09:12:43.623612+0000","last_active":"2024-09-13T09:12:50.303260+0000","last_peered":"2024-09-13T09:12:50.303260+0000","last_clean":"2024-09-13T09:12:50.303260+0000","last_became_active":"2024-09-13T09:12:43.533088+0000","last_became_peered":"2024-09-13T09:12:43.533088+0000","last_unstale":"2024-09-13T09:12:50.303260+0000","last_undegraded":"2024-09-13T09:12:50.303260+0000","last_fullsized":"2024-09-13T09:12:50.303260+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:43:49.678463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059255799999999997,"stat_sum":{"num_bytes":2678908,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":661,"num_read_kb":1263,"num_write":114,"num_write_kb":1329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":635004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"55'84","reported_seq":827,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.366624+0000","last_change":"2024-09-13T09:12:16.389277+0000","last_active":"2024-09-13T09:12:51.366624+0000","last_peered":"2024-09-13T09:12:51.366624+0000","last_clean":"2024-09-13T09:12:51.366624+0000","last_became_active":"2024-09-13T09:12:13.367757+0000","last_became_peered":"2024-09-13T09:12:13.367757+0000","last_unstale":"2024-09-13T09:12:51.366624+0000","last_undegraded":"2024-09-13T09:12:51.366624+0000","last_fullsized":"2024-09-13T09:12:51.366624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:33.596156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011263219999999999,"stat_sum":{"num_bytes":8565852,"num_objects":12,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":721,"num_read_kb":4898,"num_write":81,"num_write_kb":871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"53'125","reported_seq":434,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T09:12:50.880961+0000","last_change":"2024-09-13T09:12:43.614382+0000","last_active":"2024-09-13T09:12:50.880961+0000","last_peered":"2024-09-13T09:12:50.880961+0000","last_clean":"2024-09-13T09:12:50.880961+0000","last_became_active":"2024-09-13T09:12:43.532504+0000","last_became_peered":"2024-09-13T09:12:43.532504+0000","last_unstale":"2024-09-13T09:12:50.880961+0000","last_undegraded":"2024-09-13T09:12:50.880961+0000","last_fullsized":"2024-09-13T09:12:50.880961+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:25:04.225563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034391499999999998,"stat_sum":{"num_bytes":11360807,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":928,"num_read_kb":11657,"num_write":129,"num_write_kb":1596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":694311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"55'167","reported_seq":7571,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.428666+0000","last_change":"2024-09-13T09:12:46.248982+0000","last_active":"2024-09-13T09:12:51.428666+0000","last_peered":"2024-09-13T09:12:51.428666+0000","last_clean":"2024-09-13T09:12:51.428666+0000","last_became_active":"2024-09-13T09:12:46.248261+0000","last_became_peered":"2024-09-13T09:12:46.248261+0000","last_unstale":"2024-09-13T09:12:51.428666+0000","last_undegraded":"2024-09-13T09:12:51.428666+0000","last_fullsized":"2024-09-13T09:12:51.428666+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:44.056762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049261300000000004,"stat_sum":{"num_bytes":18003968,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":7227,"num_read_kb":19668,"num_write":166,"num_write_kb":1674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":676864,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"55'339","reported_seq":3933,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T09:12:51.478669+0000","last_change":"2024-09-13T09:12:46.251913+0000","last_active":"2024-09-13T09:12:51.478669+0000","last_peered":"2024-09-13T09:12:51.478669+0000","last_clean":"2024-09-13T09:12:51.478669+0000","last_became_active":"2024-09-13T09:12:46.250552+0000","last_became_peered":"2024-09-13T09:12:46.250552+0000","last_unstale":"2024-09-13T09:12:51.478669+0000","last_undegraded":"2024-09-13T09:12:51.478669+0000","last_fullsized":"2024-09-13T09:12:51.478669+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:47.142721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059370899999999999,"stat_sum":{"num_bytes":4122729,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":4142,"num_read_kb":4708,"num_write":317,"num_write_kb":945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":28777,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":632750095,"num_objects":571,"num_object_clones":187,"num_object_copies":1142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":571,"num_whiteouts":170,"num_read":106168,"num_read_kb":423143,"num_write":8446,"num_write_kb":63946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":8739591,"num_keys_recovered":33,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":47992832,"data_stored":52231198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7160,"ondisk_log_size":7160,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640270,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45904,"kb_used_data":6600,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94325936,"statfs":{"total":96636764160,"available":96589758464,"internally_reserved":0,"allocated":6758400,"data_stored":6708214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]}]},{"osd":4,"up_from":11,"seq":47244640271,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":10764,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":11022336,"data_stored":12048839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640272,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43760,"kb_used_data":8552,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94328080,"statfs":{"total":96636764160,"available":96591953920,"internally_reserved":0,"allocated":8757248,"data_stored":9241505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640270,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39780,"kb_used_data":4572,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332060,"statfs":{"total":96636764160,"available":96596029440,"internally_reserved":0,"allocated":4681728,"data_stored":4587364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]}]},{"osd":1,"up_from":11,"seq":47244640271,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42324,"kb_used_data":7116,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94329516,"statfs":{"total":96636764160,"available":96593424384,"internally_reserved":0,"allocated":7286784,"data_stored":7663640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36043205},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]}]},{"osd":0,"up_from":11,"seq":47244640270,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52768,"kb_used_data":13456,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94319072,"statfs":{"total":96636764160,"available":96582729728,"internally_reserved":0,"allocated":13778944,"data_stored":14083048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12562432,"data_stored":13246709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6717440,"data_stored":7461305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4456216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7540736,"data_stored":8405166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10620928,"data_stored":12005807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6356992,"data_stored":6655995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:12:54.745 INFO:tasks.thrashosds.thrasher:Setting 3.1b to [2, 3] 2024-09-13T09:12:54.745 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1b', '2', '3'] 2024-09-13T09:12:54.746 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.1b 2 3 2024-09-13T09:12:54.748 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:54.827 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:54.826+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:54.855 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:54.855 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:54.864 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:54.907 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:54.906+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:54.943 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-13T09:12:54.976 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:54.976 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:54.986 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:55.031 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:55.030+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:55.109 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:55.108+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:55.111 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:55.111 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:55.121 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:55.199 INFO:teuthology.orchestra.run.smithi157.stdout:2771 write 0x601000 thru 0x60d2c6 (0xc2c7 bytes) 2024-09-13T09:12:55.200 INFO:teuthology.orchestra.run.smithi157.stdout:2772 write 0x28c800 thru 0x29784f (0xb050 bytes) 2024-09-13T09:12:55.210 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:55.209+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:55.282 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:55.283 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:55.291 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:55.313 INFO:teuthology.orchestra.run.smithi044.stderr:set 3.1b pg_upmap mapping to [2,3] 2024-09-13T09:12:55.314 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:55.313+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:55.363 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:12:55.412 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:55.411+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:55.432 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:55.432 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:55.441 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:55.476 INFO:teuthology.orchestra.run.smithi157.stdout:2775 write 0xd16da00 thru 0xd173ba5 (0x61a6 bytes) 2024-09-13T09:12:55.476 INFO:teuthology.orchestra.run.smithi157.stdout:2776 read 0x7e74400 thru 0x7e753ed (0xfee bytes) 2024-09-13T09:12:55.477 INFO:teuthology.orchestra.run.smithi157.stdout:2777 punch from 0x24a4c00 to 0x24ac200, (0x7600 bytes) 2024-09-13T09:12:55.480 INFO:teuthology.orchestra.run.smithi157.stdout:2778 write 0x8f4dc00 thru 0x8f5dbf2 (0xfff3 bytes) 2024-09-13T09:12:55.513 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:55.512+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:55.569 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:55.570 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:55.579 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:55.617 INFO:teuthology.orchestra.run.smithi157.stdout:2780 write 0xe1e7600 thru 0xe1eb487 (0x3e88 bytes) 2024-09-13T09:12:55.618 INFO:teuthology.orchestra.run.smithi157.stdout:2781 punch from 0xd9c4a00 to 0xd9cf400, (0xaa00 bytes) 2024-09-13T09:12:55.623 INFO:teuthology.orchestra.run.smithi157.stdout:2782 read 0x3c98c00 thru 0x3c9fd77 (0x7178 bytes) 2024-09-13T09:12:55.623 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:55.622+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:55.631 INFO:teuthology.orchestra.run.smithi157.stdout:2783 punch from 0x2c13c00 to 0x2c16e00, (0x3200 bytes) 2024-09-13T09:12:55.641 INFO:teuthology.orchestra.run.smithi157.stdout:2784 trunc from 0xe1eb600 to 0x60bce00 2024-09-13T09:12:55.693 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:12:55.716 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:55.716 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:55.716 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:55.715+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:55.728 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:55.817 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:55.816+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:55.838 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:55.839 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:55.859 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:55.859 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:55.874 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:55.875 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:55.885 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:55.886 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:55.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:55.886 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:55.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.886 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.886 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:12:55.886 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.380434508, 2024-09-13T09:12:55.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:12:55.886 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.886 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.887 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.887 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.887 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.887 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:12:55.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.887 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:12:55.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:12:55.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:12:55.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:12:55.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:12:55.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:12:55.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:12:55.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:12:55.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:12:55.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:12:55.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:12:55.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:12:55.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:12:55.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:12:55.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:12:55.890 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.891 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.891 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.891 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.891 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:12:55.891 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.380406824, 2024-09-13T09:12:55.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:12:55.891 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.891 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.891 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.892 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.892 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.892 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:12:55.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.892 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:12:55.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:12:55.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:12:55.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:12:55.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:12:55.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:12:55.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:12:55.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:12:55.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:12:55.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:12:55.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:12:55.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:12:55.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:12:55.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:12:55.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:12:55.896 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.896 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.896 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.896 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.896 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:12:55.896 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.380382483, 2024-09-13T09:12:55.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:12:55.896 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.896 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.897 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.897 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.897 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.897 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:12:55.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.897 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:12:55.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:12:55.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:12:55.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:12:55.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:12:55.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:12:55.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:12:55.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:12:55.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:12:55.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:12:55.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:12:55.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:12:55.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:12:55.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:12:55.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:12:55.900 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.900 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.900 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.901 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.901 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:12:55.901 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.380366163, 2024-09-13T09:12:55.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:12:55.901 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.901 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.901 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.901 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.901 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.901 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:12:55.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.902 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:12:55.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:12:55.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:12:55.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:12:55.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:12:55.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:12:55.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:12:55.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:12:55.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:12:55.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:12:55.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:12:55.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:12:55.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:12:55.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:12:55.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:12:55.905 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.905 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.905 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.906 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.906 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:12:55.906 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.380349748, 2024-09-13T09:12:55.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:12:55.906 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.906 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.906 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.906 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.906 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.906 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:12:55.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.906 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:12:55.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:12:55.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:12:55.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:12:55.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:12:55.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:12:55.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:12:55.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:12:55.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:12:55.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:12:55.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:12:55.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:12:55.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:12:55.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:12:55.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:12:55.910 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.910 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.910 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.910 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.910 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:12:55.910 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.38033364, 2024-09-13T09:12:55.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:12:55.910 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.911 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.911 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.911 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.911 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.911 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:12:55.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.911 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:12:55.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:12:55.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:12:55.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:12:55.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:12:55.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:12:55.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:12:55.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:12:55.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:12:55.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:12:55.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:12:55.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:12:55.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:12:55.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:12:55.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:12:55.915 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.915 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.915 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.915 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.915 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:12:55.915 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.380318079, 2024-09-13T09:12:55.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:12:55.915 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.915 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.915 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.915 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.916 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.916 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:12:55.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.916 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:12:55.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:12:55.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:12:55.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:12:55.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:12:55.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:12:55.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:12:55.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:12:55.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:12:55.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:12:55.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:12:55.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.380301241, 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:12:55.919 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.920 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.920 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.920 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.920 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.920 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:12:55.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.920 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:12:55.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:12:55.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:12:55.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:12:55.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:12:55.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:12:55.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:12:55.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:12:55.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:12:55.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:12:55.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:12:55.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:12:55.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:12:55.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:12:55.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:12:55.923 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.923 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.923 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.924 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.924 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:12:55.924 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.380285249, 2024-09-13T09:12:55.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:12:55.924 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.924 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.924 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.924 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.924 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.924 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:12:55.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.924 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:12:55.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:12:55.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:12:55.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:12:55.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:12:55.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:12:55.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:12:55.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:12:55.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:12:55.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:12:55.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:12:55.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:12:55.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:12:55.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:12:55.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:12:55.928 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.928 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.928 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.928 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.928 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:12:55.928 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.380257751, 2024-09-13T09:12:55.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:12:55.929 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.929 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.929 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.929 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.929 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.929 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:12:55.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.929 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.929 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.929 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:12:55.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:12:55.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:12:55.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:12:55.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:12:55.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:12:55.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:12:55.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:12:55.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:12:55.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:12:55.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:12:55.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:12:55.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:12:55.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:12:55.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:12:55.933 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.933 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.933 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.933 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.933 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:12:55.934 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.380231891999999, 2024-09-13T09:12:55.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:12:55.934 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.934 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.934 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.934 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.934 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.934 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:12:55.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.934 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:12:55.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:12:55.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:12:55.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:12:55.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:12:55.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:12:55.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:12:55.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:12:55.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:12:55.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:12:55.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:12:55.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:12:55.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:12:55.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:12:55.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:12:55.938 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.938 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.938 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.938 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.938 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:12:55.938 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.380190402, 2024-09-13T09:12:55.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:12:55.938 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.938 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.939 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.939 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.939 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.939 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:12:55.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.939 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:12:55.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:12:55.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:12:55.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:12:55.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:12:55.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.941 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:12:55.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:12:55.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.941 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:12:55.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:12:55.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:12:55.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:12:55.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:12:55.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:12:55.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:12:55.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:12:55.942 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.942 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.943 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.943 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.943 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:12:55.943 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.380169164, 2024-09-13T09:12:55.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:12:55.943 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.943 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.943 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.943 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.943 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.943 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:12:55.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.944 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:12:55.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:12:55.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:12:55.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:12:55.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:12:55.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:12:55.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:12:55.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:12:55.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:12:55.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:12:55.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:12:55.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:12:55.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:12:55.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:12:55.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:12:55.947 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.947 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.947 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.948 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.948 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:12:55.948 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.380146364, 2024-09-13T09:12:55.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:12:55.948 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.948 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.948 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.948 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.948 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.948 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:12:55.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.948 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:12:55.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:12:55.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:12:55.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:12:55.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:12:55.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:12:55.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:12:55.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:12:55.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:12:55.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:12:55.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:12:55.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:12:55.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:12:55.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:12:55.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:12:55.952 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.952 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.952 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.952 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.953 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:12:55.953 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.380129908000001, 2024-09-13T09:12:55.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:12:55.953 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.953 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.953 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.953 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.953 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.953 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:12:55.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.953 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:12:55.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:12:55.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:12:55.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:12:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:12:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:12:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:12:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:12:55.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:12:55.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:12:55.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:12:55.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:12:55.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:12:55.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:12:55.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:12:55.957 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.957 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.957 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.957 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.957 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:12:55.957 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.379884771, 2024-09-13T09:12:55.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:12:55.958 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.958 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.958 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.958 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.958 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.958 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:12:55.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.958 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:12:55.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:12:55.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:12:55.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:12:55.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:12:55.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:12:55.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:12:55.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:12:55.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:12:55.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:12:55.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:12:55.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:12:55.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:12:55.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:12:55.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:12:55.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:12:55.962 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.962 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.962 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.962 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20408 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5582848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.962 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.505992+0000", 2024-09-13T09:12:55.962 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.378020190999999, 2024-09-13T09:12:55.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050463591000000002, 2024-09-13T09:12:55.963 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.963 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.963 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.963 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.963 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.963 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20408 2024-09-13T09:12:55.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.963 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505992+0000", 2024-09-13T09:12:55.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505992+0000", 2024-09-13T09:12:55.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505991+0000", 2024-09-13T09:12:55.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505994+0000", 2024-09-13T09:12:55.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6259999999999999e-06 2024-09-13T09:12:55.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506002+0000", 2024-09-13T09:12:55.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1550000000000007e-06 2024-09-13T09:12:55.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506008+0000", 2024-09-13T09:12:55.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.553e-06 2024-09-13T09:12:55.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556375+0000", 2024-09-13T09:12:55.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050366850999999997 2024-09-13T09:12:55.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556412+0000", 2024-09-13T09:12:55.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7138999999999998e-05 2024-09-13T09:12:55.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556455+0000", 2024-09-13T09:12:55.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3733999999999998e-05 2024-09-13T09:12:55.967 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.967 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.967 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.967 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.967 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:12:55.967 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.377999212000001, 2024-09-13T09:12:55.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:12:55.967 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.967 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.967 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.968 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.968 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.968 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:12:55.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.968 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:12:55.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:12:55.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:12:55.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:12:55.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:12:55.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:12:55.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:12:55.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:12:55.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:12:55.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:12:55.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:12:55.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:12:55.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:12:55.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:12:55.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:12:55.971 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.971 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.971 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.972 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.972 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:12:55.972 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.377977846, 2024-09-13T09:12:55.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:12:55.972 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.972 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.972 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.972 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.972 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.972 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:12:55.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.972 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:12:55.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:12:55.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:12:55.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:55.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:12:55.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:12:55.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:12:55.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:12:55.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:12:55.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:12:55.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:12:55.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:12:55.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:12:55.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:12:55.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:12:55.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:12:55.976 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.976 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.976 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.976 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:55.976 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:12:55.976 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.377772478000001, 2024-09-13T09:12:55.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:12:55.976 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:55.977 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:55.977 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:55.977 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:55.977 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:55.977 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:12:55.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.977 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:55.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:55.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:12:55.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:55.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:12:55.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:55.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:55.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:12:55.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:12:55.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:55.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:12:55.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:12:55.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:55.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:12:55.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:12:55.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:55.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:12:55.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:12:55.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:55.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:12:55.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:12:55.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:55.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:12:55.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:12:55.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:55.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:55.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:55.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:12:55.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:12:55.980 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.980 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.980 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.980 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:55.981 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:55.981 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:55.982 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:55.982 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:55.983 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:55.983 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:55.984 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:55.984 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:55.981+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:56.083 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:56.082+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:56.091 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:56.091 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:56.091 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:56.091 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:56.099 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:56.130 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:56.130 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:56.184 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:56.183+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:56.206 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:56.207 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:56.207 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:56.207 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:56.207 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:56.215 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:56.301 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:56.300+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:56.319 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-09-13T09:12:56.367 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:56.367 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:12:56.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:12:56.367 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:12:56.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:6867 3.13 3:cd293326:::rbd_header.112326fb43d3:head [watch reconnect cookie 140383087120544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T09:12:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:12:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.8362006, 2024-09-13T09:12:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0277181360000001, 2024-09-13T09:12:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:12:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:12:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6867 2024-09-13T09:12:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:12:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:12:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529067+0000", 2024-09-13T09:12:56.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-09-13T09:12:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529073+0000", 2024-09-13T09:12:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:12:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529087+0000", 2024-09-13T09:12:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3825000000000001e-05 2024-09-13T09:12:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529097+0000", 2024-09-13T09:12:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9559999999999992e-06 2024-09-13T09:12:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529149+0000", 2024-09-13T09:12:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1668000000000001e-05 2024-09-13T09:12:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529161+0000", 2024-09-13T09:12:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1831000000000001e-05 2024-09-13T09:12:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.554941+0000", 2024-09-13T09:12:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0257799729999999 2024-09-13T09:12:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.555239+0000", 2024-09-13T09:12:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00029829100000000001 2024-09-13T09:12:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.556787+0000", 2024-09-13T09:12:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001547617 2024-09-13T09:12:56.372 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20898 3.13 3:cd293326:::rbd_header.112326fb43d3:head [watch reconnect cookie 140642395970896 gen 16] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T09:12:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:12:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.831537782, 2024-09-13T09:12:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0236151950000001, 2024-09-13T09:12:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:12:56.373 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20898 2024-09-13T09:12:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:12:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:12:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533729+0000", 2024-09-13T09:12:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-09-13T09:12:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533736+0000", 2024-09-13T09:12:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9670000000000004e-06 2024-09-13T09:12:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533749+0000", 2024-09-13T09:12:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2449000000000001e-05 2024-09-13T09:12:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533757+0000", 2024-09-13T09:12:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:12:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533813+0000", 2024-09-13T09:12:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5683000000000002e-05 2024-09-13T09:12:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533823+0000", 2024-09-13T09:12:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0168e-05 2024-09-13T09:12:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.555940+0000", 2024-09-13T09:12:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0221170559999999 2024-09-13T09:12:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.556047+0000", 2024-09-13T09:12:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000106881 2024-09-13T09:12:56.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.557347+0000", 2024-09-13T09:12:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001299323 2024-09-13T09:12:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1898 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798886+0000", 2024-09-13T09:12:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.5663829170000003, 2024-09-13T09:12:56.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014226477, 2024-09-13T09:12:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1898 2024-09-13T09:12:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798886+0000", 2024-09-13T09:12:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.379 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798886+0000", 2024-09-13T09:12:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798886+0000", 2024-09-13T09:12:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798895+0000", 2024-09-13T09:12:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7429999999999997e-06 2024-09-13T09:12:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798903+0000", 2024-09-13T09:12:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3180000000000001e-06 2024-09-13T09:12:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798908+0000", 2024-09-13T09:12:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2680000000000002e-06 2024-09-13T09:12:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.812091+0000", 2024-09-13T09:12:56.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013183122 2024-09-13T09:12:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.812162+0000", 2024-09-13T09:12:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0995999999999996e-05 2024-09-13T09:12:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813113+0000", 2024-09-13T09:12:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00095045500000000001 2024-09-13T09:12:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.382 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1899 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798913+0000", 2024-09-13T09:12:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.5663557309999998, 2024-09-13T09:12:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014390494, 2024-09-13T09:12:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.383 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1899 2024-09-13T09:12:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798913+0000", 2024-09-13T09:12:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798913+0000", 2024-09-13T09:12:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798913+0000", 2024-09-13T09:12:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798915+0000", 2024-09-13T09:12:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1890000000000001e-06 2024-09-13T09:12:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798921+0000", 2024-09-13T09:12:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2910000000000004e-06 2024-09-13T09:12:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798926+0000", 2024-09-13T09:12:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8740000000000003e-06 2024-09-13T09:12:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813139+0000", 2024-09-13T09:12:56.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014213046 2024-09-13T09:12:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813216+0000", 2024-09-13T09:12:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6464000000000005e-05 2024-09-13T09:12:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813304+0000", 2024-09-13T09:12:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8059999999999994e-05 2024-09-13T09:12:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1900 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798931+0000", 2024-09-13T09:12:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.5663383789999994, 2024-09-13T09:12:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014961771, 2024-09-13T09:12:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.388 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1900 2024-09-13T09:12:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798931+0000", 2024-09-13T09:12:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798931+0000", 2024-09-13T09:12:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798930+0000", 2024-09-13T09:12:56.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798932+0000", 2024-09-13T09:12:56.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1110000000000002e-06 2024-09-13T09:12:56.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798938+0000", 2024-09-13T09:12:56.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0140000000000004e-06 2024-09-13T09:12:56.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798943+0000", 2024-09-13T09:12:56.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:12:56.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813330+0000", 2024-09-13T09:12:56.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014387525999999999 2024-09-13T09:12:56.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813407+0000", 2024-09-13T09:12:56.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6341999999999995e-05 2024-09-13T09:12:56.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.392 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.392 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813892+0000", 2024-09-13T09:12:56.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00048584499999999999 2024-09-13T09:12:56.392 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.392 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.392 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.392 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1901 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 139264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.392 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798947+0000", 2024-09-13T09:12:56.393 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.5663221499999995, 2024-09-13T09:12:56.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015125339, 2024-09-13T09:12:56.393 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.393 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.393 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.393 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.393 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.393 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1901 2024-09-13T09:12:56.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.393 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.393 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798947+0000", 2024-09-13T09:12:56.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798947+0000", 2024-09-13T09:12:56.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798947+0000", 2024-09-13T09:12:56.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798949+0000", 2024-09-13T09:12:56.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0420000000000001e-06 2024-09-13T09:12:56.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798955+0000", 2024-09-13T09:12:56.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.879e-06 2024-09-13T09:12:56.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798959+0000", 2024-09-13T09:12:56.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4299999999999999e-06 2024-09-13T09:12:56.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.396 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.813920+0000", 2024-09-13T09:12:56.396 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014961502 2024-09-13T09:12:56.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.396 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814007+0000", 2024-09-13T09:12:56.396 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6643000000000004e-05 2024-09-13T09:12:56.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.396 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814072+0000", 2024-09-13T09:12:56.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5183000000000003e-05 2024-09-13T09:12:56.397 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.397 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.397 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.397 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1902 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 157696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.397 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798963+0000", 2024-09-13T09:12:56.397 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.5663058779999997, 2024-09-13T09:12:56.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01528532, 2024-09-13T09:12:56.397 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.397 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.397 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1902 2024-09-13T09:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798963+0000", 2024-09-13T09:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798963+0000", 2024-09-13T09:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798963+0000", 2024-09-13T09:12:56.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798965+0000", 2024-09-13T09:12:56.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9750000000000001e-06 2024-09-13T09:12:56.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798971+0000", 2024-09-13T09:12:56.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:12:56.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.400 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798975+0000", 2024-09-13T09:12:56.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3479999999999997e-06 2024-09-13T09:12:56.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.400 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814104+0000", 2024-09-13T09:12:56.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015128697 2024-09-13T09:12:56.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.401 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.401 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814181+0000", 2024-09-13T09:12:56.401 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7319999999999998e-05 2024-09-13T09:12:56.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.401 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.401 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814249+0000", 2024-09-13T09:12:56.401 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7436999999999998e-05 2024-09-13T09:12:56.401 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.401 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.401 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.402 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1903 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 192512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.402 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798979+0000", 2024-09-13T09:12:56.402 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.5662898110000008, 2024-09-13T09:12:56.402 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015442285, 2024-09-13T09:12:56.402 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.402 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.402 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.402 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.402 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.402 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1903 2024-09-13T09:12:56.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.402 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798979+0000", 2024-09-13T09:12:56.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798979+0000", 2024-09-13T09:12:56.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798979+0000", 2024-09-13T09:12:56.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.404 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.404 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798981+0000", 2024-09-13T09:12:56.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9479999999999998e-06 2024-09-13T09:12:56.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.404 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.404 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798987+0000", 2024-09-13T09:12:56.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8159999999999999e-06 2024-09-13T09:12:56.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.404 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.404 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798991+0000", 2024-09-13T09:12:56.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4950000000000002e-06 2024-09-13T09:12:56.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.405 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814278+0000", 2024-09-13T09:12:56.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015286978999999999 2024-09-13T09:12:56.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.405 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814358+0000", 2024-09-13T09:12:56.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.941e-05 2024-09-13T09:12:56.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814422+0000", 2024-09-13T09:12:56.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3979000000000005e-05 2024-09-13T09:12:56.406 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.406 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.406 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.406 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1904 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 210944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.406 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.798995+0000", 2024-09-13T09:12:56.406 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.5662737080000007, 2024-09-13T09:12:56.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015597725999999999, 2024-09-13T09:12:56.406 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.407 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.407 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.407 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.407 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.407 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1904 2024-09-13T09:12:56.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.407 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798995+0000", 2024-09-13T09:12:56.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798995+0000", 2024-09-13T09:12:56.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798995+0000", 2024-09-13T09:12:56.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.798997+0000", 2024-09-13T09:12:56.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.84e-06 2024-09-13T09:12:56.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799002+0000", 2024-09-13T09:12:56.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3809999999999998e-06 2024-09-13T09:12:56.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799007+0000", 2024-09-13T09:12:56.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3630000000000004e-06 2024-09-13T09:12:56.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814448+0000", 2024-09-13T09:12:56.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015441041000000001 2024-09-13T09:12:56.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814525+0000", 2024-09-13T09:12:56.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7539000000000004e-05 2024-09-13T09:12:56.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814593+0000", 2024-09-13T09:12:56.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8009000000000006e-05 2024-09-13T09:12:56.410 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.410 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.410 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.410 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:1905 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.411 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:47.799011+0000", 2024-09-13T09:12:56.411 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.5662585749999991, 2024-09-13T09:12:56.411 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015754799, 2024-09-13T09:12:56.411 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.411 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.411 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.411 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.411 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.411 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1905 2024-09-13T09:12:56.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.411 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.411 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.411 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799011+0000", 2024-09-13T09:12:56.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799011+0000", 2024-09-13T09:12:56.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799010+0000", 2024-09-13T09:12:56.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799012+0000", 2024-09-13T09:12:56.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9570000000000001e-06 2024-09-13T09:12:56.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799018+0000", 2024-09-13T09:12:56.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9689999999999999e-06 2024-09-13T09:12:56.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.799022+0000", 2024-09-13T09:12:56.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1899999999999997e-06 2024-09-13T09:12:56.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814620+0000", 2024-09-13T09:12:56.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015597313 2024-09-13T09:12:56.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814701+0000", 2024-09-13T09:12:56.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1403999999999995e-05 2024-09-13T09:12:56.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:47.814765+0000", 2024-09-13T09:12:56.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4302999999999997e-05 2024-09-13T09:12:56.415 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.415 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.415 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.415 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:6512 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.415 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:48.202079+0000", 2024-09-13T09:12:56.415 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.1631905010000008, 2024-09-13T09:12:56.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013761887, 2024-09-13T09:12:56.415 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.415 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.415 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.415 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.416 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.416 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6512 2024-09-13T09:12:56.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.416 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.202079+0000", 2024-09-13T09:12:56.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.202079+0000", 2024-09-13T09:12:56.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.417 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.417 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.202078+0000", 2024-09-13T09:12:56.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.417 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.417 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.202080+0000", 2024-09-13T09:12:56.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9999999999999999e-06 2024-09-13T09:12:56.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.418 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.418 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.202086+0000", 2024-09-13T09:12:56.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4489999999999998e-06 2024-09-13T09:12:56.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.418 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.418 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.202090+0000", 2024-09-13T09:12:56.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3710000000000002e-06 2024-09-13T09:12:56.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.418 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.418 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.215686+0000", 2024-09-13T09:12:56.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013596433 2024-09-13T09:12:56.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.215763+0000", 2024-09-13T09:12:56.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6836999999999996e-05 2024-09-13T09:12:56.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.215840+0000", 2024-09-13T09:12:56.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7138999999999994e-05 2024-09-13T09:12:56.419 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.419 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.419 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.420 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:6923 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.420 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:48.205153+0000", 2024-09-13T09:12:56.420 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.1601162209999991, 2024-09-13T09:12:56.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013787676, 2024-09-13T09:12:56.420 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.420 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.420 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.420 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.420 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.420 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6923 2024-09-13T09:12:56.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.420 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205153+0000", 2024-09-13T09:12:56.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205153+0000", 2024-09-13T09:12:56.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205152+0000", 2024-09-13T09:12:56.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205155+0000", 2024-09-13T09:12:56.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6520000000000002e-06 2024-09-13T09:12:56.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205162+0000", 2024-09-13T09:12:56.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:12:56.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205170+0000", 2024-09-13T09:12:56.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0239999999999997e-06 2024-09-13T09:12:56.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.218771+0000", 2024-09-13T09:12:56.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013600432000000001 2024-09-13T09:12:56.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.218853+0000", 2024-09-13T09:12:56.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2329000000000004e-05 2024-09-13T09:12:56.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.218941+0000", 2024-09-13T09:12:56.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7488e-05 2024-09-13T09:12:56.424 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.424 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.424 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.424 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:6924 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 112640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.424 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:48.205179+0000", 2024-09-13T09:12:56.424 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.1600905879999992, 2024-09-13T09:12:56.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013952529, 2024-09-13T09:12:56.424 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.424 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.424 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.425 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.425 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.425 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6924 2024-09-13T09:12:56.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.425 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205179+0000", 2024-09-13T09:12:56.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205179+0000", 2024-09-13T09:12:56.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205178+0000", 2024-09-13T09:12:56.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205181+0000", 2024-09-13T09:12:56.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5909999999999998e-06 2024-09-13T09:12:56.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205188+0000", 2024-09-13T09:12:56.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0670000000000003e-06 2024-09-13T09:12:56.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205193+0000", 2024-09-13T09:12:56.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8609999999999997e-06 2024-09-13T09:12:56.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.218968+0000", 2024-09-13T09:12:56.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013775351 2024-09-13T09:12:56.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219044+0000", 2024-09-13T09:12:56.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6507000000000005e-05 2024-09-13T09:12:56.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219131+0000", 2024-09-13T09:12:56.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6596999999999998e-05 2024-09-13T09:12:56.428 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.428 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.429 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.429 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:6925 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.429 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:48.205197+0000", 2024-09-13T09:12:56.429 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.1600716779999996, 2024-09-13T09:12:56.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014097465, 2024-09-13T09:12:56.429 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.429 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.429 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.429 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.429 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.430 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6925 2024-09-13T09:12:56.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.430 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.430 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.430 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205197+0000", 2024-09-13T09:12:56.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.430 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.430 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205197+0000", 2024-09-13T09:12:56.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205197+0000", 2024-09-13T09:12:56.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205199+0000", 2024-09-13T09:12:56.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1270000000000001e-06 2024-09-13T09:12:56.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205205+0000", 2024-09-13T09:12:56.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1659999999999999e-06 2024-09-13T09:12:56.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205210+0000", 2024-09-13T09:12:56.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:12:56.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219157+0000", 2024-09-13T09:12:56.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013946248 2024-09-13T09:12:56.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219232+0000", 2024-09-13T09:12:56.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5809000000000006e-05 2024-09-13T09:12:56.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219295+0000", 2024-09-13T09:12:56.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2376e-05 2024-09-13T09:12:56.433 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.433 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.433 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.433 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:6926 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 165888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.434 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:48.205215+0000", 2024-09-13T09:12:56.434 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.1600539179999991, 2024-09-13T09:12:56.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014240512, 2024-09-13T09:12:56.434 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.434 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.434 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.434 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.434 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.434 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6926 2024-09-13T09:12:56.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.434 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205215+0000", 2024-09-13T09:12:56.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205215+0000", 2024-09-13T09:12:56.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205215+0000", 2024-09-13T09:12:56.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205217+0000", 2024-09-13T09:12:56.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3180000000000001e-06 2024-09-13T09:12:56.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205223+0000", 2024-09-13T09:12:56.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2310000000000001e-06 2024-09-13T09:12:56.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205228+0000", 2024-09-13T09:12:56.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7879999999999997e-06 2024-09-13T09:12:56.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219320+0000", 2024-09-13T09:12:56.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01409183 2024-09-13T09:12:56.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219392+0000", 2024-09-13T09:12:56.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2490000000000006e-05 2024-09-13T09:12:56.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219456+0000", 2024-09-13T09:12:56.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3368999999999996e-05 2024-09-13T09:12:56.438 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.438 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.438 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.438 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:6927 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 184320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.438 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:48.205233+0000", 2024-09-13T09:12:56.438 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.1600365349999997, 2024-09-13T09:12:56.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014385121000000001, 2024-09-13T09:12:56.438 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.438 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.438 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.438 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.439 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.439 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6927 2024-09-13T09:12:56.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.439 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205233+0000", 2024-09-13T09:12:56.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205233+0000", 2024-09-13T09:12:56.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205232+0000", 2024-09-13T09:12:56.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205234+0000", 2024-09-13T09:12:56.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1210000000000001e-06 2024-09-13T09:12:56.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205240+0000", 2024-09-13T09:12:56.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:12:56.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205245+0000", 2024-09-13T09:12:56.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7199999999999997e-06 2024-09-13T09:12:56.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219481+0000", 2024-09-13T09:12:56.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014235952 2024-09-13T09:12:56.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219553+0000", 2024-09-13T09:12:56.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2138999999999995e-05 2024-09-13T09:12:56.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.442 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.442 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219618+0000", 2024-09-13T09:12:56.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4391000000000001e-05 2024-09-13T09:12:56.442 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.442 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.442 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.442 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:6928 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 219136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.442 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:48.205250+0000", 2024-09-13T09:12:56.442 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.1600194889999997, 2024-09-13T09:12:56.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014531452, 2024-09-13T09:12:56.443 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.443 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.443 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.443 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.443 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.443 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6928 2024-09-13T09:12:56.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.443 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.443 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205250+0000", 2024-09-13T09:12:56.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.444 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205250+0000", 2024-09-13T09:12:56.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.444 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205249+0000", 2024-09-13T09:12:56.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205251+0000", 2024-09-13T09:12:56.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0559999999999999e-06 2024-09-13T09:12:56.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205257+0000", 2024-09-13T09:12:56.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2400000000000004e-06 2024-09-13T09:12:56.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205262+0000", 2024-09-13T09:12:56.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:12:56.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219643+0000", 2024-09-13T09:12:56.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014381335 2024-09-13T09:12:56.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219719+0000", 2024-09-13T09:12:56.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:12:56.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219781+0000", 2024-09-13T09:12:56.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1641000000000002e-05 2024-09-13T09:12:56.447 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.447 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.447 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.447 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:6929 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.447 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:48.205266+0000", 2024-09-13T09:12:56.447 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.1600025939999998, 2024-09-13T09:12:56.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014683392999999999, 2024-09-13T09:12:56.447 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.447 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.448 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.448 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.448 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.448 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6929 2024-09-13T09:12:56.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.448 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205266+0000", 2024-09-13T09:12:56.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205266+0000", 2024-09-13T09:12:56.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205266+0000", 2024-09-13T09:12:56.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205268+0000", 2024-09-13T09:12:56.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.074e-06 2024-09-13T09:12:56.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.450 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205274+0000", 2024-09-13T09:12:56.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.054e-06 2024-09-13T09:12:56.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.450 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205279+0000", 2024-09-13T09:12:56.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:12:56.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.450 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219807+0000", 2024-09-13T09:12:56.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014527563 2024-09-13T09:12:56.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219887+0000", 2024-09-13T09:12:56.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0644999999999998e-05 2024-09-13T09:12:56.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219950+0000", 2024-09-13T09:12:56.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2706000000000006e-05 2024-09-13T09:12:56.451 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.451 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.451 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.452 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4957.0:6930 3.7 3:e2565498:::rbd_data.112326fb43d3.0000000000000007:11 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-13T09:12:56.452 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:48.205290+0000", 2024-09-13T09:12:56.452 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.1599794639999992, 2024-09-13T09:12:56.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014829362, 2024-09-13T09:12:56.452 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.452 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:12:56.452 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.452 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4957", 2024-09-13T09:12:56.452 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2731225161", 2024-09-13T09:12:56.452 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6930 2024-09-13T09:12:56.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.453 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205290+0000", 2024-09-13T09:12:56.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205290+0000", 2024-09-13T09:12:56.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205289+0000", 2024-09-13T09:12:56.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205292+0000", 2024-09-13T09:12:56.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:12:56.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205298+0000", 2024-09-13T09:12:56.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6579999999999996e-06 2024-09-13T09:12:56.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.455 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.455 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.205304+0000", 2024-09-13T09:12:56.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2209999999999996e-06 2024-09-13T09:12:56.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.455 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.455 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.219976+0000", 2024-09-13T09:12:56.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014671899 2024-09-13T09:12:56.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.456 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.220052+0000", 2024-09-13T09:12:56.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6138999999999997e-05 2024-09-13T09:12:56.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.456 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:48.220119+0000", 2024-09-13T09:12:56.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7287999999999997e-05 2024-09-13T09:12:56.456 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.456 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.456 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.457 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5050.0:1966 3.8 3:102dac2d:::rbd_data.11238aebf726.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 14155776~16252928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:12:56.457 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:52.977151+0000", 2024-09-13T09:12:56.457 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.388118097, 2024-09-13T09:12:56.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.028432987, 2024-09-13T09:12:56.457 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:12:56.457 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:12:56.457 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:12:56.457 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5050", 2024-09-13T09:12:56.457 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3978981581", 2024-09-13T09:12:56.457 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1966 2024-09-13T09:12:56.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.458 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:12:56.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:12:56.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:52.977151+0000", 2024-09-13T09:12:56.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:12:56.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:52.977151+0000", 2024-09-13T09:12:56.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:12:56.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:12:56.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:52.977149+0000", 2024-09-13T09:12:56.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-09-13T09:12:56.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:12:56.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:52.983999+0000", 2024-09-13T09:12:56.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068496609999999999 2024-09-13T09:12:56.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:12:56.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:52.984024+0000", 2024-09-13T09:12:56.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4692999999999999e-05 2024-09-13T09:12:56.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.460 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:52.984033+0000", 2024-09-13T09:12:56.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6849999999999996e-06 2024-09-13T09:12:56.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.460 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:12:56.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:52.984083+0000", 2024-09-13T09:12:56.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9206999999999998e-05 2024-09-13T09:12:56.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.460 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:12:56.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:52.984197+0000", 2024-09-13T09:12:56.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000114837 2024-09-13T09:12:56.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:12:56.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:52.984724+0000", 2024-09-13T09:12:56.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052602200000000001 2024-09-13T09:12:56.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:12:56.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:53.005071+0000", 2024-09-13T09:12:56.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.020347700999999999 2024-09-13T09:12:56.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:12:56.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:53.005480+0000", 2024-09-13T09:12:56.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00040853100000000001 2024-09-13T09:12:56.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:12:56.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:53.005518+0000", 2024-09-13T09:12:56.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7864999999999997e-05 2024-09-13T09:12:56.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:12:56.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:12:56.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:12:56.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:53.005584+0000", 2024-09-13T09:12:56.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6379999999999998e-05 2024-09-13T09:12:56.462 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.462 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.462 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.462 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:12:56.463 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:12:56.463 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:56.464 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:56.465 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:56.462+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:56.565 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:56.564+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:56.577 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:56.577 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:56.577 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:56.577 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:56.586 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:56.666 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:56.664+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:56.688 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:56.688 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:56.689 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:56.689 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:56.689 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:56.697 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:56.767 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:56.766+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:56.841 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:56.841 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:56.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:56.842 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:56.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.842 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.842 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:12:56.842 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.318375555999999, 2024-09-13T09:12:56.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:12:56.842 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.842 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.842 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.842 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.842 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.842 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:12:56.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.843 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:12:56.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:12:56.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:12:56.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:12:56.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:12:56.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:12:56.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:12:56.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:12:56.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:12:56.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:12:56.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:12:56.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:12:56.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:12:56.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:12:56.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:12:56.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:12:56.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:12:56.847 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.847 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.847 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.847 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.847 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:12:56.847 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.3183265, 2024-09-13T09:12:56.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:12:56.847 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.847 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.848 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:12:56.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.848 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:12:56.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:12:56.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:12:56.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:12:56.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:12:56.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:12:56.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:12:56.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:12:56.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:12:56.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:12:56.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:12:56.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:12:56.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:12:56.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:12:56.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:12:56.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:12:56.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:12:56.851 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.851 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.852 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.852 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.852 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:12:56.852 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.318286425, 2024-09-13T09:12:56.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:12:56.852 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.852 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.852 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:12:56.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.853 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:12:56.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:12:56.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:12:56.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:12:56.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:12:56.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:12:56.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:12:56.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:12:56.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:12:56.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:12:56.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:12:56.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:12:56.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:12:56.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:12:56.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:12:56.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:12:56.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:12:56.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.856 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.857 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.857 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:12:56.857 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.318256610000001, 2024-09-13T09:12:56.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:12:56.857 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.857 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.857 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.857 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.857 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.857 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:12:56.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.857 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:12:56.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:12:56.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:12:56.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:12:56.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:12:56.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:12:56.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:12:56.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:12:56.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:12:56.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:12:56.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:12:56.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:12:56.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:12:56.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:12:56.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:12:56.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:12:56.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:12:56.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.862 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.862 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.862 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:12:56.862 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.318226949, 2024-09-13T09:12:56.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:12:56.862 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.862 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.862 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.863 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:12:56.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.863 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:12:56.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:12:56.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:12:56.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:12:56.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:12:56.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:12:56.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:12:56.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:12:56.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:12:56.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:12:56.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:12:56.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:12:56.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:12:56.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:12:56.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:12:56.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:12:56.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:12:56.867 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.867 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.867 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.867 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.867 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:12:56.867 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.318197742000001, 2024-09-13T09:12:56.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:12:56.867 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.868 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.868 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:12:56.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.868 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:12:56.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:12:56.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:12:56.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:12:56.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:12:56.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:12:56.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:12:56.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:12:56.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:12:56.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:12:56.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:12:56.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:12:56.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:12:56.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:12:56.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:12:56.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:12:56.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:12:56.874 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.874 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.875 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.875 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.875 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:12:56.875 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.318141561000001, 2024-09-13T09:12:56.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:12:56.875 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.875 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.875 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.876 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:12:56.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.876 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:12:56.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:12:56.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:12:56.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:12:56.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:12:56.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:12:56.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:12:56.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:12:56.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:12:56.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:12:56.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:12:56.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:12:56.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:12:56.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:12:56.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:12:56.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:12:56.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:12:56.880 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.880 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.880 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.880 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.880 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:12:56.880 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.318088100000001, 2024-09-13T09:12:56.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:12:56.881 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.881 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.881 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:12:56.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.881 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:12:56.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:12:56.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:12:56.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:12:56.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:12:56.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:12:56.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:12:56.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:12:56.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:12:56.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:12:56.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:12:56.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:12:56.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:12:56.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:12:56.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:12:56.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:12:56.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:12:56.885 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.885 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.885 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.885 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.885 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:12:56.885 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.318031270000001, 2024-09-13T09:12:56.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:12:56.885 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.886 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.886 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.886 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.886 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.886 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:12:56.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.886 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:12:56.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:12:56.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:12:56.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:12:56.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:12:56.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:12:56.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:12:56.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:12:56.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:12:56.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:12:56.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:12:56.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:12:56.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:12:56.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:12:56.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:12:56.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:12:56.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:12:56.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.891 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.891 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.891 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:12:56.891 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.317980242999999, 2024-09-13T09:12:56.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:12:56.891 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.891 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.891 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.892 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:12:56.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.892 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:12:56.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:12:56.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:12:56.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:12:56.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:12:56.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:12:56.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:12:56.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:12:56.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:12:56.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:12:56.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:12:56.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:12:56.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:12:56.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:12:56.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:12:56.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:12:56.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:12:56.897 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.897 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.898 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.898 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.898 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:12:56.899 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.317939053, 2024-09-13T09:12:56.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:12:56.899 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.899 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.899 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:12:56.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.900 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:12:56.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:12:56.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:12:56.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:12:56.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:12:56.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:12:56.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:12:56.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:12:56.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:12:56.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:12:56.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:12:56.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:12:56.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:12:56.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:12:56.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:12:56.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:12:56.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:12:56.907 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.907 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.907 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.907 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.907 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:12:56.907 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.317896937, 2024-09-13T09:12:56.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:12:56.908 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.908 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.908 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:12:56.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.908 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:12:56.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:12:56.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:12:56.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:56.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:12:56.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:12:56.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:12:56.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:12:56.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:12:56.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:12:56.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:12:56.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:12:56.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:12:56.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:12:56.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:12:56.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:12:56.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:12:56.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:12:56.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.913 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.913 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.913 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:12:56.913 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.317853058000001, 2024-09-13T09:12:56.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:12:56.913 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.913 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.914 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:12:56.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.914 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:12:56.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:12:56.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:12:56.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:56.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:12:56.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:12:56.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:12:56.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:12:56.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:12:56.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:12:56.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:12:56.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:12:56.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:12:56.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:12:56.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:12:56.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:12:56.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:12:56.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:12:56.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.918 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.919 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.919 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:12:56.919 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.317807140999999, 2024-09-13T09:12:56.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:12:56.919 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.919 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.919 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.919 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.919 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.919 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:12:56.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.920 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:12:56.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:12:56.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:12:56.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:12:56.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:12:56.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:12:56.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:12:56.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:12:56.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:12:56.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:12:56.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:12:56.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:12:56.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:12:56.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:12:56.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:12:56.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:12:56.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:12:56.924 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.924 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.924 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.924 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.925 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:12:56.925 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.317757370000001, 2024-09-13T09:12:56.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:12:56.925 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.925 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.925 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:12:56.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.926 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:12:56.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:12:56.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:12:56.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:56.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:12:56.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:12:56.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:12:56.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:12:56.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:12:56.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:12:56.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:12:56.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:12:56.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:12:56.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:12:56.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:12:56.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:12:56.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:12:56.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:12:56.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.930 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.931 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.931 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:12:56.931 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.317711965999999, 2024-09-13T09:12:56.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:12:56.931 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.931 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.931 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:12:56.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.932 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:12:56.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:12:56.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:12:56.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:56.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:12:56.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:12:56.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:12:56.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:12:56.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:12:56.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:12:56.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:12:56.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:12:56.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:12:56.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:12:56.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:12:56.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:12:56.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:12:56.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:12:56.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.936 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.937 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.937 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:12:56.937 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.317661328, 2024-09-13T09:12:56.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:12:56.937 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.937 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.937 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:12:56.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.937 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:12:56.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:12:56.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:12:56.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:12:56.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:12:56.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:12:56.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:12:56.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:12:56.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:12:56.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:12:56.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:12:56.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:12:56.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:12:56.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:12:56.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:12:56.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:12:56.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:12:56.942 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.942 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.942 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.942 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.942 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:12:56.943 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.317605485, 2024-09-13T09:12:56.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:12:56.943 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.943 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.943 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.943 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.943 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.943 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:12:56.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.943 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:12:56.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:12:56.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:12:56.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:12:56.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:12:56.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:12:56.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:12:56.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:12:56.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:12:56.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:12:56.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:12:56.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:12:56.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:12:56.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:12:56.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:12:56.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:12:56.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:12:56.947 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.947 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.947 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.947 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.948 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:12:56.948 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.317548563000001, 2024-09-13T09:12:56.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:12:56.948 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.948 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.948 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.948 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.948 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.948 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:12:56.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.948 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:12:56.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:12:56.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:12:56.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:56.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:12:56.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:12:56.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:12:56.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:12:56.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:12:56.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:12:56.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:12:56.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:12:56.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:12:56.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:12:56.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:12:56.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:12:56.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:12:56.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:12:56.952 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.952 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.952 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.953 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:12:56.953 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:12:56.953 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.317494999999999, 2024-09-13T09:12:56.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:12:56.953 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:56.953 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:12:56.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:56.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:56.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:56.953 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:12:56.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.953 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:56.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:56.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:12:56.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:56.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:12:56.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:56.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:56.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:12:56.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:56.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:56.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:12:56.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:12:56.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:56.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:12:56.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:12:56.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:56.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:12:56.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:12:56.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:12:56.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:12:56.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:12:56.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:12:56.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:12:56.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:56.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:12:56.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:12:56.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:56.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:56.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:56.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:12:56.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:12:56.957 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.957 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.957 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:56.958 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:56.958 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:56.959 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:56.960 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:56.958+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:57.061 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:57.059+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:57.062 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:57.063 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:57.063 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:57.063 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:57.071 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:57.161 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:57.159+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:57.173 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:57.173 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:57.173 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:57.173 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:57.173 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:57.182 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:57.262 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:57.261+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:57.328 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:57.329 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:57.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:57.329 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:57.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.329 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17752 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2125824~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.329 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138424+0000", 2024-09-13T09:12:57.329 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.188403481, 2024-09-13T09:12:57.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045520176000000002, 2024-09-13T09:12:57.329 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.329 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.330 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.330 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17752 2024-09-13T09:12:57.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.330 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138424+0000", 2024-09-13T09:12:57.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138424+0000", 2024-09-13T09:12:57.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138423+0000", 2024-09-13T09:12:57.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138425+0000", 2024-09-13T09:12:57.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0640000000000001e-06 2024-09-13T09:12:57.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138431+0000", 2024-09-13T09:12:57.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0900000000000001e-06 2024-09-13T09:12:57.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138437+0000", 2024-09-13T09:12:57.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0329999999999997e-06 2024-09-13T09:12:57.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183854+0000", 2024-09-13T09:12:57.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045416099000000001 2024-09-13T09:12:57.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183894+0000", 2024-09-13T09:12:57.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0935e-05 2024-09-13T09:12:57.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183944+0000", 2024-09-13T09:12:57.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9329999999999997e-05 2024-09-13T09:12:57.334 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.334 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.334 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.334 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17753 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2148352~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.334 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138442+0000", 2024-09-13T09:12:57.334 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.18838476, 2024-09-13T09:12:57.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045633606, 2024-09-13T09:12:57.334 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.334 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.334 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.335 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.335 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.335 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17753 2024-09-13T09:12:57.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.335 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138442+0000", 2024-09-13T09:12:57.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138442+0000", 2024-09-13T09:12:57.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138441+0000", 2024-09-13T09:12:57.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:12:57.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138444+0000", 2024-09-13T09:12:57.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0120000000000001e-06 2024-09-13T09:12:57.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138450+0000", 2024-09-13T09:12:57.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6180000000000003e-06 2024-09-13T09:12:57.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138458+0000", 2024-09-13T09:12:57.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9130000000000003e-06 2024-09-13T09:12:57.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183973+0000", 2024-09-13T09:12:57.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045514741999999997 2024-09-13T09:12:57.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184022+0000", 2024-09-13T09:12:57.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9521999999999998e-05 2024-09-13T09:12:57.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184076+0000", 2024-09-13T09:12:57.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3888000000000003e-05 2024-09-13T09:12:57.339 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.339 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.339 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.340 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17754 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2170880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.340 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:12:57.340 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.188355818, 2024-09-13T09:12:57.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045714312999999999, 2024-09-13T09:12:57.340 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.340 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.340 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.340 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.340 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.340 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17754 2024-09-13T09:12:57.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.341 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:12:57.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:12:57.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:12:57.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138474+0000", 2024-09-13T09:12:57.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8540000000000001e-06 2024-09-13T09:12:57.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138480+0000", 2024-09-13T09:12:57.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5139999999999998e-06 2024-09-13T09:12:57.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138485+0000", 2024-09-13T09:12:57.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.989e-06 2024-09-13T09:12:57.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184095+0000", 2024-09-13T09:12:57.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045609702000000002 2024-09-13T09:12:57.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184135+0000", 2024-09-13T09:12:57.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0510999999999999e-05 2024-09-13T09:12:57.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184186+0000", 2024-09-13T09:12:57.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0262000000000001e-05 2024-09-13T09:12:57.344 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.344 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.344 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.344 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17755 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2193408~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.344 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:12:57.344 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.18833663, 2024-09-13T09:12:57.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045899349999999998, 2024-09-13T09:12:57.344 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.345 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.345 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17755 2024-09-13T09:12:57.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.345 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:12:57.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:12:57.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:12:57.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138492+0000", 2024-09-13T09:12:57.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4310000000000001e-06 2024-09-13T09:12:57.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138498+0000", 2024-09-13T09:12:57.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7400000000000001e-06 2024-09-13T09:12:57.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138503+0000", 2024-09-13T09:12:57.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:12:57.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184222+0000", 2024-09-13T09:12:57.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045718996999999997 2024-09-13T09:12:57.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184306+0000", 2024-09-13T09:12:57.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4293000000000001e-05 2024-09-13T09:12:57.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184390+0000", 2024-09-13T09:12:57.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3718e-05 2024-09-13T09:12:57.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.349 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.349 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17756 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2215936~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.349 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138507+0000", 2024-09-13T09:12:57.349 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.188320083000001, 2024-09-13T09:12:57.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046004902, 2024-09-13T09:12:57.349 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.350 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.350 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17756 2024-09-13T09:12:57.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.350 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138507+0000", 2024-09-13T09:12:57.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138507+0000", 2024-09-13T09:12:57.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138506+0000", 2024-09-13T09:12:57.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138509+0000", 2024-09-13T09:12:57.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:12:57.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138514+0000", 2024-09-13T09:12:57.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.665e-06 2024-09-13T09:12:57.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138519+0000", 2024-09-13T09:12:57.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7530000000000001e-06 2024-09-13T09:12:57.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184415+0000", 2024-09-13T09:12:57.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045895575000000001 2024-09-13T09:12:57.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184459+0000", 2024-09-13T09:12:57.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4506999999999999e-05 2024-09-13T09:12:57.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184512+0000", 2024-09-13T09:12:57.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2685999999999997e-05 2024-09-13T09:12:57.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.354 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.354 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17766 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2441216~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.354 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140631+0000", 2024-09-13T09:12:57.354 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.186196525, 2024-09-13T09:12:57.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045443425000000003, 2024-09-13T09:12:57.354 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.355 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.355 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17766 2024-09-13T09:12:57.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.355 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140631+0000", 2024-09-13T09:12:57.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140631+0000", 2024-09-13T09:12:57.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140630+0000", 2024-09-13T09:12:57.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140632+0000", 2024-09-13T09:12:57.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1770000000000001e-06 2024-09-13T09:12:57.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140638+0000", 2024-09-13T09:12:57.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5550000000000002e-06 2024-09-13T09:12:57.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140643+0000", 2024-09-13T09:12:57.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5589999999999999e-06 2024-09-13T09:12:57.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.185929+0000", 2024-09-13T09:12:57.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045286356999999999 2024-09-13T09:12:57.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.185985+0000", 2024-09-13T09:12:57.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6042999999999999e-05 2024-09-13T09:12:57.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186074+0000", 2024-09-13T09:12:57.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9083000000000001e-05 2024-09-13T09:12:57.359 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.359 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.359 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.359 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17767 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2463744~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.360 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:12:57.360 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.186180732, 2024-09-13T09:12:57.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045589383999999997, 2024-09-13T09:12:57.360 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.360 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.360 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.360 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.360 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.360 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17767 2024-09-13T09:12:57.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.360 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:12:57.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:12:57.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:12:57.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140648+0000", 2024-09-13T09:12:57.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.001e-06 2024-09-13T09:12:57.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140654+0000", 2024-09-13T09:12:57.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6690000000000003e-06 2024-09-13T09:12:57.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140658+0000", 2024-09-13T09:12:57.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4220000000000002e-06 2024-09-13T09:12:57.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186115+0000", 2024-09-13T09:12:57.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045456435000000003 2024-09-13T09:12:57.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186172+0000", 2024-09-13T09:12:57.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7088e-05 2024-09-13T09:12:57.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186236+0000", 2024-09-13T09:12:57.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4082000000000001e-05 2024-09-13T09:12:57.365 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.365 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.365 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.365 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17768 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2486272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.365 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:12:57.365 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.186164765000001, 2024-09-13T09:12:57.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045865601999999998, 2024-09-13T09:12:57.365 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.365 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.365 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.365 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.366 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.366 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17768 2024-09-13T09:12:57.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.366 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:12:57.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:12:57.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:12:57.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140664+0000", 2024-09-13T09:12:57.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1449999999999998e-06 2024-09-13T09:12:57.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140670+0000", 2024-09-13T09:12:57.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9460000000000003e-06 2024-09-13T09:12:57.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140675+0000", 2024-09-13T09:12:57.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5770000000000004e-06 2024-09-13T09:12:57.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186267+0000", 2024-09-13T09:12:57.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045591957000000002 2024-09-13T09:12:57.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186322+0000", 2024-09-13T09:12:57.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5049000000000001e-05 2024-09-13T09:12:57.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186528+0000", 2024-09-13T09:12:57.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020626600000000001 2024-09-13T09:12:57.369 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.369 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.369 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.370 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17769 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2508800~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.370 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:12:57.370 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.186148704000001, 2024-09-13T09:12:57.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045981877999999997, 2024-09-13T09:12:57.370 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.370 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.370 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.371 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17769 2024-09-13T09:12:57.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.371 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:12:57.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:12:57.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:12:57.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140680+0000", 2024-09-13T09:12:57.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0209999999999998e-06 2024-09-13T09:12:57.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140686+0000", 2024-09-13T09:12:57.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2840000000000003e-06 2024-09-13T09:12:57.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140691+0000", 2024-09-13T09:12:57.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.369e-06 2024-09-13T09:12:57.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186562+0000", 2024-09-13T09:12:57.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045870834999999999 2024-09-13T09:12:57.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186606+0000", 2024-09-13T09:12:57.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4631e-05 2024-09-13T09:12:57.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186660+0000", 2024-09-13T09:12:57.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4042000000000002e-05 2024-09-13T09:12:57.375 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.375 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.375 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.376 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17885 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2217984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.376 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:12:57.376 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.184175605, 2024-09-13T09:12:57.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045427829000000003, 2024-09-13T09:12:57.376 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.376 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.376 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17885 2024-09-13T09:12:57.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.377 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:12:57.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:12:57.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:12:57.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142653+0000", 2024-09-13T09:12:57.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2500000000000001e-06 2024-09-13T09:12:57.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142659+0000", 2024-09-13T09:12:57.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.366e-06 2024-09-13T09:12:57.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142663+0000", 2024-09-13T09:12:57.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5190000000000003e-06 2024-09-13T09:12:57.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187980+0000", 2024-09-13T09:12:57.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045316233999999997 2024-09-13T09:12:57.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188032+0000", 2024-09-13T09:12:57.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1981000000000003e-05 2024-09-13T09:12:57.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188079+0000", 2024-09-13T09:12:57.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7803000000000003e-05 2024-09-13T09:12:57.380 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.380 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.380 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.381 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17886 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2240512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.381 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142686+0000", 2024-09-13T09:12:57.381 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.184141197000001, 2024-09-13T09:12:57.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045507535000000002, 2024-09-13T09:12:57.381 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.381 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.381 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.381 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.381 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.381 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17886 2024-09-13T09:12:57.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.381 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142686+0000", 2024-09-13T09:12:57.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142686+0000", 2024-09-13T09:12:57.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142685+0000", 2024-09-13T09:12:57.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142688+0000", 2024-09-13T09:12:57.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.83e-06 2024-09-13T09:12:57.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142694+0000", 2024-09-13T09:12:57.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2910000000000004e-06 2024-09-13T09:12:57.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142699+0000", 2024-09-13T09:12:57.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9599999999999999e-06 2024-09-13T09:12:57.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188102+0000", 2024-09-13T09:12:57.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045402459999999999 2024-09-13T09:12:57.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188143+0000", 2024-09-13T09:12:57.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1607000000000003e-05 2024-09-13T09:12:57.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188193+0000", 2024-09-13T09:12:57.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9947000000000001e-05 2024-09-13T09:12:57.385 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.385 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.386 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.386 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17887 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2263040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.386 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142704+0000", 2024-09-13T09:12:57.386 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.184123206000001, 2024-09-13T09:12:57.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045652102999999999, 2024-09-13T09:12:57.386 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.386 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.386 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.386 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.387 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17887 2024-09-13T09:12:57.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.387 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142704+0000", 2024-09-13T09:12:57.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142704+0000", 2024-09-13T09:12:57.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142703+0000", 2024-09-13T09:12:57.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142706+0000", 2024-09-13T09:12:57.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.351e-06 2024-09-13T09:12:57.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142712+0000", 2024-09-13T09:12:57.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8799999999999996e-06 2024-09-13T09:12:57.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142716+0000", 2024-09-13T09:12:57.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-13T09:12:57.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188230+0000", 2024-09-13T09:12:57.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045513801 2024-09-13T09:12:57.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188303+0000", 2024-09-13T09:12:57.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3032000000000001e-05 2024-09-13T09:12:57.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188356+0000", 2024-09-13T09:12:57.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2899000000000003e-05 2024-09-13T09:12:57.390 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.390 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.391 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.391 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17893 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2398208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.391 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:12:57.391 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.183071671, 2024-09-13T09:12:57.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045574960999999997, 2024-09-13T09:12:57.391 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.392 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.392 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.392 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.392 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.392 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17893 2024-09-13T09:12:57.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.392 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:12:57.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:12:57.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:12:57.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143757+0000", 2024-09-13T09:12:57.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2000000000000001e-06 2024-09-13T09:12:57.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143763+0000", 2024-09-13T09:12:57.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.553e-06 2024-09-13T09:12:57.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143768+0000", 2024-09-13T09:12:57.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0019999999999997e-06 2024-09-13T09:12:57.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189160+0000", 2024-09-13T09:12:57.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045392147000000001 2024-09-13T09:12:57.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189243+0000", 2024-09-13T09:12:57.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2594000000000002e-05 2024-09-13T09:12:57.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189330+0000", 2024-09-13T09:12:57.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7764999999999997e-05 2024-09-13T09:12:57.395 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.395 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.396 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.396 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17894 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2420736~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.396 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.143886+0000", 2024-09-13T09:12:57.396 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.182941187999999, 2024-09-13T09:12:57.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045562601000000001, 2024-09-13T09:12:57.396 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.396 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.396 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.396 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.396 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.396 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17894 2024-09-13T09:12:57.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.397 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143886+0000", 2024-09-13T09:12:57.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143886+0000", 2024-09-13T09:12:57.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143771+0000", 2024-09-13T09:12:57.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998856 2024-09-13T09:12:57.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143889+0000", 2024-09-13T09:12:57.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000117314 2024-09-13T09:12:57.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143896+0000", 2024-09-13T09:12:57.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5920000000000003e-06 2024-09-13T09:12:57.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143902+0000", 2024-09-13T09:12:57.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:12:57.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189359+0000", 2024-09-13T09:12:57.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045457414000000002 2024-09-13T09:12:57.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189399+0000", 2024-09-13T09:12:57.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9653999999999997e-05 2024-09-13T09:12:57.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189449+0000", 2024-09-13T09:12:57.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9456999999999998e-05 2024-09-13T09:12:57.400 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.400 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.400 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.400 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17895 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2443264~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.400 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.143990+0000", 2024-09-13T09:12:57.400 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.182837212999999, 2024-09-13T09:12:57.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045666642, 2024-09-13T09:12:57.401 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.401 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.401 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17895 2024-09-13T09:12:57.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.401 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143990+0000", 2024-09-13T09:12:57.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143990+0000", 2024-09-13T09:12:57.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143907+0000", 2024-09-13T09:12:57.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999166 2024-09-13T09:12:57.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143993+0000", 2024-09-13T09:12:57.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6049000000000002e-05 2024-09-13T09:12:57.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144000+0000", 2024-09-13T09:12:57.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6550000000000004e-06 2024-09-13T09:12:57.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144006+0000", 2024-09-13T09:12:57.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4450000000000004e-06 2024-09-13T09:12:57.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189483+0000", 2024-09-13T09:12:57.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045477228000000001 2024-09-13T09:12:57.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189567+0000", 2024-09-13T09:12:57.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3721999999999996e-05 2024-09-13T09:12:57.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189657+0000", 2024-09-13T09:12:57.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9764000000000003e-05 2024-09-13T09:12:57.405 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.405 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.405 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.405 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17896 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2465792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.406 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144260+0000", 2024-09-13T09:12:57.406 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.182567346000001, 2024-09-13T09:12:57.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045647222000000001, 2024-09-13T09:12:57.406 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.406 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.406 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17896 2024-09-13T09:12:57.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.406 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144260+0000", 2024-09-13T09:12:57.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144260+0000", 2024-09-13T09:12:57.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144010+0000", 2024-09-13T09:12:57.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9997506 2024-09-13T09:12:57.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144263+0000", 2024-09-13T09:12:57.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025252299999999999 2024-09-13T09:12:57.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144270+0000", 2024-09-13T09:12:57.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5259999999999996e-06 2024-09-13T09:12:57.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144276+0000", 2024-09-13T09:12:57.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:12:57.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189682+0000", 2024-09-13T09:12:57.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045406482999999997 2024-09-13T09:12:57.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189723+0000", 2024-09-13T09:12:57.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1059000000000001e-05 2024-09-13T09:12:57.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189907+0000", 2024-09-13T09:12:57.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018378099999999999 2024-09-13T09:12:57.410 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.410 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.410 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.410 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17897 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2488320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.410 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144430+0000", 2024-09-13T09:12:57.410 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.182397204000001, 2024-09-13T09:12:57.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045662734000000003, 2024-09-13T09:12:57.410 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.410 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.411 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17897 2024-09-13T09:12:57.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.411 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144430+0000", 2024-09-13T09:12:57.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144430+0000", 2024-09-13T09:12:57.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144281+0000", 2024-09-13T09:12:57.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998507 2024-09-13T09:12:57.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144433+0000", 2024-09-13T09:12:57.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00015261800000000001 2024-09-13T09:12:57.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144441+0000", 2024-09-13T09:12:57.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6790000000000005e-06 2024-09-13T09:12:57.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144446+0000", 2024-09-13T09:12:57.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5280000000000003e-06 2024-09-13T09:12:57.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189943+0000", 2024-09-13T09:12:57.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045496178999999998 2024-09-13T09:12:57.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190025+0000", 2024-09-13T09:12:57.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2325999999999997e-05 2024-09-13T09:12:57.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190093+0000", 2024-09-13T09:12:57.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7780000000000005e-05 2024-09-13T09:12:57.414 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.414 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.415 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.415 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17898 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2510848~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.415 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144655+0000", 2024-09-13T09:12:57.415 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.182172029, 2024-09-13T09:12:57.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045552875, 2024-09-13T09:12:57.415 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.415 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.415 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17898 2024-09-13T09:12:57.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.416 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144655+0000", 2024-09-13T09:12:57.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144655+0000", 2024-09-13T09:12:57.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144451+0000", 2024-09-13T09:12:57.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9997959 2024-09-13T09:12:57.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144658+0000", 2024-09-13T09:12:57.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000207188 2024-09-13T09:12:57.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144666+0000", 2024-09-13T09:12:57.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1380000000000007e-06 2024-09-13T09:12:57.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144672+0000", 2024-09-13T09:12:57.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7980000000000002e-06 2024-09-13T09:12:57.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190116+0000", 2024-09-13T09:12:57.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045443413000000002 2024-09-13T09:12:57.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190158+0000", 2024-09-13T09:12:57.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2055000000000003e-05 2024-09-13T09:12:57.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190208+0000", 2024-09-13T09:12:57.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0387999999999999e-05 2024-09-13T09:12:57.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.419 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.419 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17899 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2533376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.420 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144737+0000", 2024-09-13T09:12:57.420 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.182089886, 2024-09-13T09:12:57.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045677290000000002, 2024-09-13T09:12:57.420 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.420 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.420 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17899 2024-09-13T09:12:57.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.420 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144737+0000", 2024-09-13T09:12:57.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144737+0000", 2024-09-13T09:12:57.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144678+0000", 2024-09-13T09:12:57.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999409 2024-09-13T09:12:57.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144740+0000", 2024-09-13T09:12:57.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2116e-05 2024-09-13T09:12:57.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144748+0000", 2024-09-13T09:12:57.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2130000000000008e-06 2024-09-13T09:12:57.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144754+0000", 2024-09-13T09:12:57.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.66e-06 2024-09-13T09:12:57.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190238+0000", 2024-09-13T09:12:57.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045483824999999999 2024-09-13T09:12:57.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190322+0000", 2024-09-13T09:12:57.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4438999999999996e-05 2024-09-13T09:12:57.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190415+0000", 2024-09-13T09:12:57.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2227999999999999e-05 2024-09-13T09:12:57.424 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.424 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.424 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.424 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17900 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2555904~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:12:57.424 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144938+0000", 2024-09-13T09:12:57.424 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.181888985000001, 2024-09-13T09:12:57.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045612509000000002, 2024-09-13T09:12:57.425 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.425 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.425 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.425 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:12:57.425 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:12:57.425 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17900 2024-09-13T09:12:57.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.425 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144938+0000", 2024-09-13T09:12:57.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144938+0000", 2024-09-13T09:12:57.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144758+0000", 2024-09-13T09:12:57.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998202 2024-09-13T09:12:57.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144941+0000", 2024-09-13T09:12:57.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000182688 2024-09-13T09:12:57.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144949+0000", 2024-09-13T09:12:57.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7300000000000005e-06 2024-09-13T09:12:57.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144954+0000", 2024-09-13T09:12:57.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4090000000000002e-06 2024-09-13T09:12:57.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190448+0000", 2024-09-13T09:12:57.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045493901000000003 2024-09-13T09:12:57.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190499+0000", 2024-09-13T09:12:57.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0677999999999998e-05 2024-09-13T09:12:57.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190551+0000", 2024-09-13T09:12:57.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1820999999999997e-05 2024-09-13T09:12:57.428 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.429 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.429 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.429 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.429 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.429 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:57.431 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:12:57.431 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:57.429+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:57.532 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:57.531+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:57.575 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:57.575 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:57.575 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:12:57.575 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:57.583 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:12:57.632 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:57.631+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:57.635 INFO:teuthology.orchestra.run.smithi157.stdout:2785 write 0x6b5c000 thru 0x6b5fa83 (0x3a84 bytes) 2024-09-13T09:12:57.637 INFO:teuthology.orchestra.run.smithi157.stdout:2786 read 0x3c98400 thru 0x3ca812e (0xfd2f bytes) 2024-09-13T09:12:57.656 INFO:teuthology.orchestra.run.smithi157.stdout:2788 write 0x8bcd000 thru 0x8bd58ad (0x88ae bytes) 2024-09-13T09:12:57.665 INFO:teuthology.orchestra.run.smithi157.stdout:2789 trunc from 0x8bd5a00 to 0xd647600 2024-09-13T09:12:57.683 INFO:teuthology.orchestra.run.smithi157.stdout:2791 read 0x9b31400 thru 0x9b38e5d (0x7a5e bytes) 2024-09-13T09:12:57.688 INFO:teuthology.orchestra.run.smithi157.stdout:2792 punch from 0x6af6600 to 0x6afe600, (0x8000 bytes) 2024-09-13T09:12:57.693 INFO:teuthology.orchestra.run.smithi157.stdout:2793 trunc from 0xd647600 to 0xaa2c000 2024-09-13T09:12:57.698 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:57.698 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:12:57.699 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:12:57.699 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:12:57.699 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:57.704 INFO:teuthology.orchestra.run.smithi157.stdout:2794 write 0x11b4200 thru 0x11befa5 (0xada6 bytes) 2024-09-13T09:12:57.706 INFO:teuthology.orchestra.run.smithi157.stdout:2795 write 0x893be00 thru 0x893f381 (0x3582 bytes) 2024-09-13T09:12:57.706 INFO:teuthology.orchestra.run.smithi157.stdout:2797 punch from 0xd09200 to 0xd0a400, (0x1200 bytes) 2024-09-13T09:12:57.709 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:12:57.710 INFO:teuthology.orchestra.run.smithi157.stdout:2799 read 0x8aec000 thru 0x8aee524 (0x2525 bytes) 2024-09-13T09:12:57.715 INFO:teuthology.orchestra.run.smithi157.stdout:2801 trunc from 0xaa2c000 to 0x7c65a00 2024-09-13T09:12:57.733 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:57.732+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:57.835 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:57.834+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:57.858 INFO:teuthology.orchestra.run.smithi157.stdout:2802 write 0x1435c00 thru 0x143d194 (0x7595 bytes) 2024-09-13T09:12:57.864 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:12:57.864 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:12:57.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:12:57.864 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:12:57.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.865 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9035 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2342912~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.865 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:12:57.865 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.11703155699999999, 2024-09-13T09:12:57.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070293606999999994, 2024-09-13T09:12:57.865 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.865 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.865 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9035 2024-09-13T09:12:57.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.865 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:12:57.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:12:57.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:12:57.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745602+0000", 2024-09-13T09:12:57.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-06 2024-09-13T09:12:57.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745607+0000", 2024-09-13T09:12:57.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0770000000000001e-06 2024-09-13T09:12:57.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745609+0000", 2024-09-13T09:12:57.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3499999999999999e-06 2024-09-13T09:12:57.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815626+0000", 2024-09-13T09:12:57.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070017398999999994 2024-09-13T09:12:57.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815672+0000", 2024-09-13T09:12:57.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5275999999999998e-05 2024-09-13T09:12:57.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815895+0000", 2024-09-13T09:12:57.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022328300000000001 2024-09-13T09:12:57.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.869 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.869 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.869 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:12:57.869 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.11701901300000001, 2024-09-13T09:12:57.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:12:57.870 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.870 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.870 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:12:57.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.870 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:12:57.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:12:57.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:12:57.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:12:57.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:12:57.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:12:57.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:12:57.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:12:57.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:12:57.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:12:57.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:12:57.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:12:57.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:12:57.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:12:57.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:12:57.874 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.874 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.874 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.874 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.874 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:12:57.874 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.11701033499999999, 2024-09-13T09:12:57.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:12:57.875 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.875 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.875 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.875 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.875 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.875 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:12:57.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.875 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:12:57.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:12:57.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:12:57.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:57.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:12:57.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:12:57.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:12:57.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:12:57.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:12:57.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:12:57.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:12:57.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:12:57.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:12:57.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:12:57.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:12:57.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:12:57.879 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.879 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.879 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.879 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.879 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:12:57.880 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.117001349, 2024-09-13T09:12:57.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:12:57.880 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.880 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.880 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.880 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.880 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.880 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:12:57.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.880 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:12:57.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:12:57.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:12:57.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:12:57.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:12:57.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:12:57.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:12:57.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:12:57.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:12:57.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:12:57.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:12:57.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:12:57.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:12:57.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:12:57.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:12:57.884 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.884 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.884 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.885 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.885 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:12:57.885 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.116993377, 2024-09-13T09:12:57.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:12:57.885 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.885 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.885 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.885 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:12:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:12:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:12:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:12:57.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:57.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:12:57.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:12:57.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:12:57.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:12:57.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:12:57.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:12:57.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:12:57.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:12:57.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:12:57.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:12:57.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:12:57.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:12:57.889 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.889 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.890 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.890 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.890 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:12:57.890 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.115118543, 2024-09-13T09:12:57.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:12:57.891 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.891 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.891 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.891 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.891 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.891 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:12:57.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.892 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:12:57.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:12:57.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:12:57.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:12:57.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:12:57.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:12:57.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:12:57.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:12:57.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:12:57.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:12:57.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:12:57.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:12:57.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:12:57.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:12:57.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:12:57.898 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.899 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.899 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.899 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.899 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:12:57.900 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.11510788199999999, 2024-09-13T09:12:57.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:12:57.900 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.900 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.901 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:12:57.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.901 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:12:57.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:12:57.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:12:57.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:57.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:12:57.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:12:57.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:12:57.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:12:57.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:12:57.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:12:57.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:12:57.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:12:57.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:12:57.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:12:57.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:12:57.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:12:57.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.908 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.908 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.908 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:12:57.908 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.115085906, 2024-09-13T09:12:57.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:12:57.909 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.909 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.909 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:12:57.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.909 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:12:57.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:12:57.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:12:57.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:12:57.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:12:57.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:12:57.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:12:57.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:12:57.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:12:57.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:12:57.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:12:57.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:12:57.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:12:57.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:12:57.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:12:57.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.913 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.913 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.913 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:12:57.913 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.115072404, 2024-09-13T09:12:57.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:12:57.913 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.913 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.914 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:12:57.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.914 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:12:57.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:12:57.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:12:57.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:12:57.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:12:57.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:12:57.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:12:57.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:12:57.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:12:57.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:12:57.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:12:57.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:12:57.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:12:57.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:12:57.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:12:57.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.918 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.919 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9055 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 6930432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.919 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:12:57.919 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.113079624, 2024-09-13T09:12:57.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070177949000000003, 2024-09-13T09:12:57.919 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.919 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.920 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.920 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.920 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.920 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9055 2024-09-13T09:12:57.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.920 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:12:57.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:12:57.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:12:57.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:57.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749554+0000", 2024-09-13T09:12:57.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.11e-06 2024-09-13T09:12:57.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749557+0000", 2024-09-13T09:12:57.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7949999999999999e-06 2024-09-13T09:12:57.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749559+0000", 2024-09-13T09:12:57.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:12:57.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819633+0000", 2024-09-13T09:12:57.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073995 2024-09-13T09:12:57.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819679+0000", 2024-09-13T09:12:57.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6006e-05 2024-09-13T09:12:57.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819731+0000", 2024-09-13T09:12:57.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1934000000000001e-05 2024-09-13T09:12:57.927 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.927 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.927 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.927 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9056 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 7159808~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.927 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:12:57.927 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.11307157499999999, 2024-09-13T09:12:57.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070283126000000001, 2024-09-13T09:12:57.927 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.927 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.928 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9056 2024-09-13T09:12:57.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.928 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:12:57.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:12:57.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:12:57.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:57.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749562+0000", 2024-09-13T09:12:57.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1340000000000001e-06 2024-09-13T09:12:57.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749565+0000", 2024-09-13T09:12:57.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8760000000000001e-06 2024-09-13T09:12:57.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749567+0000", 2024-09-13T09:12:57.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2340000000000001e-06 2024-09-13T09:12:57.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819750+0000", 2024-09-13T09:12:57.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070182537000000003 2024-09-13T09:12:57.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819792+0000", 2024-09-13T09:12:57.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2141999999999997e-05 2024-09-13T09:12:57.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819844+0000", 2024-09-13T09:12:57.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2428e-05 2024-09-13T09:12:57.932 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.932 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.932 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.932 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.932 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:12:57.932 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.113063306, 2024-09-13T09:12:57.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:12:57.933 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.933 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.933 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.933 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.933 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.933 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:12:57.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.933 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:12:57.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:12:57.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:12:57.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:12:57.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:12:57.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:12:57.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:12:57.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:12:57.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:12:57.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:12:57.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:12:57.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:12:57.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:12:57.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:12:57.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:12:57.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.936 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.937 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.937 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:12:57.937 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.113038984, 2024-09-13T09:12:57.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:12:57.937 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.937 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.937 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:12:57.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.937 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:12:57.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:12:57.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:12:57.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:12:57.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:12:57.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:12:57.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:12:57.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:12:57.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:12:57.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:12:57.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:12:57.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:12:57.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:12:57.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:12:57.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:12:57.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.941 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.941 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.941 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:12:57.941 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.11302801, 2024-09-13T09:12:57.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:12:57.941 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.941 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.942 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:12:57.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.942 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:12:57.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:12:57.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:12:57.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:12:57.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:12:57.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:12:57.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:12:57.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:12:57.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:12:57.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:12:57.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:12:57.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:12:57.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:12:57.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:12:57.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:12:57.945 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.945 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.945 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.945 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9063 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1589248~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.946 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:12:57.946 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.111124368, 2024-09-13T09:12:57.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070391924999999994, 2024-09-13T09:12:57.946 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.946 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.946 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9063 2024-09-13T09:12:57.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.946 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:12:57.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:12:57.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:12:57.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:57.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751509+0000", 2024-09-13T09:12:57.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.142e-06 2024-09-13T09:12:57.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751513+0000", 2024-09-13T09:12:57.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6669999999999998e-06 2024-09-13T09:12:57.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751520+0000", 2024-09-13T09:12:57.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5910000000000001e-06 2024-09-13T09:12:57.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821593+0000", 2024-09-13T09:12:57.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073519000000001 2024-09-13T09:12:57.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821648+0000", 2024-09-13T09:12:57.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:12:57.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821900+0000", 2024-09-13T09:12:57.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025263999999999999 2024-09-13T09:12:57.950 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.950 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.950 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.950 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.950 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:12:57.950 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.11111038500000001, 2024-09-13T09:12:57.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:12:57.950 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.950 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.951 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:12:57.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.951 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:12:57.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:12:57.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:12:57.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:12:57.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:12:57.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:12:57.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:12:57.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:12:57.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:12:57.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:12:57.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:12:57.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:12:57.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:12:57.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:12:57.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:12:57.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:12:57.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.954 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.954 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.954 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:12:57.955 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.111100988, 2024-09-13T09:12:57.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:12:57.955 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.955 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.955 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:12:57.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.955 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:12:57.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:12:57.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:12:57.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:57.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:12:57.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:12:57.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:12:57.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:12:57.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:12:57.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:12:57.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:12:57.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:12:57.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:12:57.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:12:57.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:12:57.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:12:57.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.958 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.959 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.959 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9069 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2965504~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.959 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:12:57.959 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.109141609, 2024-09-13T09:12:57.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070184959000000005, 2024-09-13T09:12:57.959 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.959 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.959 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9069 2024-09-13T09:12:57.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.960 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:12:57.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:12:57.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:12:57.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:57.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753492+0000", 2024-09-13T09:12:57.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.403e-06 2024-09-13T09:12:57.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753495+0000", 2024-09-13T09:12:57.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9390000000000002e-06 2024-09-13T09:12:57.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753504+0000", 2024-09-13T09:12:57.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5129999999999996e-06 2024-09-13T09:12:57.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823390+0000", 2024-09-13T09:12:57.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.069886632000000004 2024-09-13T09:12:57.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823445+0000", 2024-09-13T09:12:57.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4880999999999999e-05 2024-09-13T09:12:57.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823676+0000", 2024-09-13T09:12:57.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000230812 2024-09-13T09:12:57.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.963 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.963 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9070 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3194880~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.963 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:12:57.963 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.10912651800000001, 2024-09-13T09:12:57.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070280588000000005, 2024-09-13T09:12:57.963 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.963 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.964 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9070 2024-09-13T09:12:57.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.964 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:12:57.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:12:57.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:12:57.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:57.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753507+0000", 2024-09-13T09:12:57.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:12:57.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753510+0000", 2024-09-13T09:12:57.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0460000000000001e-06 2024-09-13T09:12:57.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753513+0000", 2024-09-13T09:12:57.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:12:57.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823689+0000", 2024-09-13T09:12:57.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070176083 2024-09-13T09:12:57.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823734+0000", 2024-09-13T09:12:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5056999999999999e-05 2024-09-13T09:12:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823787+0000", 2024-09-13T09:12:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3121000000000002e-05 2024-09-13T09:12:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.967 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9071 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:12:57.968 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:12:57.968 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.10911797099999999, 2024-09-13T09:12:57.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070386377999999999, 2024-09-13T09:12:57.968 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:12:57.968 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:12:57.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:12:57.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:12:57.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:12:57.968 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9071 2024-09-13T09:12:57.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.968 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:12:57.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:12:57.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:12:57.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:12:57.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:12:57.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:12:57.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:12:57.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:12:57.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:12:57.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:12:57.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753516+0000", 2024-09-13T09:12:57.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1519999999999999e-06 2024-09-13T09:12:57.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:12:57.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753519+0000", 2024-09-13T09:12:57.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6559999999999998e-06 2024-09-13T09:12:57.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:12:57.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753522+0000", 2024-09-13T09:12:57.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:12:57.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:12:57.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823798+0000", 2024-09-13T09:12:57.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070275907999999998 2024-09-13T09:12:57.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:12:57.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823845+0000", 2024-09-13T09:12:57.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7593000000000003e-05 2024-09-13T09:12:57.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:12:57.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:12:57.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:12:57.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823901+0000", 2024-09-13T09:12:57.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5934999999999999e-05 2024-09-13T09:12:57.971 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.972 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:12:57.972 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:12:57.972 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:12:57.973 INFO:teuthology.orchestra.run.smithi157.stdout:2804 trunc from 0x7c65a00 to 0x29fbe00 2024-09-13T09:12:57.973 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:57.972+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:57.974 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:12:58.074 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:58.073+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:58.088 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:58.088 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:58.088 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:12:58.088 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:58.097 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:12:58.104 INFO:teuthology.orchestra.run.smithi157.stdout:2806 read 0x293a400 thru 0x2942820 (0x8421 bytes) 2024-09-13T09:12:58.129 INFO:teuthology.orchestra.run.smithi157.stdout:2813 write 0xc712200 thru 0xc7175a4 (0x53a5 bytes) 2024-09-13T09:12:58.130 INFO:teuthology.orchestra.run.smithi157.stdout:2815 read 0x2f2e600 thru 0x2f2f526 (0xf27 bytes) 2024-09-13T09:12:58.130 INFO:teuthology.orchestra.run.smithi157.stdout:2816 read 0x4add800 thru 0x4aea467 (0xcc68 bytes) 2024-09-13T09:12:58.136 INFO:teuthology.orchestra.run.smithi157.stdout:2818 trunc from 0xc717600 to 0x76b4000 2024-09-13T09:12:58.142 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:12:58.155 INFO:teuthology.orchestra.run.smithi157.stdout:2819 punch from 0x34b3c00 to 0x34bda00, (0x9e00 bytes) 2024-09-13T09:12:58.157 INFO:teuthology.orchestra.run.smithi157.stdout:2820 write 0x22ac600 thru 0x22b050f (0x3f10 bytes) 2024-09-13T09:12:58.180 INFO:teuthology.orchestra.run.smithi157.stdout:2830 write 0xd78e400 thru 0xd78e97b (0x57c bytes) 2024-09-13T09:12:58.181 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:58.180+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:58.182 INFO:teuthology.orchestra.run.smithi157.stdout:2831 read 0x9291e00 thru 0x929d0ec (0xb2ed bytes) 2024-09-13T09:12:58.182 INFO:teuthology.orchestra.run.smithi157.stdout:2832 punch from 0x4378400 to 0x4383a00, (0xb600 bytes) 2024-09-13T09:12:58.182 INFO:teuthology.orchestra.run.smithi157.stdout:2834 read 0xa993800 thru 0xa9a3018 (0xf819 bytes) 2024-09-13T09:12:58.192 INFO:teuthology.orchestra.run.smithi157.stdout:2835 trunc from 0xd78ea00 to 0x4197400 2024-09-13T09:12:58.209 INFO:teuthology.orchestra.run.smithi157.stdout:2836 punch from 0x272fa00 to 0x273ce00, (0xd400 bytes) 2024-09-13T09:12:58.215 INFO:teuthology.orchestra.run.smithi157.stdout:2838 trunc from 0x4197400 to 0xaa2200 2024-09-13T09:12:58.261 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:12:58.261 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:12:58.262 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:12:58.262 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:12:58.262 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:12:58.272 INFO:teuthology.orchestra.run.smithi157.stdout:2839 write 0xe2bb400 thru 0xe2c249b (0x709c bytes) 2024-09-13T09:12:58.272 INFO:teuthology.orchestra.run.smithi157.stdout:2841 read 0x2553a00 thru 0x2562dd8 (0xf3d9 bytes) 2024-09-13T09:12:58.273 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:12:58.274 INFO:teuthology.orchestra.run.smithi157.stdout:2842 punch from 0x2e65a00 to 0x2e72e00, (0xd400 bytes) 2024-09-13T09:12:58.274 INFO:teuthology.orchestra.run.smithi157.stdout:2846 read 0xa77000 thru 0xa7ebe6 (0x7be7 bytes) 2024-09-13T09:12:58.274 INFO:teuthology.orchestra.run.smithi157.stdout:2848 write 0x8c89200 thru 0x8c94670 (0xb471 bytes) 2024-09-13T09:12:58.274 INFO:teuthology.orchestra.run.smithi157.stdout:2849 read 0xdd9d400 thru 0xdda82d8 (0xaed9 bytes) 2024-09-13T09:12:58.274 INFO:teuthology.orchestra.run.smithi157.stdout:2850 write 0x69bea00 thru 0x69cca9a (0xe09b bytes) 2024-09-13T09:12:58.275 INFO:teuthology.orchestra.run.smithi157.stdout:2851 trunc from 0xe2c2600 to 0xe9c5800 2024-09-13T09:12:58.275 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:58.273+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:58.294 INFO:teuthology.orchestra.run.smithi157.stdout:2852 trunc from 0xe9c5800 to 0xeb67000 2024-09-13T09:12:58.304 INFO:teuthology.orchestra.run.smithi157.stdout:2854 read 0x5370c00 thru 0x537edc2 (0xe1c3 bytes) 2024-09-13T09:12:58.305 INFO:teuthology.orchestra.run.smithi157.stdout:2855 write 0x8124a00 thru 0x812d03d (0x863e bytes) 2024-09-13T09:12:58.305 INFO:teuthology.orchestra.run.smithi157.stdout:2856 read 0x4ab1800 thru 0x4abc76b (0xaf6c bytes) 2024-09-13T09:12:58.312 INFO:teuthology.orchestra.run.smithi157.stdout:2857 trunc from 0xeb67000 to 0x96c7800 2024-09-13T09:12:58.334 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:12:58.335 INFO:teuthology.orchestra.run.smithi157.stdout:2858 write 0x2968a00 thru 0x29780c3 (0xf6c4 bytes) 2024-09-13T09:12:58.337 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:12:58.338 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:12:58.339 INFO:teuthology.orchestra.run.smithi157.stdout:2860 write 0x700a00 thru 0x70404a (0x364b bytes) 2024-09-13T09:12:58.340 INFO:teuthology.orchestra.run.smithi157.stdout:2862 punch from 0x7de5600 to 0x7deee00, (0x9800 bytes) 2024-09-13T09:12:58.347 INFO:teuthology.orchestra.run.smithi157.stdout:2864 trunc from 0x96c7800 to 0x3618a00 2024-09-13T09:12:58.365 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:12:58.366 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:12:58.376 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:58.374+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:58.386 INFO:teuthology.orchestra.run.smithi157.stdout:2866 write 0x5ac2c00 thru 0x5acf8bf (0xccc0 bytes) 2024-09-13T09:12:58.387 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:12:58.387 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:12:58.388 INFO:teuthology.orchestra.run.smithi157.stdout:2868 write 0x2a4f000 thru 0x2a59376 (0xa377 bytes) 2024-09-13T09:12:58.397 INFO:teuthology.orchestra.run.smithi157.stdout:2869 trunc from 0x5acfa00 to 0xd973800 2024-09-13T09:12:58.418 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:12:58.419 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:12:58.424 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:58.424 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:58.434 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:12:58.435 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:12:58.435 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:12:58.453 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:12:58.454 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:12:58.507 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:58.506+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:58.545 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:58.545 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:58.554 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:12:58.595 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:58.594+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:58.678 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:12:58.677+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:12:58.700 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:58.700 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:58.709 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:12:58.779 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:58.777+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:58.856 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:58.856 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:58.865 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:12:58.879 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:58.878+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:58.970 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:58.971 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:58.979 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:12:58.980 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:58.979+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:59.081 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:59.079+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:59.085 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:59.085 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:59.098 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:12:59.182 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:59.181+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:59.206 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:59.206 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:59.215 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:12:59.283 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:12:59.281+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:12:59.330 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:59.330 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:59.331 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-09-13T09:12:59.341 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:12:59.354 INFO:teuthology.orchestra.run.smithi157.stdout:2870 punch from 0x31b1600 to 0x31b9a00, (0x8400 bytes) 2024-09-13T09:12:59.380 INFO:teuthology.orchestra.run.smithi157.stdout:2871 trunc from 0xd973800 to 0xd85600 2024-09-13T09:12:59.383 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:59.382+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:59.408 INFO:teuthology.orchestra.run.smithi157.stdout:2872 trunc from 0xd85600 to 0x68c1800 2024-09-13T09:12:59.418 INFO:teuthology.orchestra.run.smithi157.stdout:2873 read 0x30b0000 thru 0x30b280a (0x280b bytes) 2024-09-13T09:12:59.419 INFO:teuthology.orchestra.run.smithi157.stdout:2879 punch from 0x819200 to 0x822a00, (0x9800 bytes) 2024-09-13T09:12:59.431 INFO:teuthology.orchestra.run.smithi157.stdout:2880 write 0x7d2a400 thru 0x7d397e4 (0xf3e5 bytes) 2024-09-13T09:12:59.438 INFO:teuthology.orchestra.run.smithi157.stdout:2881 trunc from 0x7d39800 to 0x2707200 2024-09-13T09:12:59.484 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:12:59.483+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:12:59.486 INFO:teuthology.orchestra.run.smithi157.stdout:2882 write 0x5295c00 thru 0x5295cd9 (0xda bytes) 2024-09-13T09:12:59.486 INFO:teuthology.orchestra.run.smithi157.stdout:2883 read 0x2408800 thru 0x240fb37 (0x7338 bytes) 2024-09-13T09:12:59.487 INFO:teuthology.orchestra.run.smithi157.stdout:2887 punch from 0x39eec00 to 0x39f4600, (0x5a00 bytes) 2024-09-13T09:12:59.487 INFO:teuthology.orchestra.run.smithi157.stdout:2889 read 0x1598600 thru 0x15a0ccf (0x86d0 bytes) 2024-09-13T09:12:59.487 INFO:teuthology.orchestra.run.smithi157.stdout:2890 read 0x2b1ec00 thru 0x2b268a3 (0x7ca4 bytes) 2024-09-13T09:12:59.489 INFO:teuthology.orchestra.run.smithi157.stdout:2891 trunc from 0x5295e00 to 0x46f4800 2024-09-13T09:12:59.495 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:59.495 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:59.505 INFO:teuthology.orchestra.run.smithi157.stdout:2892 punch from 0x149b400 to 0x14a4e00, (0x9a00 bytes) 2024-09-13T09:12:59.506 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:12:59.506 INFO:teuthology.orchestra.run.smithi157.stdout:2893 clone 19 order 18 su 4096 sc 6 2024-09-13T09:12:59.585 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:59.584+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:59.648 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:59.648 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:59.657 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:12:59.686 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:12:59.685+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:12:59.759 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:59.759 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:59.767 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:12:59.787 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:12:59.787+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:12:59.870 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:12:59.871 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:12:59.879 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:12:59.942 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:12:59.921+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:12:59.982 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:00.033 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:00.035 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:00.035 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:00.033+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:00.135 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:00.134+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:00.177 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:00.178 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:00.186 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:00.237 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:00.236+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:00.328 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-13T09:13:00.329 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:13:00.329 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T09:13:00.329 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:13:00.330 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:00.330 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:00.338 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:00.338 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:00.336+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:00.438 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:00.437+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:00.467 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:13:00.484 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:00.484 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:00.484 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone18 from 0x46f4800 (overlap 0xaa2200) to 0x551000 2024-09-13T09:13:00.494 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:00.555 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:00.554+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:00.601 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:00.602 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:00.620 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:00.620 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:00.627 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:00.627 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:00.627 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:00.627 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:00.636 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:00.638 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:13:00.639 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:13:00.641 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:00.639+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:00.654 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:13:00.654 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:00.654 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 64, 2024-09-13T09:13:00.654 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "06cda68b-dd73-4266-b5e6-93ae2b3f7bd5", 2024-09-13T09:13:00.655 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2024-09-13T09:11:45.458147+0000", 2024-09-13T09:13:00.655 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T09:13:00.334436+0000", 2024-09-13T09:13:00.655 INFO:teuthology.orchestra.run.smithi044.stdout: "last_up_change": "2024-09-13T09:11:56.271910+0000", 2024-09-13T09:13:00.655 INFO:teuthology.orchestra.run.smithi044.stdout: "last_in_change": "2024-09-13T09:12:42.513669+0000", 2024-09-13T09:13:00.655 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:13:00.655 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_num": 5799936, 2024-09-13T09:13:00.655 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_set": [ 2024-09-13T09:13:00.655 INFO:teuthology.orchestra.run.smithi044.stdout: "pglog_hardlimit", 2024-09-13T09:13:00.655 INFO:teuthology.orchestra.run.smithi044.stdout: "purged_snapdirs", 2024-09-13T09:13:00.655 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_deletes", 2024-09-13T09:13:00.655 INFO:teuthology.orchestra.run.smithi044.stdout: "sortbitwise" 2024-09-13T09:13:00.655 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:13:00.655 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_version": 4, 2024-09-13T09:13:00.656 INFO:teuthology.orchestra.run.smithi044.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:13:00.656 INFO:teuthology.orchestra.run.smithi044.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:13:00.656 INFO:teuthology.orchestra.run.smithi044.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:13:00.656 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_snapshot": "", 2024-09-13T09:13:00.656 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_max": 3, 2024-09-13T09:13:00.656 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd": 6, 2024-09-13T09:13:00.656 INFO:teuthology.orchestra.run.smithi044.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:13:00.656 INFO:teuthology.orchestra.run.smithi044.stdout: "min_compat_client": "luminous", 2024-09-13T09:13:00.656 INFO:teuthology.orchestra.run.smithi044.stdout: "require_osd_release": "reef", 2024-09-13T09:13:00.656 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_crimson": false, 2024-09-13T09:13:00.656 INFO:teuthology.orchestra.run.smithi044.stdout: "pools": [ 2024-09-13T09:13:00.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.656 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 1, 2024-09-13T09:13:00.657 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": ".mgr", 2024-09-13T09:13:00.657 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-09-13T09:11:57.562930+0000", 2024-09-13T09:13:00.657 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T09:13:00.657 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool", 2024-09-13T09:13:00.657 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-09-13T09:13:00.657 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-09-13T09:13:00.657 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-09-13T09:13:00.657 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-09-13T09:13:00.657 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:13:00.657 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:13:00.657 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:13:00.657 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:13:00.657 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-09-13T09:13:00.657 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:13:00.657 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 1, 2024-09-13T09:13:00.658 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 1, 2024-09-13T09:13:00.658 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 1, 2024-09-13T09:13:00.658 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 1, 2024-09-13T09:13:00.658 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 1, 2024-09-13T09:13:00.658 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-09-13T09:13:00.658 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-09-13T09:13:00.658 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-09-13T09:13:00.658 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-09-13T09:13:00.658 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-09-13T09:13:00.658 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-09-13T09:13:00.658 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-09-13T09:13:00.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.658 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "15", 2024-09-13T09:13:00.659 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-09-13T09:13:00.659 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:13:00.659 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:13:00.659 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-09-13T09:13:00.659 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:13:00.659 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 0, 2024-09-13T09:13:00.659 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 0, 2024-09-13T09:13:00.659 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-09-13T09:13:00.659 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-09-13T09:13:00.659 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-09-13T09:13:00.659 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-09-13T09:13:00.659 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-09-13T09:13:00.659 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-09-13T09:13:00.659 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-09-13T09:13:00.660 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-09-13T09:13:00.660 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-09-13T09:13:00.660 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-09-13T09:13:00.660 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-09-13T09:13:00.660 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:13:00.660 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:13:00.660 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:13:00.660 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-09-13T09:13:00.660 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-09-13T09:13:00.660 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-09-13T09:13:00.660 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-09-13T09:13:00.660 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-09-13T09:13:00.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.660 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-09-13T09:13:00.661 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-09-13T09:13:00.661 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-09-13T09:13:00.661 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:13:00.661 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:13:00.661 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:13:00.661 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:13:00.661 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-09-13T09:13:00.661 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-09-13T09:13:00.661 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-09-13T09:13:00.661 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-09-13T09:13:00.661 INFO:teuthology.orchestra.run.smithi044.stdout: "options": { 2024-09-13T09:13:00.661 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_max": 32, 2024-09-13T09:13:00.661 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_min": 1 2024-09-13T09:13:00.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.662 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-09-13T09:13:00.662 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": {} 2024-09-13T09:13:00.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.662 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-09-13T09:13:00.662 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 6.059999942779541, 2024-09-13T09:13:00.662 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 6.059999942779541, 2024-09-13T09:13:00.662 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 0.33000001311302185, 2024-09-13T09:13:00.662 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 2, 2024-09-13T09:13:00.662 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 2, 2024-09-13T09:13:00.662 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:13:00.662 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 1, 2024-09-13T09:13:00.662 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:13:00.662 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.663 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 2, 2024-09-13T09:13:00.663 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": "rbd", 2024-09-13T09:13:00.663 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-09-13T09:12:03.214789+0000", 2024-09-13T09:13:00.663 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 8193, 2024-09-13T09:13:00.663 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:13:00.663 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-09-13T09:13:00.663 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-09-13T09:13:00.663 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-09-13T09:13:00.663 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-09-13T09:13:00.663 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:13:00.663 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:13:00.663 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:13:00.664 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:13:00.664 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-09-13T09:13:00.664 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:13:00.664 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 8, 2024-09-13T09:13:00.664 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 8, 2024-09-13T09:13:00.664 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 8, 2024-09-13T09:13:00.664 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 8, 2024-09-13T09:13:00.664 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 8, 2024-09-13T09:13:00.664 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-09-13T09:13:00.664 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-09-13T09:13:00.664 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-09-13T09:13:00.664 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-09-13T09:13:00.664 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-09-13T09:13:00.664 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-09-13T09:13:00.665 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-09-13T09:13:00.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.665 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "19", 2024-09-13T09:13:00.665 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-09-13T09:13:00.665 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:13:00.665 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:13:00.665 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-09-13T09:13:00.665 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:13:00.665 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 3, 2024-09-13T09:13:00.665 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 19, 2024-09-13T09:13:00.665 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-09-13T09:13:00.665 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-09-13T09:13:00.665 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-09-13T09:13:00.665 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-09-13T09:13:00.666 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-09-13T09:13:00.666 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-09-13T09:13:00.666 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-09-13T09:13:00.666 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-09-13T09:13:00.666 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-09-13T09:13:00.666 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-09-13T09:13:00.666 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-09-13T09:13:00.666 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:13:00.666 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:13:00.666 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:13:00.666 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-09-13T09:13:00.666 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-09-13T09:13:00.666 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-09-13T09:13:00.666 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-09-13T09:13:00.667 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-09-13T09:13:00.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.667 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-09-13T09:13:00.667 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-09-13T09:13:00.667 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-09-13T09:13:00.667 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:13:00.667 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:13:00.667 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:13:00.667 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:13:00.667 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-09-13T09:13:00.667 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-09-13T09:13:00.667 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-09-13T09:13:00.667 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-09-13T09:13:00.667 INFO:teuthology.orchestra.run.smithi044.stdout: "options": {}, 2024-09-13T09:13:00.668 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-09-13T09:13:00.668 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd": {} 2024-09-13T09:13:00.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.668 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-09-13T09:13:00.668 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 2.25, 2024-09-13T09:13:00.668 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 2.25, 2024-09-13T09:13:00.668 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 1, 2024-09-13T09:13:00.668 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 2.25, 2024-09-13T09:13:00.668 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 2.25, 2024-09-13T09:13:00.668 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:13:00.668 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 1, 2024-09-13T09:13:00.668 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:13:00.668 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.669 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 3, 2024-09-13T09:13:00.669 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": "pool_client.0", 2024-09-13T09:13:00.669 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-09-13T09:12:12.120436+0000", 2024-09-13T09:13:00.669 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 8193, 2024-09-13T09:13:00.669 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:13:00.669 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-09-13T09:13:00.669 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-09-13T09:13:00.669 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-09-13T09:13:00.669 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-09-13T09:13:00.670 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:13:00.670 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:13:00.670 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:13:00.670 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:13:00.670 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-09-13T09:13:00.670 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:13:00.670 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 32, 2024-09-13T09:13:00.670 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 32, 2024-09-13T09:13:00.670 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 32, 2024-09-13T09:13:00.670 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 32, 2024-09-13T09:13:00.671 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 32, 2024-09-13T09:13:00.671 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-09-13T09:13:00.671 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-09-13T09:13:00.671 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-09-13T09:13:00.671 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-09-13T09:13:00.671 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-09-13T09:13:00.671 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-09-13T09:13:00.671 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-09-13T09:13:00.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.671 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "64", 2024-09-13T09:13:00.671 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-09-13T09:13:00.671 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:13:00.671 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:13:00.671 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-09-13T09:13:00.672 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:13:00.672 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 22, 2024-09-13T09:13:00.672 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 64, 2024-09-13T09:13:00.672 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-09-13T09:13:00.672 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-09-13T09:13:00.672 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-09-13T09:13:00.672 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-09-13T09:13:00.672 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-09-13T09:13:00.672 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-09-13T09:13:00.672 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-09-13T09:13:00.672 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-09-13T09:13:00.672 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-09-13T09:13:00.672 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-09-13T09:13:00.673 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-09-13T09:13:00.673 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:13:00.673 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:13:00.673 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:13:00.673 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-09-13T09:13:00.673 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-09-13T09:13:00.673 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-09-13T09:13:00.673 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-09-13T09:13:00.673 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-09-13T09:13:00.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.673 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-09-13T09:13:00.673 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-09-13T09:13:00.673 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-09-13T09:13:00.673 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:13:00.674 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:13:00.674 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:13:00.674 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:13:00.674 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-09-13T09:13:00.674 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-09-13T09:13:00.674 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-09-13T09:13:00.674 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-09-13T09:13:00.674 INFO:teuthology.orchestra.run.smithi044.stdout: "options": {}, 2024-09-13T09:13:00.674 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-09-13T09:13:00.674 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd": {} 2024-09-13T09:13:00.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.674 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-09-13T09:13:00.674 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 1.5, 2024-09-13T09:13:00.675 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 1.5, 2024-09-13T09:13:00.675 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 1, 2024-09-13T09:13:00.675 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 1.5, 2024-09-13T09:13:00.675 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 1.5, 2024-09-13T09:13:00.675 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:13:00.675 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 1, 2024-09-13T09:13:00.675 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:13:00.675 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.675 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.675 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:13:00.675 INFO:teuthology.orchestra.run.smithi044.stdout: "osds": [ 2024-09-13T09:13:00.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.675 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 0, 2024-09-13T09:13:00.676 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "156c6730-f804-4806-bc2d-3a7d0856c2b2", 2024-09-13T09:13:00.676 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:13:00.676 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:13:00.676 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:13:00.676 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:13:00.676 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:13:00.676 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:13:00.676 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:13:00.676 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 61, 2024-09-13T09:13:00.676 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:13:00.676 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:13:00.676 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:13:00.676 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.677 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.677 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6808", 2024-09-13T09:13:00.677 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:00.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.677 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.677 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6810", 2024-09-13T09:13:00.677 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:00.677 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.677 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.677 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:13:00.677 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.678 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.678 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6813", 2024-09-13T09:13:00.678 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:00.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.678 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.678 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6815", 2024-09-13T09:13:00.678 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:00.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.678 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.678 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:13:00.678 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.678 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.679 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6821", 2024-09-13T09:13:00.679 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:00.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.679 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.679 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6823", 2024-09-13T09:13:00.679 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:00.679 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.679 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.679 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:13:00.679 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.679 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.680 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6817", 2024-09-13T09:13:00.680 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:00.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.680 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.680 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6819", 2024-09-13T09:13:00.680 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:00.680 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.680 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.681 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6810/2268459392", 2024-09-13T09:13:00.681 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6815/2268459392", 2024-09-13T09:13:00.681 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6823/2268459392", 2024-09-13T09:13:00.681 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6819/2268459392", 2024-09-13T09:13:00.681 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:13:00.681 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:13:00.681 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:13:00.681 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.681 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 1, 2024-09-13T09:13:00.681 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "00735025-3bf9-4dd0-8b42-f08f1bd1c421", 2024-09-13T09:13:00.681 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:13:00.682 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:13:00.682 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:13:00.682 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:13:00.682 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:13:00.682 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:13:00.682 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:13:00.682 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 48, 2024-09-13T09:13:00.682 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:13:00.682 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:13:00.682 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:13:00.682 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.682 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.682 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6800", 2024-09-13T09:13:00.683 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:00.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.683 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.683 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6801", 2024-09-13T09:13:00.683 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:00.683 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.683 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.683 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:13:00.683 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.683 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.683 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6802", 2024-09-13T09:13:00.684 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:00.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.684 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.684 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6803", 2024-09-13T09:13:00.684 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:00.684 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.684 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.684 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:13:00.684 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.684 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.684 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6806", 2024-09-13T09:13:00.685 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:00.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.685 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.685 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6807", 2024-09-13T09:13:00.685 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:00.685 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.685 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.685 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:13:00.685 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.685 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.685 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6804", 2024-09-13T09:13:00.685 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:00.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.686 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.686 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6805", 2024-09-13T09:13:00.686 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:00.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.686 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.686 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6801/2209612586", 2024-09-13T09:13:00.686 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6803/2209612586", 2024-09-13T09:13:00.686 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6807/2209612586", 2024-09-13T09:13:00.686 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6805/2209612586", 2024-09-13T09:13:00.686 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:13:00.686 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:13:00.687 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:13:00.687 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.687 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 2, 2024-09-13T09:13:00.687 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "395516ac-33b6-4f43-8333-dcb1f4cded32", 2024-09-13T09:13:00.687 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:13:00.687 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:13:00.687 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:13:00.687 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:13:00.687 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:13:00.687 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:13:00.687 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:13:00.687 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 62, 2024-09-13T09:13:00.688 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:13:00.688 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:13:00.688 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:13:00.688 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.688 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.688 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6809", 2024-09-13T09:13:00.688 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:00.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.688 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.688 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6811", 2024-09-13T09:13:00.688 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:00.688 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.689 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.689 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:13:00.689 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.689 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.689 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6812", 2024-09-13T09:13:00.689 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:00.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.689 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.689 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6814", 2024-09-13T09:13:00.689 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:00.689 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.690 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.690 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:13:00.690 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.690 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.690 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6820", 2024-09-13T09:13:00.690 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:00.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.690 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.690 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6822", 2024-09-13T09:13:00.690 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:00.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.691 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.691 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:13:00.691 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.691 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6816", 2024-09-13T09:13:00.691 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:00.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.691 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6818", 2024-09-13T09:13:00.691 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:00.691 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.692 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.692 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6811/770926265", 2024-09-13T09:13:00.692 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6814/770926265", 2024-09-13T09:13:00.692 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6822/770926265", 2024-09-13T09:13:00.692 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6818/770926265", 2024-09-13T09:13:00.692 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:13:00.692 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:13:00.692 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:13:00.692 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.692 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 3, 2024-09-13T09:13:00.692 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "429e877f-5462-402a-80e5-92a7b17a2eaa", 2024-09-13T09:13:00.693 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:13:00.693 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:13:00.693 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:13:00.693 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:13:00.693 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:13:00.693 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:13:00.693 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:13:00.693 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 49, 2024-09-13T09:13:00.693 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:13:00.693 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:13:00.693 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:13:00.693 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.693 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.694 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6808", 2024-09-13T09:13:00.694 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:00.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.694 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.694 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6809", 2024-09-13T09:13:00.694 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:00.694 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.694 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.694 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:13:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6810", 2024-09-13T09:13:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6811", 2024-09-13T09:13:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:13:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6814", 2024-09-13T09:13:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6815", 2024-09-13T09:13:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:13:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6812", 2024-09-13T09:13:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6813", 2024-09-13T09:13:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.129:6809/2132151186", 2024-09-13T09:13:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.129:6811/2132151186", 2024-09-13T09:13:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.129:6815/2132151186", 2024-09-13T09:13:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.129:6813/2132151186", 2024-09-13T09:13:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:13:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:13:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:13:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 4, 2024-09-13T09:13:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "9b3be757-15ee-4157-80a8-0537cd1bcf28", 2024-09-13T09:13:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:13:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:13:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:13:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:13:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:13:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:13:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:13:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 48, 2024-09-13T09:13:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:13:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:13:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:13:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6800", 2024-09-13T09:13:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6801", 2024-09-13T09:13:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:13:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6802", 2024-09-13T09:13:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6803", 2024-09-13T09:13:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:13:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6806", 2024-09-13T09:13:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6807", 2024-09-13T09:13:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:13:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6804", 2024-09-13T09:13:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6805", 2024-09-13T09:13:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.129:6801/3393542416", 2024-09-13T09:13:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.129:6803/3393542416", 2024-09-13T09:13:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.129:6807/3393542416", 2024-09-13T09:13:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.129:6805/3393542416", 2024-09-13T09:13:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:13:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:13:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:13:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 5, 2024-09-13T09:13:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "c7601d5e-2164-45e1-ab21-368d9e3af27e", 2024-09-13T09:13:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:13:00.707 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:13:00.707 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:13:00.707 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:13:00.707 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:13:00.707 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:13:00.707 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:13:00.707 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 49, 2024-09-13T09:13:00.707 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:13:00.707 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:13:00.707 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:13:00.707 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.708 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.708 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6816", 2024-09-13T09:13:00.708 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:00.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.708 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.708 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6817", 2024-09-13T09:13:00.708 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:00.708 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.708 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.709 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:13:00.709 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.709 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.709 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6818", 2024-09-13T09:13:00.709 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:00.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.709 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.709 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6819", 2024-09-13T09:13:00.709 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:00.710 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.710 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.710 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:13:00.710 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.710 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.710 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6822", 2024-09-13T09:13:00.710 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:00.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.710 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.711 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6823", 2024-09-13T09:13:00.711 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:00.711 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.711 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.711 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:13:00.711 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:00.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.711 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:00.711 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6820", 2024-09-13T09:13:00.711 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:00.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.712 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:00.712 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6821", 2024-09-13T09:13:00.712 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:00.712 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.712 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.712 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.129:6817/1803198188", 2024-09-13T09:13:00.712 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.129:6819/1803198188", 2024-09-13T09:13:00.712 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.129:6823/1803198188", 2024-09-13T09:13:00.712 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.129:6821/1803198188", 2024-09-13T09:13:00.713 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:13:00.713 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:13:00.713 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:13:00.713 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.713 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.713 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:13:00.713 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_xinfo": [ 2024-09-13T09:13:00.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.713 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 0, 2024-09-13T09:13:00.713 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:13:00.713 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:13:00.713 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:13:00.714 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:13:00.714 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:13:00.714 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.230474+0000", 2024-09-13T09:13:00.714 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:13:00.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.714 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 1, 2024-09-13T09:13:00.714 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:13:00.714 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:13:00.714 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:13:00.714 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:13:00.714 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:13:00.715 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.195476+0000", 2024-09-13T09:13:00.715 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:13:00.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.715 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 2, 2024-09-13T09:13:00.715 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:13:00.715 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:13:00.715 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:13:00.715 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:13:00.715 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:13:00.715 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.248729+0000", 2024-09-13T09:13:00.715 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:13:00.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.716 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 3, 2024-09-13T09:13:00.716 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:13:00.716 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:13:00.716 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:13:00.716 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:13:00.716 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:13:00.716 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.195482+0000", 2024-09-13T09:13:00.716 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:13:00.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.717 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 4, 2024-09-13T09:13:00.717 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:13:00.717 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:13:00.717 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:13:00.717 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:13:00.717 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:13:00.717 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.198746+0000", 2024-09-13T09:13:00.717 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:13:00.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.717 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 5, 2024-09-13T09:13:00.718 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:13:00.718 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:13:00.718 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:13:00.718 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:13:00.718 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:13:00.718 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.279957+0000", 2024-09-13T09:13:00.718 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:13:00.718 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.718 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:13:00.718 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap": [ 2024-09-13T09:13:00.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.718 INFO:teuthology.orchestra.run.smithi044.stdout: "pgid": "3.1b", 2024-09-13T09:13:00.719 INFO:teuthology.orchestra.run.smithi044.stdout: "osds": [ 2024-09-13T09:13:00.719 INFO:teuthology.orchestra.run.smithi044.stdout: 2, 2024-09-13T09:13:00.719 INFO:teuthology.orchestra.run.smithi044.stdout: 3 2024-09-13T09:13:00.719 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.719 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.719 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:13:00.719 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap_items": [], 2024-09-13T09:13:00.719 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap_primaries": [], 2024-09-13T09:13:00.719 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_temp": [], 2024-09-13T09:13:00.719 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_temp": [], 2024-09-13T09:13:00.719 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklist": {}, 2024-09-13T09:13:00.719 INFO:teuthology.orchestra.run.smithi044.stdout: "range_blocklist": {}, 2024-09-13T09:13:00.720 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profiles": { 2024-09-13T09:13:00.720 INFO:teuthology.orchestra.run.smithi044.stdout: "default": { 2024-09-13T09:13:00.720 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-failure-domain": "osd", 2024-09-13T09:13:00.720 INFO:teuthology.orchestra.run.smithi044.stdout: "k": "2", 2024-09-13T09:13:00.720 INFO:teuthology.orchestra.run.smithi044.stdout: "m": "1", 2024-09-13T09:13:00.720 INFO:teuthology.orchestra.run.smithi044.stdout: "plugin": "jerasure", 2024-09-13T09:13:00.720 INFO:teuthology.orchestra.run.smithi044.stdout: "technique": "reed_sol_van" 2024-09-13T09:13:00.720 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.721 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps_queue": [], 2024-09-13T09:13:00.721 INFO:teuthology.orchestra.run.smithi044.stdout: "new_removed_snaps": [], 2024-09-13T09:13:00.721 INFO:teuthology.orchestra.run.smithi044.stdout: "new_purged_snaps": [], 2024-09-13T09:13:00.721 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_node_flags": {}, 2024-09-13T09:13:00.721 INFO:teuthology.orchestra.run.smithi044.stdout: "device_class_flags": {}, 2024-09-13T09:13:00.721 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": { 2024-09-13T09:13:00.721 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode_enabled": false, 2024-09-13T09:13:00.721 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_bucket_count": 0, 2024-09-13T09:13:00.722 INFO:teuthology.orchestra.run.smithi044.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:13:00.722 INFO:teuthology.orchestra.run.smithi044.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:13:00.722 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode_bucket": 0 2024-09-13T09:13:00.722 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.722 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:00.723 INFO:tasks.thrashosds.thrasher:j is {'epoch': 64, 'fsid': '06cda68b-dd73-4266-b5e6-93ae2b3f7bd5', 'created': '2024-09-13T09:11:45.458147+0000', 'modified': '2024-09-13T09:13:00.334436+0000', 'last_up_change': '2024-09-13T09:11:56.271910+0000', 'last_in_change': '2024-09-13T09:12:42.513669+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:11:57.562930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:12:03.214789+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:12:12.120436+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 22, 'snap_epoch': 64, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '156c6730-f804-4806-bc2d-3a7d0856c2b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6808', 'nonce': 2268459392}, {'type': 'v1', 'addr': '172.21.15.44:6810', 'nonce': 2268459392}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6813', 'nonce': 2268459392}, {'type': 'v1', 'addr': '172.21.15.44:6815', 'nonce': 2268459392}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6821', 'nonce': 2268459392}, {'type': 'v1', 'addr': '172.21.15.44:6823', 'nonce': 2268459392}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6817', 'nonce': 2268459392}, {'type': 'v1', 'addr': '172.21.15.44:6819', 'nonce': 2268459392}]}, 'public_addr': '172.21.15.44:6810/2268459392', 'cluster_addr': '172.21.15.44:6815/2268459392', 'heartbeat_back_addr': '172.21.15.44:6823/2268459392', 'heartbeat_front_addr': '172.21.15.44:6819/2268459392', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '00735025-3bf9-4dd0-8b42-f08f1bd1c421', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6800', 'nonce': 2209612586}, {'type': 'v1', 'addr': '172.21.15.44:6801', 'nonce': 2209612586}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6802', 'nonce': 2209612586}, {'type': 'v1', 'addr': '172.21.15.44:6803', 'nonce': 2209612586}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6806', 'nonce': 2209612586}, {'type': 'v1', 'addr': '172.21.15.44:6807', 'nonce': 2209612586}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6804', 'nonce': 2209612586}, {'type': 'v1', 'addr': '172.21.15.44:6805', 'nonce': 2209612586}]}, 'public_addr': '172.21.15.44:6801/2209612586', 'cluster_addr': '172.21.15.44:6803/2209612586', 'heartbeat_back_addr': '172.21.15.44:6807/2209612586', 'heartbeat_front_addr': '172.21.15.44:6805/2209612586', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '395516ac-33b6-4f43-8333-dcb1f4cded32', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6809', 'nonce': 770926265}, {'type': 'v1', 'addr': '172.21.15.44:6811', 'nonce': 770926265}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6812', 'nonce': 770926265}, {'type': 'v1', 'addr': '172.21.15.44:6814', 'nonce': 770926265}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6820', 'nonce': 770926265}, {'type': 'v1', 'addr': '172.21.15.44:6822', 'nonce': 770926265}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6816', 'nonce': 770926265}, {'type': 'v1', 'addr': '172.21.15.44:6818', 'nonce': 770926265}]}, 'public_addr': '172.21.15.44:6811/770926265', 'cluster_addr': '172.21.15.44:6814/770926265', 'heartbeat_back_addr': '172.21.15.44:6822/770926265', 'heartbeat_front_addr': '172.21.15.44:6818/770926265', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '429e877f-5462-402a-80e5-92a7b17a2eaa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6808', 'nonce': 2132151186}, {'type': 'v1', 'addr': '172.21.15.129:6809', 'nonce': 2132151186}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6810', 'nonce': 2132151186}, {'type': 'v1', 'addr': '172.21.15.129:6811', 'nonce': 2132151186}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6814', 'nonce': 2132151186}, {'type': 'v1', 'addr': '172.21.15.129:6815', 'nonce': 2132151186}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6812', 'nonce': 2132151186}, {'type': 'v1', 'addr': '172.21.15.129:6813', 'nonce': 2132151186}]}, 'public_addr': '172.21.15.129:6809/2132151186', 'cluster_addr': '172.21.15.129:6811/2132151186', 'heartbeat_back_addr': '172.21.15.129:6815/2132151186', 'heartbeat_front_addr': '172.21.15.129:6813/2132151186', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '9b3be757-15ee-4157-80a8-0537cd1bcf28', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6800', 'nonce': 3393542416}, {'type': 'v1', 'addr': '172.21.15.129:6801', 'nonce': 3393542416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6802', 'nonce': 3393542416}, {'type': 'v1', 'addr': '172.21.15.129:6803', 'nonce': 3393542416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6806', 'nonce': 3393542416}, {'type': 'v1', 'addr': '172.21.15.129:6807', 'nonce': 3393542416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6804', 'nonce': 3393542416}, {'type': 'v1', 'addr': '172.21.15.129:6805', 'nonce': 3393542416}]}, 'public_addr': '172.21.15.129:6801/3393542416', 'cluster_addr': '172.21.15.129:6803/3393542416', 'heartbeat_back_addr': '172.21.15.129:6807/3393542416', 'heartbeat_front_addr': '172.21.15.129:6805/3393542416', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c7601d5e-2164-45e1-ab21-368d9e3af27e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6816', 'nonce': 1803198188}, {'type': 'v1', 'addr': '172.21.15.129:6817', 'nonce': 1803198188}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6818', 'nonce': 1803198188}, {'type': 'v1', 'addr': '172.21.15.129:6819', 'nonce': 1803198188}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6822', 'nonce': 1803198188}, {'type': 'v1', 'addr': '172.21.15.129:6823', 'nonce': 1803198188}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6820', 'nonce': 1803198188}, {'type': 'v1', 'addr': '172.21.15.129:6821', 'nonce': 1803198188}]}, 'public_addr': '172.21.15.129:6817/1803198188', 'cluster_addr': '172.21.15.129:6819/1803198188', 'heartbeat_back_addr': '172.21.15.129:6823/1803198188', 'heartbeat_front_addr': '172.21.15.129:6821/1803198188', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.230474+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.195476+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.248729+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.195482+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.198746+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.279957+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1b', 'osds': [2, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:13:00.724 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:13:00.741 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:00.740+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:00.746 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:00.746 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:00.747 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:00.747 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:00.747 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:00.756 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:00.842 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:00.841+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:00.860 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:00.860 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:00.876 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:00.876 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:00.892 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:00.893 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:00.912 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:00.912 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:00.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:00.913 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:00.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.913 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:00.913 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:00.913 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.380356093, 2024-09-13T09:13:00.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:13:00.913 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:00.913 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:00.913 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:00.913 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:00.914 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:00.914 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:13:00.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.914 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:00.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:00.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:00.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:00.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:00.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:00.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:13:00.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:00.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:00.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:13:00.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:13:00.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:00.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:13:00.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:13:00.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:00.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:13:00.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:13:00.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:13:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:13:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:13:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:13:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:13:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:13:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:13:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:13:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:13:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:13:00.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:13:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:13:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:13:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:13:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.919 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.920 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:00.920 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:00.920 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.380338058, 2024-09-13T09:13:00.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:13:00.920 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:00.920 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:00.920 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:00.920 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:00.920 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:00.921 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:13:00.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.921 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:00.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:00.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:00.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:00.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:00.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:00.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:00.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:00.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:00.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:13:00.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:13:00.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:00.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:13:00.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:13:00.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:00.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:13:00.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:13:00.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:13:00.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:13:00.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:13:00.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:13:00.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:13:00.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:13:00.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:13:00.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:13:00.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:13:00.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:13:00.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:00.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:13:00.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:13:00.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:00.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:13:00.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:13:00.927 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.927 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.927 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.927 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:00.927 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:00.927 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.380321387999999, 2024-09-13T09:13:00.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:13:00.928 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:00.928 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:00.928 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:00.928 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:00.928 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:00.928 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:13:00.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.928 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:00.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:00.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:00.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.929 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:00.929 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:00.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.929 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:00.929 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:13:00.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:00.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:00.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:13:00.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:00.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:00.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:13:00.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:13:00.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:00.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:13:00.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:13:00.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:13:00.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:13:00.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:13:00.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:13:00.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:13:00.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:13:00.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:13:00.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:13:00.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:13:00.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:13:00.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:00.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:13:00.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:13:00.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:00.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:13:00.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:13:00.934 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.934 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.934 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.934 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:00.934 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:00.934 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.380303756, 2024-09-13T09:13:00.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:13:00.935 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:00.935 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:00.935 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:00.935 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:00.935 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:00.935 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:13:00.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.935 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:00.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:00.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:00.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:00.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:00.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:00.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:00.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:00.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:00.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:13:00.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:13:00.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:00.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:13:00.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:13:00.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:00.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:13:00.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:13:00.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:13:00.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:13:00.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:13:00.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:13:00.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.941 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:13:00.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:13:00.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.941 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:13:00.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:13:00.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:13:00.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:13:00.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:00.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:13:00.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:13:00.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:00.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:13:00.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:13:00.943 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.943 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.943 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.943 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:00.943 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:00.943 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.380257450999999, 2024-09-13T09:13:00.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:13:00.943 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:00.944 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:00.944 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:00.944 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:00.944 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:00.944 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:13:00.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.944 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:00.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:00.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:00.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:00.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:00.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:00.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:13:00.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:00.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:00.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:13:00.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:13:00.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:00.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:13:00.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:13:00.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:00.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:13:00.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:13:00.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:13:00.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:13:00.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:13:00.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:13:00.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:13:00.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:13:00.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:13:00.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:13:00.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:13:00.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:13:00.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:00.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:13:00.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:13:00.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:00.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:13:00.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:13:00.950 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.950 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.950 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.950 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:00.950 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:00.950 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.380232150000001, 2024-09-13T09:13:00.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:13:00.951 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:00.951 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:00.951 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:00.951 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:00.951 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:00.951 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:13:00.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.951 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:00.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:00.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:00.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:00.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:00.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:00.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:13:00.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:00.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:00.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:13:00.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:13:00.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:00.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:13:00.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:13:00.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:00.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:13:00.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:13:00.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:13:00.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:13:00.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:13:00.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:13:00.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:13:00.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:13:00.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:13:00.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:13:00.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:13:00.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:13:00.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:00.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:13:00.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:13:00.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:00.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:13:00.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:13:00.957 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.957 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.957 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.957 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:00.957 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:00.957 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.380213920999999, 2024-09-13T09:13:00.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:13:00.958 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:00.958 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:00.958 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:00.958 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:00.958 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:00.958 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:13:00.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.958 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:00.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:00.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:00.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:00.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:00.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:00.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:00.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:00.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:00.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:13:00.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:13:00.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:00.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:13:00.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:13:00.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:00.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:13:00.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:13:00.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:13:00.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:13:00.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:13:00.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:13:00.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:13:00.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:13:00.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:13:00.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:13:00.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:13:00.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:13:00.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:00.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:13:00.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:13:00.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:00.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:13:00.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:13:00.963 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.963 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.963 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.964 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:00.964 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:00.964 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.380166601999999, 2024-09-13T09:13:00.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:13:00.964 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:00.964 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:00.964 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:00.964 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:00.964 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:00.964 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:13:00.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.965 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:00.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:00.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:00.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:00.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:00.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:00.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:00.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:00.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:00.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:13:00.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:13:00.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:00.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:13:00.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:13:00.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:00.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:13:00.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:13:00.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:13:00.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:13:00.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:13:00.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:13:00.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:13:00.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:13:00.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:13:00.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:13:00.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:13:00.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:13:00.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:00.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:13:00.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:13:00.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:00.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:13:00.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:13:00.970 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.970 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.970 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.971 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:00.971 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:00.971 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.379652364000002, 2024-09-13T09:13:00.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:13:00.971 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:00.971 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:00.971 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:00.971 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:00.971 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:00.971 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:13:00.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.972 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:00.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:00.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:00.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:00.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:00.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:00.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:13:00.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:13:00.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:00.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:13:00.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:13:00.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:00.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:13:00.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:13:00.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:00.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:13:00.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:13:00.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:13:00.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:13:00.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:13:00.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:13:00.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:13:00.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:13:00.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:13:00.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:13:00.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:13:00.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:13:00.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:00.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:13:00.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:13:00.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:00.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:13:00.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:13:00.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.977 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.978 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:00.978 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:00.978 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.379617834000001, 2024-09-13T09:13:00.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:13:00.978 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:00.978 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:00.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:00.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:00.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:00.979 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:13:00.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.979 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:00.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:00.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:00.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:00.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:00.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:00.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:00.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:00.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:00.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:13:00.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:13:00.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:00.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:13:00.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:13:00.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:00.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:13:00.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:13:00.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:13:00.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:13:00.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:13:00.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:13:00.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:13:00.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:13:00.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:13:00.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:13:00.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:13:00.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:13:00.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:00.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:13:00.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:13:00.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:00.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:13:00.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:13:00.984 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.379599927000001, 2024-09-13T09:13:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:13:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:13:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:13:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:13:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:13:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:13:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:13:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:13:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:13:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:13:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:13:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:13:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:13:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:13:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:13:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:13:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:13:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:13:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:13:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:13:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:13:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:13:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.370473079, 2024-09-13T09:13:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:13:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:13:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:13:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:13:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:13:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:13:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:13:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:13:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:13:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:13:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:13:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:13:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:13:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:13:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:13:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:13:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:13:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:13:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:13:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:13:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:13:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:13:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.370421498999999, 2024-09-13T09:13:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:13:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:13:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:13:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:13:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:13:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:13:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:13:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:13:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:13:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:13:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:13:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:13:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:13:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:13:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:13:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:13:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:13:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:13:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:13:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:13:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:13:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:13:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:13:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.370405222999999, 2024-09-13T09:13:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:13:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:13:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:13:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:13:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:13:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:13:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:13:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:13:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:13:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:13:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:13:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:13:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:13:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:13:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:13:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:13:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:13:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:13:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:13:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:13:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:13:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:13:01.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:13:01.014 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.014 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.014 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.014 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.014 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:01.014 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.370389449000001, 2024-09-13T09:13:01.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:13:01.014 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.015 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.015 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.015 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.015 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.015 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:13:01.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.015 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.015 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:01.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:13:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:13:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:13:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:13:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:13:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:13:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:13:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:13:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:13:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:13:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:13:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:13:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:13:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:13:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:13:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:13:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:13:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:13:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:13:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:13:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:13:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.370374574, 2024-09-13T09:13:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:13:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:13:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:13:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:13:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:13:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:13:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:13:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:13:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:13:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:13:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:13:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:13:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:13:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:13:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:13:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:13:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:13:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:13:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:13:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:13:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:13:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:13:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.370357112000001, 2024-09-13T09:13:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:13:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:13:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:13:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:13:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:13:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:13:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:13:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:13:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:13:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:13:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:13:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:13:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:13:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:13:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:13:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:13:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:13:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:13:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:13:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:13:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:13:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:13:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:13:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.368756479999998, 2024-09-13T09:13:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:13:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:13:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:13:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:13:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:13:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:13:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:13:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:13:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:13:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:13:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:13:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:13:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:13:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:13:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:13:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:13:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:13:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:13:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:13:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:13:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:13:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:13:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:13:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.368716428999999, 2024-09-13T09:13:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:13:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:13:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:13:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:13:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:13:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:13:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:13:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:13:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:13:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:13:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:13:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:13:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:13:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:13:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:13:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:13:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:13:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:13:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:13:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:13:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:13:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:13:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:13:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.368672665999998, 2024-09-13T09:13:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:13:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:13:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:13:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:13:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:13:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:13:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:13:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:13:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:13:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:13:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:13:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:13:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:13:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:13:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:13:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:13:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:13:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:13:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:13:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:13:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:13:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:13:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.056 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:01.058 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:01.056+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:01.059 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:01.060 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:13:01.061 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:13:01.062 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-09-13T09:13:00.339566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":831111332,"num_objects":614,"num_object_clones":195,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":614,"num_whiteouts":178,"num_read":122950,"num_read_kb":538295,"num_write":9040,"num_write_kb":140978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":9920327,"num_keys_recovered":33,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7664,"ondisk_log_size":7664,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":392224,"kb_used_data":164540,"kb_used_omap":9,"kb_used_meta":227574,"kb_avail":565838816,"statfs":{"total":579820584960,"available":579418947584,"internally_reserved":0,"allocated":168488960,"data_stored":170182434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9563,"internal_metadata":233036453},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":197770850,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":8,"num_read":16666,"num_read_kb":115052,"num_write":433,"num_write_kb":74614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.021388"},"pg_stats":[{"pgid":"3.1f","version":"59'129","reported_seq":1373,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.910227+0000","last_change":"2024-09-13T09:12:43.692870+0000","last_active":"2024-09-13T09:12:55.910227+0000","last_peered":"2024-09-13T09:12:55.910227+0000","last_clean":"2024-09-13T09:12:55.910227+0000","last_became_active":"2024-09-13T09:12:43.534990+0000","last_became_peered":"2024-09-13T09:12:43.534990+0000","last_unstale":"2024-09-13T09:12:55.910227+0000","last_undegraded":"2024-09-13T09:12:55.910227+0000","last_fullsized":"2024-09-13T09:12:55.910227+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:45.810610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058630900000000003,"stat_sum":{"num_bytes":8514576,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":1741,"num_read_kb":3462,"num_write":109,"num_write_kb":1463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":848912,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"59'131","reported_seq":1269,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.938174+0000","last_change":"2024-09-13T09:12:43.652049+0000","last_active":"2024-09-13T09:12:55.938174+0000","last_peered":"2024-09-13T09:12:55.938174+0000","last_clean":"2024-09-13T09:12:55.938174+0000","last_became_active":"2024-09-13T09:12:43.532920+0000","last_became_peered":"2024-09-13T09:12:43.532920+0000","last_unstale":"2024-09-13T09:12:55.938174+0000","last_undegraded":"2024-09-13T09:12:55.938174+0000","last_fullsized":"2024-09-13T09:12:55.938174+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:10:38.824649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088145900000000002,"stat_sum":{"num_bytes":32482832,"num_objects":25,"num_object_clones":9,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":9,"num_read":1598,"num_read_kb":8821,"num_write":123,"num_write_kb":5321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":311808,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"58'706","reported_seq":3116,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T09:12:54.537438+0000","last_change":"2024-09-13T09:12:16.388369+0000","last_active":"2024-09-13T09:12:54.537438+0000","last_peered":"2024-09-13T09:12:54.537438+0000","last_clean":"2024-09-13T09:12:54.537438+0000","last_became_active":"2024-09-13T09:12:13.362666+0000","last_became_peered":"2024-09-13T09:12:13.362666+0000","last_unstale":"2024-09-13T09:12:54.537438+0000","last_undegraded":"2024-09-13T09:12:54.537438+0000","last_fullsized":"2024-09-13T09:12:54.537438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:04.594826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039115800000000001,"stat_sum":{"num_bytes":5155995,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":5320,"num_read_kb":5622,"num_write":1071,"num_write_kb":1989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"59'259","reported_seq":3419,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.949304+0000","last_change":"2024-09-13T09:12:16.388972+0000","last_active":"2024-09-13T09:12:55.949304+0000","last_peered":"2024-09-13T09:12:55.949304+0000","last_clean":"2024-09-13T09:12:55.949304+0000","last_became_active":"2024-09-13T09:12:13.364622+0000","last_became_peered":"2024-09-13T09:12:13.364622+0000","last_unstale":"2024-09-13T09:12:55.949304+0000","last_undegraded":"2024-09-13T09:12:55.949304+0000","last_fullsized":"2024-09-13T09:12:55.949304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:34:20.301837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088725000000000002,"stat_sum":{"num_bytes":47583846,"num_objects":27,"num_object_clones":9,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":7,"num_read":4697,"num_read_kb":21964,"num_write":220,"num_write_kb":8745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"53'219","reported_seq":4746,"reported_epoch":58,"state":"remapped+peering","last_fresh":"2024-09-13T09:12:54.923611+0000","last_change":"2024-09-13T09:12:56.321713+0000","last_active":"2024-09-13T09:12:56.321713+0000","last_peered":"2024-09-13T09:12:54.923611+0000","last_clean":"2024-09-13T09:12:54.923611+0000","last_became_active":"2024-09-13T09:12:13.368138+0000","last_became_peered":"2024-09-13T09:12:13.368138+0000","last_unstale":"2024-09-13T09:12:54.923611+0000","last_undegraded":"2024-09-13T09:12:54.923611+0000","last_fullsized":"2024-09-13T09:12:54.923611+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:07:09.043503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082447100000000001,"stat_sum":{"num_bytes":35081288,"num_objects":29,"num_object_clones":10,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":4823,"num_read_kb":36304,"num_write":282,"num_write_kb":5612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3,4],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"54'199","reported_seq":5704,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.618290+0000","last_change":"2024-09-13T09:12:16.380731+0000","last_active":"2024-09-13T09:12:55.618290+0000","last_peered":"2024-09-13T09:12:55.618290+0000","last_clean":"2024-09-13T09:12:55.618290+0000","last_became_active":"2024-09-13T09:12:13.362533+0000","last_became_peered":"2024-09-13T09:12:13.362533+0000","last_unstale":"2024-09-13T09:12:55.618290+0000","last_undegraded":"2024-09-13T09:12:55.618290+0000","last_fullsized":"2024-09-13T09:12:55.618290+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:50.399315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072061100000000002,"stat_sum":{"num_bytes":23726627,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":6417,"num_read_kb":22037,"num_write":156,"num_write_kb":1179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"55'154","reported_seq":4832,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T09:12:54.870998+0000","last_change":"2024-09-13T09:12:16.376447+0000","last_active":"2024-09-13T09:12:54.870998+0000","last_peered":"2024-09-13T09:12:54.870998+0000","last_clean":"2024-09-13T09:12:54.870998+0000","last_became_active":"2024-09-13T09:12:13.368221+0000","last_became_peered":"2024-09-13T09:12:13.368221+0000","last_unstale":"2024-09-13T09:12:54.870998+0000","last_undegraded":"2024-09-13T09:12:54.870998+0000","last_fullsized":"2024-09-13T09:12:54.870998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:10.966835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000740483,"stat_sum":{"num_bytes":51515443,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":4850,"num_read_kb":9621,"num_write":176,"num_write_kb":1495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"59'147","reported_seq":1286,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.939738+0000","last_change":"2024-09-13T09:12:16.377490+0000","last_active":"2024-09-13T09:12:55.939738+0000","last_peered":"2024-09-13T09:12:55.939738+0000","last_clean":"2024-09-13T09:12:55.939738+0000","last_became_active":"2024-09-13T09:12:13.367773+0000","last_became_peered":"2024-09-13T09:12:13.367773+0000","last_unstale":"2024-09-13T09:12:55.939738+0000","last_undegraded":"2024-09-13T09:12:55.939738+0000","last_fullsized":"2024-09-13T09:12:55.939738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:34.023964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000479879,"stat_sum":{"num_bytes":6115328,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":2270,"num_read_kb":3413,"num_write":112,"num_write_kb":675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"53'204","reported_seq":3167,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.379346+0000","last_change":"2024-09-13T09:12:16.381209+0000","last_active":"2024-09-13T09:12:55.379346+0000","last_peered":"2024-09-13T09:12:55.379346+0000","last_clean":"2024-09-13T09:12:55.379346+0000","last_became_active":"2024-09-13T09:12:13.365820+0000","last_became_peered":"2024-09-13T09:12:13.365820+0000","last_unstale":"2024-09-13T09:12:55.379346+0000","last_undegraded":"2024-09-13T09:12:55.379346+0000","last_fullsized":"2024-09-13T09:12:55.379346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:48.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097766600000000008,"stat_sum":{"num_bytes":11883008,"num_objects":16,"num_object_clones":6,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":3814,"num_read_kb":21995,"num_write":211,"num_write_kb":5119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"55'317","reported_seq":2350,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T09:12:54.314401+0000","last_change":"2024-09-13T09:12:46.251509+0000","last_active":"2024-09-13T09:12:54.314401+0000","last_peered":"2024-09-13T09:12:54.314401+0000","last_clean":"2024-09-13T09:12:54.314401+0000","last_became_active":"2024-09-13T09:12:46.250870+0000","last_became_peered":"2024-09-13T09:12:46.250870+0000","last_unstale":"2024-09-13T09:12:54.314401+0000","last_undegraded":"2024-09-13T09:12:54.314401+0000","last_fullsized":"2024-09-13T09:12:54.314401+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:33.138405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040009000000000003,"stat_sum":{"num_bytes":22262387,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":2902,"num_read_kb":13374,"num_write":455,"num_write_kb":1653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":706121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"59'571","reported_seq":5816,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.937878+0000","last_change":"2024-09-13T09:12:46.255856+0000","last_active":"2024-09-13T09:12:55.937878+0000","last_peered":"2024-09-13T09:12:55.937878+0000","last_clean":"2024-09-13T09:12:55.937878+0000","last_became_active":"2024-09-13T09:12:46.250696+0000","last_became_peered":"2024-09-13T09:12:46.250696+0000","last_unstale":"2024-09-13T09:12:55.937878+0000","last_undegraded":"2024-09-13T09:12:55.937878+0000","last_fullsized":"2024-09-13T09:12:55.937878+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:43.821997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172600000000004,"stat_sum":{"num_bytes":19484709,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":7795,"num_read_kb":20827,"num_write":1001,"num_write_kb":2891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1136450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"59'259","reported_seq":3323,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.661210+0000","last_change":"2024-09-13T09:12:46.254606+0000","last_active":"2024-09-13T09:12:55.661210+0000","last_peered":"2024-09-13T09:12:55.661210+0000","last_clean":"2024-09-13T09:12:55.661210+0000","last_became_active":"2024-09-13T09:12:46.254016+0000","last_became_peered":"2024-09-13T09:12:46.254016+0000","last_unstale":"2024-09-13T09:12:55.661210+0000","last_undegraded":"2024-09-13T09:12:55.661210+0000","last_fullsized":"2024-09-13T09:12:55.661210+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:50.193567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074689299999999999,"stat_sum":{"num_bytes":45383291,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":8,"num_read":3846,"num_read_kb":17564,"num_write":214,"num_write_kb":2033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1110608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.364771+0000","last_change":"2024-09-13T09:12:43.531552+0000","last_active":"2024-09-13T09:12:55.364771+0000","last_peered":"2024-09-13T09:12:55.364771+0000","last_clean":"2024-09-13T09:12:55.364771+0000","last_became_active":"2024-09-13T09:12:43.531274+0000","last_became_peered":"2024-09-13T09:12:43.531274+0000","last_unstale":"2024-09-13T09:12:55.364771+0000","last_undegraded":"2024-09-13T09:12:55.364771+0000","last_fullsized":"2024-09-13T09:12:55.364771+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:41:06.992512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"59'414","reported_seq":4242,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.643417+0000","last_change":"2024-09-13T09:12:16.380593+0000","last_active":"2024-09-13T09:12:55.643417+0000","last_peered":"2024-09-13T09:12:55.643417+0000","last_clean":"2024-09-13T09:12:55.643417+0000","last_became_active":"2024-09-13T09:12:13.365496+0000","last_became_peered":"2024-09-13T09:12:13.365496+0000","last_unstale":"2024-09-13T09:12:55.643417+0000","last_undegraded":"2024-09-13T09:12:55.643417+0000","last_fullsized":"2024-09-13T09:12:55.643417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:29:49.115977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061775199999999995,"stat_sum":{"num_bytes":22274190,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":1,"num_read":5162,"num_read_kb":39670,"num_write":641,"num_write_kb":7171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":72,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T09:12:54.316574+0000","last_change":"2024-09-13T09:12:43.531266+0000","last_active":"2024-09-13T09:12:54.316574+0000","last_peered":"2024-09-13T09:12:54.316574+0000","last_clean":"2024-09-13T09:12:54.316574+0000","last_became_active":"2024-09-13T09:12:43.530941+0000","last_became_peered":"2024-09-13T09:12:43.530941+0000","last_unstale":"2024-09-13T09:12:54.316574+0000","last_undegraded":"2024-09-13T09:12:54.316574+0000","last_fullsized":"2024-09-13T09:12:54.316574+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:27.261467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"59'398","reported_seq":5490,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.858875+0000","last_change":"2024-09-13T09:12:16.378483+0000","last_active":"2024-09-13T09:12:55.858875+0000","last_peered":"2024-09-13T09:12:55.858875+0000","last_clean":"2024-09-13T09:12:55.858875+0000","last_became_active":"2024-09-13T09:12:13.367146+0000","last_became_peered":"2024-09-13T09:12:13.367146+0000","last_unstale":"2024-09-13T09:12:55.858875+0000","last_undegraded":"2024-09-13T09:12:55.858875+0000","last_fullsized":"2024-09-13T09:12:55.858875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:49.370653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032368300000000001,"stat_sum":{"num_bytes":14458880,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":8477,"num_read_kb":23664,"num_write":301,"num_write_kb":3250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.364591+0000","last_change":"2024-09-13T09:12:43.533829+0000","last_active":"2024-09-13T09:12:55.364591+0000","last_peered":"2024-09-13T09:12:55.364591+0000","last_clean":"2024-09-13T09:12:55.364591+0000","last_became_active":"2024-09-13T09:12:43.533557+0000","last_became_peered":"2024-09-13T09:12:43.533557+0000","last_unstale":"2024-09-13T09:12:55.364591+0000","last_undegraded":"2024-09-13T09:12:55.364591+0000","last_fullsized":"2024-09-13T09:12:55.364591+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:51.939164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"59'152","reported_seq":2627,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.924646+0000","last_change":"2024-09-13T09:12:16.376528+0000","last_active":"2024-09-13T09:12:55.924646+0000","last_peered":"2024-09-13T09:12:55.924646+0000","last_clean":"2024-09-13T09:12:55.924646+0000","last_became_active":"2024-09-13T09:12:13.367848+0000","last_became_peered":"2024-09-13T09:12:13.367848+0000","last_unstale":"2024-09-13T09:12:55.924646+0000","last_undegraded":"2024-09-13T09:12:55.924646+0000","last_fullsized":"2024-09-13T09:12:55.924646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:21.943266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067051299999999997,"stat_sum":{"num_bytes":50228736,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":3366,"num_read_kb":11981,"num_write":136,"num_write_kb":5266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":56,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T09:12:54.316618+0000","last_change":"2024-09-13T09:12:06.328091+0000","last_active":"2024-09-13T09:12:54.316618+0000","last_peered":"2024-09-13T09:12:54.316618+0000","last_clean":"2024-09-13T09:12:54.316618+0000","last_became_active":"2024-09-13T09:12:04.446508+0000","last_became_peered":"2024-09-13T09:12:04.446508+0000","last_unstale":"2024-09-13T09:12:54.316618+0000","last_undegraded":"2024-09-13T09:12:54.316618+0000","last_fullsized":"2024-09-13T09:12:54.316618+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:33.410873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042514000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"55'250","reported_seq":623,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T09:12:54.314491+0000","last_change":"2024-09-13T09:12:16.389006+0000","last_active":"2024-09-13T09:12:54.314491+0000","last_peered":"2024-09-13T09:12:54.314491+0000","last_clean":"2024-09-13T09:12:54.314491+0000","last_became_active":"2024-09-13T09:12:13.369489+0000","last_became_peered":"2024-09-13T09:12:13.369489+0000","last_unstale":"2024-09-13T09:12:54.314491+0000","last_undegraded":"2024-09-13T09:12:54.314491+0000","last_fullsized":"2024-09-13T09:12:54.314491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:33.879162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088595999999999998,"stat_sum":{"num_bytes":3100815,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":1219,"num_read_kb":2414,"num_write":369,"num_write_kb":1172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"58'193","reported_seq":6763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.845765+0000","last_change":"2024-09-13T09:12:16.380981+0000","last_active":"2024-09-13T09:12:55.845765+0000","last_peered":"2024-09-13T09:12:55.845765+0000","last_clean":"2024-09-13T09:12:55.845765+0000","last_became_active":"2024-09-13T09:12:13.367770+0000","last_became_peered":"2024-09-13T09:12:13.367770+0000","last_unstale":"2024-09-13T09:12:55.845765+0000","last_undegraded":"2024-09-13T09:12:55.845765+0000","last_fullsized":"2024-09-13T09:12:55.845765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:21:50.690425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083150800000000001,"stat_sum":{"num_bytes":45541156,"num_objects":34,"num_object_clones":13,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":6611,"num_read_kb":44424,"num_write":196,"num_write_kb":2679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":58,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T09:12:54.314293+0000","last_change":"2024-09-13T09:12:06.330649+0000","last_active":"2024-09-13T09:12:54.314293+0000","last_peered":"2024-09-13T09:12:54.314293+0000","last_clean":"2024-09-13T09:12:54.314293+0000","last_became_active":"2024-09-13T09:12:04.325470+0000","last_became_peered":"2024-09-13T09:12:04.325470+0000","last_unstale":"2024-09-13T09:12:54.314293+0000","last_undegraded":"2024-09-13T09:12:54.314293+0000","last_fullsized":"2024-09-13T09:12:54.314293+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:37.971615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093634899999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"55'144","reported_seq":1572,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T09:12:54.316381+0000","last_change":"2024-09-13T09:12:16.378750+0000","last_active":"2024-09-13T09:12:54.316381+0000","last_peered":"2024-09-13T09:12:54.316381+0000","last_clean":"2024-09-13T09:12:54.316381+0000","last_became_active":"2024-09-13T09:12:13.367656+0000","last_became_peered":"2024-09-13T09:12:13.367656+0000","last_unstale":"2024-09-13T09:12:54.316381+0000","last_undegraded":"2024-09-13T09:12:54.316381+0000","last_fullsized":"2024-09-13T09:12:54.316381+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:11.942294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003044836,"stat_sum":{"num_bytes":15526944,"num_objects":24,"num_object_clones":9,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":1424,"num_read_kb":14723,"num_write":136,"num_write_kb":1483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.312951+0000","last_change":"2024-09-13T09:12:06.845125+0000","last_active":"2024-09-13T09:12:55.312951+0000","last_peered":"2024-09-13T09:12:55.312951+0000","last_clean":"2024-09-13T09:12:55.312951+0000","last_became_active":"2024-09-13T09:12:04.321941+0000","last_became_peered":"2024-09-13T09:12:04.321941+0000","last_unstale":"2024-09-13T09:12:55.312951+0000","last_undegraded":"2024-09-13T09:12:55.312951+0000","last_fullsized":"2024-09-13T09:12:55.312951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:57.706060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031569600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"59'100","reported_seq":1240,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.662673+0000","last_change":"2024-09-13T09:12:16.380222+0000","last_active":"2024-09-13T09:12:55.662673+0000","last_peered":"2024-09-13T09:12:55.662673+0000","last_clean":"2024-09-13T09:12:55.662673+0000","last_became_active":"2024-09-13T09:12:13.367633+0000","last_became_peered":"2024-09-13T09:12:13.367633+0000","last_unstale":"2024-09-13T09:12:55.662673+0000","last_undegraded":"2024-09-13T09:12:55.662673+0000","last_fullsized":"2024-09-13T09:12:55.662673+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:34.734632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438009,"stat_sum":{"num_bytes":6182912,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":1102,"num_read_kb":8795,"num_write":99,"num_write_kb":972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.364513+0000","last_change":"2024-09-13T09:12:43.532201+0000","last_active":"2024-09-13T09:12:55.364513+0000","last_peered":"2024-09-13T09:12:55.364513+0000","last_clean":"2024-09-13T09:12:55.364513+0000","last_became_active":"2024-09-13T09:12:43.531942+0000","last_became_peered":"2024-09-13T09:12:43.531942+0000","last_unstale":"2024-09-13T09:12:55.364513+0000","last_undegraded":"2024-09-13T09:12:55.364513+0000","last_fullsized":"2024-09-13T09:12:55.364513+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:26.408883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"59'237","reported_seq":1023,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.850736+0000","last_change":"2024-09-13T09:12:16.377544+0000","last_active":"2024-09-13T09:12:55.850736+0000","last_peered":"2024-09-13T09:12:55.850736+0000","last_clean":"2024-09-13T09:12:55.850736+0000","last_became_active":"2024-09-13T09:12:13.361909+0000","last_became_peered":"2024-09-13T09:12:13.361909+0000","last_unstale":"2024-09-13T09:12:55.850736+0000","last_undegraded":"2024-09-13T09:12:55.850736+0000","last_fullsized":"2024-09-13T09:12:55.850736+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:34.570438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001546782,"stat_sum":{"num_bytes":9660043,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":1550,"num_read_kb":3782,"num_write":347,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":57,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T09:12:54.314348+0000","last_change":"2024-09-13T09:12:06.330140+0000","last_active":"2024-09-13T09:12:54.314348+0000","last_peered":"2024-09-13T09:12:54.314348+0000","last_clean":"2024-09-13T09:12:54.314348+0000","last_became_active":"2024-09-13T09:12:04.324857+0000","last_became_peered":"2024-09-13T09:12:04.324857+0000","last_unstale":"2024-09-13T09:12:54.314348+0000","last_undegraded":"2024-09-13T09:12:54.314348+0000","last_fullsized":"2024-09-13T09:12:54.314348+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:57.400286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038911000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":226,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.364450+0000","last_change":"2024-09-13T09:12:46.248714+0000","last_active":"2024-09-13T09:12:55.364450+0000","last_peered":"2024-09-13T09:12:55.364450+0000","last_clean":"2024-09-13T09:12:55.364450+0000","last_became_active":"2024-09-13T09:12:46.248451+0000","last_became_peered":"2024-09-13T09:12:46.248451+0000","last_unstale":"2024-09-13T09:12:55.364450+0000","last_undegraded":"2024-09-13T09:12:55.364450+0000","last_fullsized":"2024-09-13T09:12:55.364450+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:27.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"58'122","reported_seq":1605,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.850314+0000","last_change":"2024-09-13T09:12:16.377642+0000","last_active":"2024-09-13T09:12:55.850314+0000","last_peered":"2024-09-13T09:12:55.850314+0000","last_clean":"2024-09-13T09:12:55.850314+0000","last_became_active":"2024-09-13T09:12:13.369742+0000","last_became_peered":"2024-09-13T09:12:13.369742+0000","last_unstale":"2024-09-13T09:12:55.850314+0000","last_undegraded":"2024-09-13T09:12:55.850314+0000","last_fullsized":"2024-09-13T09:12:55.850314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:27:48.361252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060355399999999996,"stat_sum":{"num_bytes":26185278,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":1580,"num_read_kb":6441,"num_write":141,"num_write_kb":10002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"59'332","reported_seq":6466,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.661276+0000","last_change":"2024-09-13T09:12:16.380989+0000","last_active":"2024-09-13T09:12:55.661276+0000","last_peered":"2024-09-13T09:12:55.661276+0000","last_clean":"2024-09-13T09:12:55.661276+0000","last_became_active":"2024-09-13T09:12:13.370096+0000","last_became_peered":"2024-09-13T09:12:13.370096+0000","last_unstale":"2024-09-13T09:12:55.661276+0000","last_undegraded":"2024-09-13T09:12:55.661276+0000","last_fullsized":"2024-09-13T09:12:55.661276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:14.224546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000903458,"stat_sum":{"num_bytes":44025344,"num_objects":23,"num_object_clones":9,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":6,"num_read":8834,"num_read_kb":19706,"num_write":226,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"59'140","reported_seq":1781,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.937849+0000","last_change":"2024-09-13T09:12:46.253134+0000","last_active":"2024-09-13T09:12:55.937849+0000","last_peered":"2024-09-13T09:12:55.937849+0000","last_clean":"2024-09-13T09:12:55.937849+0000","last_became_active":"2024-09-13T09:12:46.252578+0000","last_became_peered":"2024-09-13T09:12:46.252578+0000","last_unstale":"2024-09-13T09:12:55.937849+0000","last_undegraded":"2024-09-13T09:12:55.937849+0000","last_fullsized":"2024-09-13T09:12:55.937849+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:03.931492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041459800000000002,"stat_sum":{"num_bytes":54887492,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":4,"num_read":1606,"num_read_kb":18290,"num_write":154,"num_write_kb":3796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2590736,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"58'118","reported_seq":3583,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.838024+0000","last_change":"2024-09-13T09:12:16.377864+0000","last_active":"2024-09-13T09:12:55.838024+0000","last_peered":"2024-09-13T09:12:55.838024+0000","last_clean":"2024-09-13T09:12:55.838024+0000","last_became_active":"2024-09-13T09:12:13.362409+0000","last_became_peered":"2024-09-13T09:12:13.362409+0000","last_unstale":"2024-09-13T09:12:55.838024+0000","last_undegraded":"2024-09-13T09:12:55.838024+0000","last_fullsized":"2024-09-13T09:12:55.838024+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:37.023594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001713904,"stat_sum":{"num_bytes":65329696,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":3466,"num_read_kb":46757,"num_write":153,"num_write_kb":10405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"58'441","reported_seq":7257,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.961133+0000","last_change":"2024-09-13T09:12:16.389470+0000","last_active":"2024-09-13T09:12:55.961133+0000","last_peered":"2024-09-13T09:12:55.961133+0000","last_clean":"2024-09-13T09:12:55.961133+0000","last_became_active":"2024-09-13T09:12:13.365758+0000","last_became_peered":"2024-09-13T09:12:13.365758+0000","last_unstale":"2024-09-13T09:12:55.961133+0000","last_undegraded":"2024-09-13T09:12:55.961133+0000","last_fullsized":"2024-09-13T09:12:55.961133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:46.649879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013123500000000001,"stat_sum":{"num_bytes":93873226,"num_objects":31,"num_object_clones":13,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":8290,"num_read_kb":58229,"num_write":686,"num_write_kb":24976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"59'117","reported_seq":351,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.644460+0000","last_change":"2024-09-13T09:12:43.630169+0000","last_active":"2024-09-13T09:12:55.644460+0000","last_peered":"2024-09-13T09:12:55.644460+0000","last_clean":"2024-09-13T09:12:55.644460+0000","last_became_active":"2024-09-13T09:12:43.533321+0000","last_became_peered":"2024-09-13T09:12:43.533321+0000","last_unstale":"2024-09-13T09:12:55.644460+0000","last_undegraded":"2024-09-13T09:12:55.644460+0000","last_fullsized":"2024-09-13T09:12:55.644460+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:43.973948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000679308,"stat_sum":{"num_bytes":8004651,"num_objects":15,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":366,"num_read_kb":2031,"num_write":121,"num_write_kb":4873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"55'110","reported_seq":1830,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.037731+0000","last_change":"2024-09-13T09:12:16.389243+0000","last_active":"2024-09-13T09:12:55.037731+0000","last_peered":"2024-09-13T09:12:55.037731+0000","last_clean":"2024-09-13T09:12:55.037731+0000","last_became_active":"2024-09-13T09:12:13.362547+0000","last_became_peered":"2024-09-13T09:12:13.362547+0000","last_unstale":"2024-09-13T09:12:55.037731+0000","last_undegraded":"2024-09-13T09:12:55.037731+0000","last_fullsized":"2024-09-13T09:12:55.037731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:24:24.557400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011202860000000001,"stat_sum":{"num_bytes":2537984,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":2265,"num_read_kb":3542,"num_write":90,"num_write_kb":612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'154","reported_seq":874,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.947284+0000","last_change":"2024-09-13T09:12:43.623612+0000","last_active":"2024-09-13T09:12:55.947284+0000","last_peered":"2024-09-13T09:12:55.947284+0000","last_clean":"2024-09-13T09:12:55.947284+0000","last_became_active":"2024-09-13T09:12:43.533088+0000","last_became_peered":"2024-09-13T09:12:43.533088+0000","last_unstale":"2024-09-13T09:12:55.947284+0000","last_undegraded":"2024-09-13T09:12:55.947284+0000","last_fullsized":"2024-09-13T09:12:55.947284+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:43:49.678463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059255799999999997,"stat_sum":{"num_bytes":2678960,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":877,"num_read_kb":1438,"num_write":190,"num_write_kb":1429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":635004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"59'109","reported_seq":1903,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.918008+0000","last_change":"2024-09-13T09:12:16.389277+0000","last_active":"2024-09-13T09:12:55.918008+0000","last_peered":"2024-09-13T09:12:55.918008+0000","last_clean":"2024-09-13T09:12:55.918008+0000","last_became_active":"2024-09-13T09:12:13.367757+0000","last_became_peered":"2024-09-13T09:12:13.367757+0000","last_unstale":"2024-09-13T09:12:55.918008+0000","last_undegraded":"2024-09-13T09:12:55.918008+0000","last_fullsized":"2024-09-13T09:12:55.918008+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:33.596156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011263219999999999,"stat_sum":{"num_bytes":20296796,"num_objects":15,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":1798,"num_read_kb":8711,"num_write":106,"num_write_kb":11969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"59'126","reported_seq":543,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.877331+0000","last_change":"2024-09-13T09:12:43.614382+0000","last_active":"2024-09-13T09:12:55.877331+0000","last_peered":"2024-09-13T09:12:55.877331+0000","last_clean":"2024-09-13T09:12:55.877331+0000","last_became_active":"2024-09-13T09:12:43.532504+0000","last_became_peered":"2024-09-13T09:12:43.532504+0000","last_unstale":"2024-09-13T09:12:55.877331+0000","last_undegraded":"2024-09-13T09:12:55.877331+0000","last_fullsized":"2024-09-13T09:12:55.877331+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:25:04.225563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034391499999999998,"stat_sum":{"num_bytes":14411815,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":1032,"num_read_kb":12149,"num_write":130,"num_write_kb":1596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":694311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"59'247","reported_seq":8497,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.664142+0000","last_change":"2024-09-13T09:12:46.248982+0000","last_active":"2024-09-13T09:12:55.664142+0000","last_peered":"2024-09-13T09:12:55.664142+0000","last_clean":"2024-09-13T09:12:55.664142+0000","last_became_active":"2024-09-13T09:12:46.248261+0000","last_became_peered":"2024-09-13T09:12:46.248261+0000","last_unstale":"2024-09-13T09:12:55.664142+0000","last_undegraded":"2024-09-13T09:12:55.664142+0000","last_fullsized":"2024-09-13T09:12:55.664142+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:44.056762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049261300000000004,"stat_sum":{"num_bytes":18003968,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":9014,"num_read_kb":21223,"num_write":210,"num_write_kb":1725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":676864,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"58'345","reported_seq":4143,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T09:12:55.364516+0000","last_change":"2024-09-13T09:12:46.251913+0000","last_active":"2024-09-13T09:12:55.364516+0000","last_peered":"2024-09-13T09:12:55.364516+0000","last_clean":"2024-09-13T09:12:55.364516+0000","last_became_active":"2024-09-13T09:12:46.250552+0000","last_became_peered":"2024-09-13T09:12:46.250552+0000","last_unstale":"2024-09-13T09:12:55.364516+0000","last_undegraded":"2024-09-13T09:12:55.364516+0000","last_fullsized":"2024-09-13T09:12:55.364516+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:47.142721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059370899999999999,"stat_sum":{"num_bytes":4122729,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":4722,"num_read_kb":5221,"num_write":317,"num_write_kb":945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":28777,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":830520945,"num_objects":610,"num_object_clones":195,"num_object_copies":1220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":610,"num_whiteouts":178,"num_read":122834,"num_read_kb":538195,"num_write":8879,"num_write_kb":138560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":8739591,"num_keys_recovered":33,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163237888,"data_stored":167395554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7544,"ondisk_log_size":7544,"up":64,"acting":63,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640271,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73012,"kb_used_data":33708,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94298828,"statfs":{"total":96636764160,"available":96561999872,"internally_reserved":0,"allocated":34516992,"data_stored":34380163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]}]},{"osd":4,"up_from":11,"seq":47244640272,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59132,"kb_used_data":19820,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94312708,"statfs":{"total":96636764160,"available":96576212992,"internally_reserved":0,"allocated":20295680,"data_stored":21264076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23599999999999999}]}]},{"osd":3,"up_from":11,"seq":47244640273,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51956,"kb_used_data":16744,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94319884,"statfs":{"total":96636764160,"available":96583561216,"internally_reserved":0,"allocated":17145856,"data_stored":17589673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640271,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60580,"kb_used_data":25364,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94311260,"statfs":{"total":96636764160,"available":96574730240,"internally_reserved":0,"allocated":25972736,"data_stored":25824843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]}]},{"osd":1,"up_from":11,"seq":47244640272,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63720,"kb_used_data":24408,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94308120,"statfs":{"total":96636764160,"available":96571514880,"internally_reserved":0,"allocated":24993792,"data_stored":25313059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40237509},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]}]},{"osd":0,"up_from":11,"seq":47244640271,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83824,"kb_used_data":44496,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94288016,"statfs":{"total":96636764160,"available":96550928384,"internally_reserved":0,"allocated":45563904,"data_stored":45810620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":44285952,"data_stored":44949237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24317952,"data_stored":25047021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":25300992,"data_stored":25558824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15867904,"data_stored":16728290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":19623936,"data_stored":20998057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":33841152,"data_stored":34114125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:13:01.065 INFO:tasks.thrashosds.thrasher:Setting 2.7 to [5, 4] 2024-09-13T09:13:01.065 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.7', '5', '4'] 2024-09-13T09:13:01.065 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.7 5 4 2024-09-13T09:13:01.114 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-13T09:13:01.158 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:01.157+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:01.190 INFO:teuthology.orchestra.run.smithi157.stdout:2896 write 0x7e99e00 thru 0x7ea9c19 (0xfe1a bytes) 2024-09-13T09:13:01.194 INFO:teuthology.orchestra.run.smithi157.stdout:2897 write 0x146ce00 thru 0x14718f4 (0x4af5 bytes) 2024-09-13T09:13:01.198 INFO:teuthology.orchestra.run.smithi157.stdout:2898 trunc from 0x7ea9e00 to 0xb045400 2024-09-13T09:13:01.228 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:01.228 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:01.228 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:01.228 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:01.239 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:01.245 INFO:teuthology.orchestra.run.smithi157.stdout:2899 write 0x63f4800 thru 0x63fd52a (0x8d2b bytes) 2024-09-13T09:13:01.246 INFO:teuthology.orchestra.run.smithi157.stdout:2901 read 0x8f40e00 thru 0x8f47fb9 (0x71ba bytes) 2024-09-13T09:13:01.258 INFO:teuthology.orchestra.run.smithi157.stdout:2902 trunc from 0xb045400 to 0x123dc00 2024-09-13T09:13:01.259 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:01.258+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:01.308 INFO:teuthology.orchestra.run.smithi157.stdout:2904 read 0x114f600 thru 0x115ee1a (0xf81b bytes) 2024-09-13T09:13:01.328 INFO:teuthology.orchestra.run.smithi157.stdout:2907 write 0x5bb5200 thru 0x5bba53b (0x533c bytes) 2024-09-13T09:13:01.329 INFO:teuthology.orchestra.run.smithi157.stdout:2908 read 0x7ae200 thru 0x7b7c40 (0x9a41 bytes) 2024-09-13T09:13:01.346 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:13:01.365 INFO:teuthology.orchestra.run.smithi157.stdout:2911 write 0x962d400 thru 0x962dad1 (0x6d2 bytes) 2024-09-13T09:13:01.365 INFO:teuthology.orchestra.run.smithi157.stdout:2912 write 0x303c400 thru 0x3048e77 (0xca78 bytes) 2024-09-13T09:13:01.369 INFO:teuthology.orchestra.run.smithi157.stdout:2913 trunc from 0x962dc00 to 0x58b2c00 2024-09-13T09:13:01.391 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:01.390+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:01.401 INFO:teuthology.orchestra.run.smithi157.stdout:2914 punch from 0x1586400 to 0x1589800, (0x3400 bytes) 2024-09-13T09:13:01.402 INFO:teuthology.orchestra.run.smithi157.stdout:2917 read 0x257d200 thru 0x257e78e (0x158f bytes) 2024-09-13T09:13:01.402 INFO:teuthology.orchestra.run.smithi157.stdout:2919 punch from 0x46bba00 to 0x46bbe00, (0x400 bytes) 2024-09-13T09:13:01.402 INFO:teuthology.orchestra.run.smithi157.stdout:2920 read 0x2f96e00 thru 0x2fa4527 (0xd728 bytes) 2024-09-13T09:13:01.402 INFO:teuthology.orchestra.run.smithi157.stdout:2922 write 0x4073600 thru 0x407d0b1 (0x9ab2 bytes) 2024-09-13T09:13:01.402 INFO:teuthology.orchestra.run.smithi157.stdout:skipping zero length punch hole 2024-09-13T09:13:01.428 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:01.428 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:01.428 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:01.428 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:01.429 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:01.434 INFO:teuthology.orchestra.run.smithi157.stdout:2926 write 0x8c2b200 thru 0x8c336c8 (0x84c9 bytes) 2024-09-13T09:13:01.435 INFO:teuthology.orchestra.run.smithi157.stdout:2929 write 0x630c200 thru 0x630d058 (0xe59 bytes) 2024-09-13T09:13:01.438 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:01.442 INFO:teuthology.orchestra.run.smithi157.stdout:2930 trunc from 0x8c33800 to 0xe4b6c00 2024-09-13T09:13:01.464 INFO:teuthology.orchestra.run.smithi157.stdout:2932 write 0xdc44e00 thru 0xdc465dd (0x17de bytes) 2024-09-13T09:13:01.464 INFO:teuthology.orchestra.run.smithi157.stdout:2934 punch from 0x5bdd000 to 0x5be7a00, (0xaa00 bytes) 2024-09-13T09:13:01.465 INFO:teuthology.orchestra.run.smithi157.stdout:2935 read 0x2b1e200 thru 0x2b26ecd (0x8cce bytes) 2024-09-13T09:13:01.465 INFO:teuthology.orchestra.run.smithi157.stdout:2937 punch from 0x88b1000 to 0x88bd400, (0xc400 bytes) 2024-09-13T09:13:01.466 INFO:teuthology.orchestra.run.smithi157.stdout:2939 write 0x3a43800 thru 0x3a4af37 (0x7738 bytes) 2024-09-13T09:13:01.466 INFO:teuthology.orchestra.run.smithi157.stdout:2942 write 0xafa8e00 thru 0xafb7221 (0xe422 bytes) 2024-09-13T09:13:01.466 INFO:teuthology.orchestra.run.smithi157.stdout:2945 read 0x292c600 thru 0x292fe0d (0x380e bytes) 2024-09-13T09:13:01.466 INFO:teuthology.orchestra.run.smithi157.stdout:2948 write 0x5843a00 thru 0x58467cc (0x2dcd bytes) 2024-09-13T09:13:01.466 INFO:teuthology.orchestra.run.smithi157.stdout:2954 punch from 0x8eeb000 to 0x8eef400, (0x4400 bytes) 2024-09-13T09:13:01.474 INFO:teuthology.orchestra.run.smithi157.stdout:2957 trunc from 0xe4b6c00 to 0x72cec00 2024-09-13T09:13:01.482 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:01.478+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:01.541 INFO:teuthology.orchestra.run.smithi157.stdout:2958 read 0x5556800 thru 0x5559afc (0x32fd bytes) 2024-09-13T09:13:01.548 INFO:teuthology.orchestra.run.smithi157.stdout:2959 trunc from 0x72cec00 to 0x1372600 2024-09-13T09:13:01.562 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:01.560+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:01.616 INFO:teuthology.orchestra.run.smithi157.stdout:2960 read 0xa16800 thru 0xa219e9 (0xb1ea bytes) 2024-09-13T09:13:01.620 INFO:teuthology.orchestra.run.smithi157.stdout:2961 read 0x143e00 thru 0x144ec9 (0x10ca bytes) 2024-09-13T09:13:01.621 INFO:teuthology.orchestra.run.smithi157.stdout:2962 read 0xf5de00 thru 0xf611e5 (0x33e6 bytes) 2024-09-13T09:13:01.621 INFO:teuthology.orchestra.run.smithi157.stdout:2963 read 0xd8bc00 thru 0xd8e757 (0x2b58 bytes) 2024-09-13T09:13:01.628 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122710476000002, 2024-09-13T09:13:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:13:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:13:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:13:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:13:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:13:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:13:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:13:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:13:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:13:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:13:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:13:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:13:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:13:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:13:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:13:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122682791999999, 2024-09-13T09:13:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:13:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:13:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:13:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:13:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:13:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:13:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:13:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:13:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:13:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:13:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:13:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:13:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:13:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122658451, 2024-09-13T09:13:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:13:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:13:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:13:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:13:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:13:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:13:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:13:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:13:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:13:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:13:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:13:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:13:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:13:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:13:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:13:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122642130999999, 2024-09-13T09:13:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:13:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:13:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:01.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:13:01.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:01.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:13:01.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:13:01.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:13:01.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:13:01.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:13:01.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:13:01.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:13:01.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:13:01.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:13:01.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:13:01.646 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.646 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.646 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.646 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.646 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:01.646 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122625716000002, 2024-09-13T09:13:01.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:13:01.647 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.647 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.647 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.647 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.647 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.647 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:13:01.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.647 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:01.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:01.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:01.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:13:01.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:13:01.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:13:01.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:13:01.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:13:01.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:13:01.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:13:01.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:13:01.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:13:01.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:13:01.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:13:01.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:13:01.650 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.650 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.650 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.651 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.651 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:01.651 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122609608000001, 2024-09-13T09:13:01.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:13:01.651 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.651 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.651 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.651 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.651 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.651 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:13:01.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.651 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:01.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:01.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:01.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:13:01.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:13:01.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:13:01.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:13:01.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:13:01.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:13:01.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:13:01.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:13:01.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:13:01.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:13:01.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:13:01.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:13:01.655 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.655 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.655 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.655 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.655 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:01.655 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122594047, 2024-09-13T09:13:01.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:13:01.655 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.655 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.655 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.655 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.656 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.656 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:13:01.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.656 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:01.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:01.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:01.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:13:01.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:13:01.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:13:01.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:13:01.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:13:01.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:13:01.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:13:01.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:13:01.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:13:01.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:13:01.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:13:01.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:13:01.659 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.659 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.659 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.659 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.659 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:01.660 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122577208999999, 2024-09-13T09:13:01.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:13:01.660 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.660 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.660 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.660 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.660 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.660 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:13:01.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.660 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:01.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:01.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:13:01.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:13:01.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:13:01.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:13:01.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:13:01.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:13:01.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:13:01.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:13:01.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:13:01.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:13:01.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:13:01.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:13:01.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:13:01.664 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.664 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.664 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.664 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.664 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:01.664 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122561217000001, 2024-09-13T09:13:01.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:13:01.664 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.664 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.664 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.665 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.665 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.665 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:13:01.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.665 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:01.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:01.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.666 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.666 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:13:01.666 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.666 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.666 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:13:01.666 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:13:01.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.666 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.666 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:13:01.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:13:01.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:13:01.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:13:01.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:13:01.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:13:01.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:13:01.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:13:01.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:13:01.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:13:01.668 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.668 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.668 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.668 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.668 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:01.669 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122533719, 2024-09-13T09:13:01.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:13:01.669 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.669 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.669 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.669 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.669 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.669 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:13:01.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.669 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:01.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:01.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:01.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:13:01.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:13:01.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:13:01.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:13:01.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:13:01.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:13:01.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:13:01.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:13:01.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:13:01.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:13:01.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:13:01.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:13:01.672 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.673 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.673 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.673 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.673 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:01.673 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122507859999999, 2024-09-13T09:13:01.673 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:13:01.673 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.673 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.673 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.673 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.673 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.673 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:13:01.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.674 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:01.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:01.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:01.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:13:01.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:13:01.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:13:01.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:13:01.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:13:01.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:13:01.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:13:01.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:13:01.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:13:01.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:13:01.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:13:01.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:13:01.677 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.677 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.677 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.677 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.677 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:01.677 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122466370000001, 2024-09-13T09:13:01.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:13:01.678 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.678 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.678 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.678 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.678 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.678 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:13:01.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.678 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:01.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.679 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.679 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:01.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.679 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.679 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:13:01.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.679 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.679 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:13:01.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:13:01.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:13:01.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:13:01.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:13:01.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:13:01.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:13:01.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:13:01.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:13:01.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:13:01.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:13:01.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:13:01.681 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.681 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.681 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.682 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.682 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:01.682 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122445131999999, 2024-09-13T09:13:01.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:13:01.682 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.682 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.682 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.682 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:13:01.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.682 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:01.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:01.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:01.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:13:01.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:13:01.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:13:01.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:13:01.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:13:01.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:13:01.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:13:01.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:13:01.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:13:01.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:13:01.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.686 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.686 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:13:01.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:13:01.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.686 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.686 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.686 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:01.686 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122422331999999, 2024-09-13T09:13:01.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:13:01.686 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:13:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:01.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:13:01.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:13:01.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:13:01.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:13:01.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:01.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:13:01.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:13:01.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:13:01.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:13:01.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:13:01.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:13:01.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:13:01.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:13:01.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.690 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.691 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.691 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:01.691 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122405875999998, 2024-09-13T09:13:01.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:13:01.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.691 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.691 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:13:01.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.691 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:01.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:01.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:01.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:13:01.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:13:01.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:13:01.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:13:01.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:13:01.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:13:01.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:13:01.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:13:01.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:13:01.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:13:01.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:13:01.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:13:01.695 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.695 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.695 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.695 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.695 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:01.695 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.122160739000002, 2024-09-13T09:13:01.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:13:01.695 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.695 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.695 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.695 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.695 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.696 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:13:01.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.696 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:01.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:01.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:13:01.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:13:01.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:13:01.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:13:01.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:13:01.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:13:01.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:13:01.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:01.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:13:01.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:13:01.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:13:01.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:13:01.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:13:01.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:13:01.699 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.699 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.699 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.699 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20408 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5582848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.699 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.505992+0000", 2024-09-13T09:13:01.700 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.120296158999999, 2024-09-13T09:13:01.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050463591000000002, 2024-09-13T09:13:01.700 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.700 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.700 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.700 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.700 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.700 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20408 2024-09-13T09:13:01.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.700 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505992+0000", 2024-09-13T09:13:01.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505992+0000", 2024-09-13T09:13:01.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505991+0000", 2024-09-13T09:13:01.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505994+0000", 2024-09-13T09:13:01.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6259999999999999e-06 2024-09-13T09:13:01.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506002+0000", 2024-09-13T09:13:01.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1550000000000007e-06 2024-09-13T09:13:01.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506008+0000", 2024-09-13T09:13:01.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.553e-06 2024-09-13T09:13:01.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556375+0000", 2024-09-13T09:13:01.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050366850999999997 2024-09-13T09:13:01.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556412+0000", 2024-09-13T09:13:01.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7138999999999998e-05 2024-09-13T09:13:01.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556455+0000", 2024-09-13T09:13:01.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3733999999999998e-05 2024-09-13T09:13:01.704 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.704 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.704 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.704 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.704 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:01.704 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.12027518, 2024-09-13T09:13:01.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:13:01.704 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.704 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.704 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.704 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.704 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.704 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:13:01.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.705 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:01.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:01.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:13:01.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:13:01.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:13:01.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:13:01.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:13:01.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:13:01.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:13:01.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:13:01.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:13:01.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:13:01.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:13:01.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:13:01.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:13:01.708 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.708 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.708 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.708 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.708 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:01.708 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.120253814000002, 2024-09-13T09:13:01.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:13:01.709 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.709 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.709 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.709 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.709 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.709 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:13:01.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.709 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:01.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:01.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:01.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:01.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:13:01.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:13:01.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:13:01.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:13:01.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:13:01.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:13:01.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:13:01.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:13:01.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:13:01.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:13:01.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:13:01.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:13:01.712 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.712 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.713 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.713 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:01.713 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:01.713 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.120048445999998, 2024-09-13T09:13:01.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:13:01.713 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:01.713 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:01.713 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:01.713 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:01.713 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:01.713 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:13:01.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.714 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:01.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:01.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:01.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:01.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:01.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:01.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:01.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:13:01.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:13:01.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:01.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:13:01.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:13:01.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:01.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:13:01.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:13:01.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:01.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:13:01.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:13:01.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:01.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:13:01.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:13:01.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:01.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:13:01.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:13:01.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:01.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:01.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:01.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:13:01.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:13:01.717 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.717 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.717 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.717 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:01.717 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:01.717 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:01.719 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:13:01.720 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:01.717+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:01.720 INFO:teuthology.orchestra.run.smithi157.stdout:2965 trunc from 0x1372600 to 0xa050800 2024-09-13T09:13:01.720 INFO:teuthology.orchestra.run.smithi157.stdout:2966 trunc from 0xa050800 to 0xd707800 2024-09-13T09:13:01.720 INFO:teuthology.orchestra.run.smithi157.stdout:2969 read 0x4b5400 thru 0x4b7326 (0x1f27 bytes) 2024-09-13T09:13:01.720 INFO:teuthology.orchestra.run.smithi157.stdout:2971 punch from 0xbe4f200 to 0xbe5ee00, (0xfc00 bytes) 2024-09-13T09:13:01.720 INFO:teuthology.orchestra.run.smithi157.stdout:2974 read 0xcb07400 thru 0xcb08563 (0x1164 bytes) 2024-09-13T09:13:01.720 INFO:teuthology.orchestra.run.smithi157.stdout:2978 punch from 0x2abc600 to 0x2ac4c00, (0x8600 bytes) 2024-09-13T09:13:01.721 INFO:teuthology.orchestra.run.smithi157.stdout:2980 punch from 0x7892800 to 0x789c400, (0x9c00 bytes) 2024-09-13T09:13:01.721 INFO:teuthology.orchestra.run.smithi157.stdout:2981 punch from 0xaa59c00 to 0xaa5e400, (0x4800 bytes) 2024-09-13T09:13:01.721 INFO:teuthology.orchestra.run.smithi157.stdout:2983 trunc from 0xd707800 to 0xdf7c800 2024-09-13T09:13:01.721 INFO:teuthology.orchestra.run.smithi157.stdout:2986 write 0x5a32e00 thru 0x5a3d28c (0xa48d bytes) 2024-09-13T09:13:01.721 INFO:teuthology.orchestra.run.smithi157.stdout:2987 trunc from 0xdf7c800 to 0xdd53c00 2024-09-13T09:13:01.749 INFO:teuthology.orchestra.run.smithi157.stdout:2991 trunc from 0xdd53c00 to 0x2498c00 2024-09-13T09:13:01.784 INFO:teuthology.orchestra.run.smithi157.stdout:2992 read 0x2147e00 thru 0x214fc2d (0x7e2e bytes) 2024-09-13T09:13:01.797 INFO:teuthology.orchestra.run.smithi157.stdout:2994 trunc from 0x2498c00 to 0xd288200 2024-09-13T09:13:01.807 INFO:teuthology.orchestra.run.smithi157.stdout:2995 write 0xc6dea00 thru 0xc6e5d0a (0x730b bytes) 2024-09-13T09:13:01.811 INFO:teuthology.orchestra.run.smithi157.stdout:2996 trunc from 0xd288200 to 0xa7af400 2024-09-13T09:13:01.819 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:01.818+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:01.881 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:01.881 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:01.881 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:01.881 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:01.890 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:13:01.917 INFO:teuthology.orchestra.run.smithi157.stdout:3000 write 0x73c7c00 thru 0x73cbfa9 (0x43aa bytes) 2024-09-13T09:13:01.921 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:01.919+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:01.940 INFO:teuthology.orchestra.run.smithi157.stdout:3002 write 0xbda9400 thru 0xbdaaabd (0x16be bytes) 2024-09-13T09:13:01.946 INFO:teuthology.orchestra.run.smithi157.stdout:3005 punch from 0x23e6400 to 0x23e6c00, (0x800 bytes) 2024-09-13T09:13:01.946 INFO:teuthology.orchestra.run.smithi157.stdout:3008 write 0xb5b9000 thru 0xb5bf7d9 (0x67da bytes) 2024-09-13T09:13:01.946 INFO:teuthology.orchestra.run.smithi157.stdout:3011 read 0x32ad600 thru 0x32b5656 (0x8057 bytes) 2024-09-13T09:13:01.946 INFO:teuthology.orchestra.run.smithi157.stdout:3012 write 0x248f600 thru 0x2499531 (0x9f32 bytes) 2024-09-13T09:13:01.946 INFO:teuthology.orchestra.run.smithi157.stdout:3014 read 0xab8a600 thru 0xab8bd07 (0x1708 bytes) 2024-09-13T09:13:01.946 INFO:teuthology.orchestra.run.smithi157.stdout:3015 read 0x59e8a00 thru 0x59f434f (0xb950 bytes) 2024-09-13T09:13:01.946 INFO:teuthology.orchestra.run.smithi157.stdout:3017 write 0x41e7e00 thru 0x41f1ab6 (0x9cb7 bytes) 2024-09-13T09:13:01.948 INFO:teuthology.orchestra.run.smithi157.stdout:3022 trunc from 0xbdaac00 to 0xa1e5200 2024-09-13T09:13:01.991 INFO:teuthology.orchestra.run.smithi157.stdout:3023 punch from 0x8e9d000 to 0x8ea5e00, (0x8e00 bytes) 2024-09-13T09:13:01.997 INFO:teuthology.orchestra.run.smithi157.stdout:3024 read 0x4b2e600 thru 0x4b39032 (0xaa33 bytes) 2024-09-13T09:13:01.997 INFO:teuthology.orchestra.run.smithi157.stdout:3026 write 0x8cd2a00 thru 0x8cd7864 (0x4e65 bytes) 2024-09-13T09:13:01.997 INFO:teuthology.orchestra.run.smithi157.stdout:3027 read 0x7ee6400 thru 0x7ef4d32 (0xe933 bytes) 2024-09-13T09:13:01.997 INFO:teuthology.orchestra.run.smithi157.stdout:3028 read 0x51e800 thru 0x52dd9f (0xf5a0 bytes) 2024-09-13T09:13:02.010 INFO:teuthology.orchestra.run.smithi157.stdout:3031 trunc from 0xa1e5200 to 0x3c76800 2024-09-13T09:13:02.021 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:02.020+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:02.042 INFO:teuthology.orchestra.run.smithi157.stdout:3035 read 0xb7b200 thru 0xb7c270 (0x1071 bytes) 2024-09-13T09:13:02.057 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:02.057 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:02.057 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:02.057 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:02.057 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:02.061 INFO:teuthology.orchestra.run.smithi157.stdout:3037 trunc from 0x3c76800 to 0xd682400 2024-09-13T09:13:02.067 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:13:02.070 INFO:teuthology.orchestra.run.smithi157.stdout:3038 clone 20 order 19 su 131072 sc 14 2024-09-13T09:13:02.151 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:02.150+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:02.217 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:02.217 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:02.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:02.217 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:02.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.217 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:6867 3.13 3:cd293326:::rbd_header.112326fb43d3:head [watch reconnect cookie 140383087120544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T09:13:02.217 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:13:02.217 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.685587001000002, 2024-09-13T09:13:02.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0277181360000001, 2024-09-13T09:13:02.217 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.218 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:02.218 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.218 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:13:02.218 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:13:02.218 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6867 2024-09-13T09:13:02.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.218 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:13:02.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:13:02.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529067+0000", 2024-09-13T09:13:02.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-09-13T09:13:02.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529073+0000", 2024-09-13T09:13:02.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:13:02.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529087+0000", 2024-09-13T09:13:02.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3825000000000001e-05 2024-09-13T09:13:02.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529097+0000", 2024-09-13T09:13:02.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9559999999999992e-06 2024-09-13T09:13:02.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529149+0000", 2024-09-13T09:13:02.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1668000000000001e-05 2024-09-13T09:13:02.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529161+0000", 2024-09-13T09:13:02.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1831000000000001e-05 2024-09-13T09:13:02.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.554941+0000", 2024-09-13T09:13:02.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0257799729999999 2024-09-13T09:13:02.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.555239+0000", 2024-09-13T09:13:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00029829100000000001 2024-09-13T09:13:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.556787+0000", 2024-09-13T09:13:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001547617 2024-09-13T09:13:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20898 3.13 3:cd293326:::rbd_header.112326fb43d3:head [watch reconnect cookie 140642395970896 gen 16] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T09:13:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:13:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.680924182999998, 2024-09-13T09:13:02.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0236151950000001, 2024-09-13T09:13:02.223 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.223 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:02.223 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.223 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:02.223 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.223 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20898 2024-09-13T09:13:02.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.223 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.223 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:13:02.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:13:02.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533729+0000", 2024-09-13T09:13:02.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-09-13T09:13:02.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533736+0000", 2024-09-13T09:13:02.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9670000000000004e-06 2024-09-13T09:13:02.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533749+0000", 2024-09-13T09:13:02.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2449000000000001e-05 2024-09-13T09:13:02.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533757+0000", 2024-09-13T09:13:02.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:13:02.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533813+0000", 2024-09-13T09:13:02.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5683000000000002e-05 2024-09-13T09:13:02.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533823+0000", 2024-09-13T09:13:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0168e-05 2024-09-13T09:13:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.555940+0000", 2024-09-13T09:13:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0221170559999999 2024-09-13T09:13:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.556047+0000", 2024-09-13T09:13:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000106881 2024-09-13T09:13:02.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.227 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.227 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.557347+0000", 2024-09-13T09:13:02.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001299323 2024-09-13T09:13:02.227 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.227 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.227 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.227 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4934.0:10508 3.8 3:102dac2d:::rbd_data.11238aebf726.0000000000000005:15 [read 5144576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:02.227 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:57.944047+0000", 2024-09-13T09:13:02.227 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2706087930000001, 2024-09-13T09:13:02.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.031703787999999997, 2024-09-13T09:13:02.227 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.227 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:02.228 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.228 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4934", 2024-09-13T09:13:02.228 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:02.228 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 10508 2024-09-13T09:13:02.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.228 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.228 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.228 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944047+0000", 2024-09-13T09:13:02.228 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.228 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.228 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944047+0000", 2024-09-13T09:13:02.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.229 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.229 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944046+0000", 2024-09-13T09:13:02.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:02.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.229 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.229 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944048+0000", 2024-09-13T09:13:02.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0890000000000001e-06 2024-09-13T09:13:02.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.230 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.230 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944050+0000", 2024-09-13T09:13:02.230 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8569999999999999e-06 2024-09-13T09:13:02.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.230 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.230 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944053+0000", 2024-09-13T09:13:02.230 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.384e-06 2024-09-13T09:13:02.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.230 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.230 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.975569+0000", 2024-09-13T09:13:02.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.031516377999999998 2024-09-13T09:13:02.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.975654+0000", 2024-09-13T09:13:02.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4635999999999994e-05 2024-09-13T09:13:02.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.975750+0000", 2024-09-13T09:13:02.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6650000000000005e-05 2024-09-13T09:13:02.231 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.232 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.232 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.232 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4934.0:10509 3.8 3:102dac2d:::rbd_data.11238aebf726.0000000000000005:15 [read 5373952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:02.232 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:57.944055+0000", 2024-09-13T09:13:02.232 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2706005710000001, 2024-09-13T09:13:02.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.031824725999999998, 2024-09-13T09:13:02.232 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.232 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:02.232 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.232 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4934", 2024-09-13T09:13:02.232 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:02.232 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 10509 2024-09-13T09:13:02.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.233 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.233 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.233 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944055+0000", 2024-09-13T09:13:02.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.233 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.233 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944055+0000", 2024-09-13T09:13:02.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.233 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944055+0000", 2024-09-13T09:13:02.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:02.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944056+0000", 2024-09-13T09:13:02.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0899999999999999e-06 2024-09-13T09:13:02.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944059+0000", 2024-09-13T09:13:02.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7709999999999998e-06 2024-09-13T09:13:02.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944061+0000", 2024-09-13T09:13:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3609999999999999e-06 2024-09-13T09:13:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.975778+0000", 2024-09-13T09:13:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.031717530000000001 2024-09-13T09:13:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.975826+0000", 2024-09-13T09:13:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7577999999999997e-05 2024-09-13T09:13:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.235 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.975880+0000", 2024-09-13T09:13:02.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3599e-05 2024-09-13T09:13:02.236 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.236 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.236 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.236 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4934.0:10510 3.8 3:102dac2d:::rbd_data.11238aebf726.0000000000000005:15 [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:02.236 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:57.944071+0000", 2024-09-13T09:13:02.236 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2705843149999998, 2024-09-13T09:13:02.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.031988745999999998, 2024-09-13T09:13:02.236 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.236 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:02.236 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.236 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4934", 2024-09-13T09:13:02.237 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:02.237 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 10510 2024-09-13T09:13:02.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.237 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944071+0000", 2024-09-13T09:13:02.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944071+0000", 2024-09-13T09:13:02.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944071+0000", 2024-09-13T09:13:02.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944072+0000", 2024-09-13T09:13:02.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.502e-06 2024-09-13T09:13:02.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944077+0000", 2024-09-13T09:13:02.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6709999999999998e-06 2024-09-13T09:13:02.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944080+0000", 2024-09-13T09:13:02.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2389999999999998e-06 2024-09-13T09:13:02.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.975914+0000", 2024-09-13T09:13:02.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.031833442000000003 2024-09-13T09:13:02.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976004+0000", 2024-09-13T09:13:02.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0704000000000004e-05 2024-09-13T09:13:02.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.240 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976060+0000", 2024-09-13T09:13:02.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5467000000000003e-05 2024-09-13T09:13:02.240 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.240 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.240 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.240 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4934.0:10511 3.8 3:102dac2d:::rbd_data.11238aebf726.0000000000000005:15 [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:02.240 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:57.944083+0000", 2024-09-13T09:13:02.240 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2705727790000001, 2024-09-13T09:13:02.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032099374999999999, 2024-09-13T09:13:02.241 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.241 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:02.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4934", 2024-09-13T09:13:02.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:02.241 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 10511 2024-09-13T09:13:02.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.241 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.241 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944083+0000", 2024-09-13T09:13:02.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944083+0000", 2024-09-13T09:13:02.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944083+0000", 2024-09-13T09:13:02.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:02.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944084+0000", 2024-09-13T09:13:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.15e-06 2024-09-13T09:13:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944088+0000", 2024-09-13T09:13:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8700000000000002e-06 2024-09-13T09:13:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944090+0000", 2024-09-13T09:13:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:13:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.243 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976082+0000", 2024-09-13T09:13:02.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.031991994000000003 2024-09-13T09:13:02.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976128+0000", 2024-09-13T09:13:02.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6158000000000001e-05 2024-09-13T09:13:02.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976182+0000", 2024-09-13T09:13:02.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4005000000000002e-05 2024-09-13T09:13:02.244 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.245 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.245 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.245 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4934.0:10512 3.8 3:102dac2d:::rbd_data.11238aebf726.0000000000000005:15 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:02.245 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:57.944092+0000", 2024-09-13T09:13:02.245 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2705634090000002, 2024-09-13T09:13:02.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032243894000000002, 2024-09-13T09:13:02.245 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.245 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:02.245 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.245 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4934", 2024-09-13T09:13:02.245 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:02.245 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 10512 2024-09-13T09:13:02.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.246 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944092+0000", 2024-09-13T09:13:02.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944092+0000", 2024-09-13T09:13:02.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944092+0000", 2024-09-13T09:13:02.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:02.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944093+0000", 2024-09-13T09:13:02.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1209999999999999e-06 2024-09-13T09:13:02.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944097+0000", 2024-09-13T09:13:02.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7900000000000001e-06 2024-09-13T09:13:02.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.248 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944099+0000", 2024-09-13T09:13:02.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:13:02.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.248 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976204+0000", 2024-09-13T09:13:02.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032104955999999997 2024-09-13T09:13:02.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976282+0000", 2024-09-13T09:13:02.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8202999999999996e-05 2024-09-13T09:13:02.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976336+0000", 2024-09-13T09:13:02.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3560999999999998e-05 2024-09-13T09:13:02.249 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.249 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.249 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.249 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4934.0:10513 3.8 3:102dac2d:::rbd_data.11238aebf726.0000000000000005:15 [read 6291456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:02.250 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:57.944101+0000", 2024-09-13T09:13:02.250 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2705541340000002, 2024-09-13T09:13:02.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032346946000000001, 2024-09-13T09:13:02.250 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.250 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:02.250 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.250 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4934", 2024-09-13T09:13:02.250 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:02.250 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 10513 2024-09-13T09:13:02.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.250 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.250 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944101+0000", 2024-09-13T09:13:02.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944101+0000", 2024-09-13T09:13:02.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944101+0000", 2024-09-13T09:13:02.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:02.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944102+0000", 2024-09-13T09:13:02.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1459999999999999e-06 2024-09-13T09:13:02.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944105+0000", 2024-09-13T09:13:02.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9840000000000001e-06 2024-09-13T09:13:02.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944108+0000", 2024-09-13T09:13:02.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.328e-06 2024-09-13T09:13:02.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976355+0000", 2024-09-13T09:13:02.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032246945999999999 2024-09-13T09:13:02.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976396+0000", 2024-09-13T09:13:02.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1267999999999999e-05 2024-09-13T09:13:02.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976448+0000", 2024-09-13T09:13:02.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2478999999999997e-05 2024-09-13T09:13:02.254 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.254 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.254 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.254 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4934.0:10514 3.8 3:102dac2d:::rbd_data.11238aebf726.0000000000000005:15 [read 6520832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:02.254 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:57.944110+0000", 2024-09-13T09:13:02.254 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2705457789999999, 2024-09-13T09:13:02.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032461702000000002, 2024-09-13T09:13:02.254 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.254 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:02.254 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.254 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4934", 2024-09-13T09:13:02.254 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:02.255 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 10514 2024-09-13T09:13:02.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.255 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944110+0000", 2024-09-13T09:13:02.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944110+0000", 2024-09-13T09:13:02.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944110+0000", 2024-09-13T09:13:02.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:02.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944116+0000", 2024-09-13T09:13:02.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:13:02.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944120+0000", 2024-09-13T09:13:02.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2139999999999998e-06 2024-09-13T09:13:02.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944123+0000", 2024-09-13T09:13:02.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6699999999999998e-06 2024-09-13T09:13:02.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976469+0000", 2024-09-13T09:13:02.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032345845999999998 2024-09-13T09:13:02.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976517+0000", 2024-09-13T09:13:02.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8381999999999999e-05 2024-09-13T09:13:02.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976571+0000", 2024-09-13T09:13:02.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4194999999999999e-05 2024-09-13T09:13:02.258 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.258 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.258 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.259 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4934.0:10515 3.8 3:102dac2d:::rbd_data.11238aebf726.0000000000000005:15 [read 6750208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:02.259 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:57.944125+0000", 2024-09-13T09:13:02.259 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2705301520000001, 2024-09-13T09:13:02.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032566652000000001, 2024-09-13T09:13:02.259 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.259 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:02.259 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.259 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4934", 2024-09-13T09:13:02.259 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:02.259 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 10515 2024-09-13T09:13:02.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.259 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944125+0000", 2024-09-13T09:13:02.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944125+0000", 2024-09-13T09:13:02.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944125+0000", 2024-09-13T09:13:02.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:02.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944126+0000", 2024-09-13T09:13:02.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1799999999999999e-06 2024-09-13T09:13:02.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944130+0000", 2024-09-13T09:13:02.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9770000000000002e-06 2024-09-13T09:13:02.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944135+0000", 2024-09-13T09:13:02.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:13:02.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976592+0000", 2024-09-13T09:13:02.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032457342 2024-09-13T09:13:02.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976632+0000", 2024-09-13T09:13:02.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9696000000000001e-05 2024-09-13T09:13:02.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976692+0000", 2024-09-13T09:13:02.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9898000000000001e-05 2024-09-13T09:13:02.263 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.263 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.263 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.263 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4934.0:10516 3.8 3:102dac2d:::rbd_data.11238aebf726.0000000000000005:15 [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:02.263 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:57.944139+0000", 2024-09-13T09:13:02.263 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.270516196, 2024-09-13T09:13:02.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032659863999999997, 2024-09-13T09:13:02.263 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.263 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:02.264 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.264 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4934", 2024-09-13T09:13:02.264 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:02.264 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 10516 2024-09-13T09:13:02.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.264 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944139+0000", 2024-09-13T09:13:02.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944139+0000", 2024-09-13T09:13:02.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944139+0000", 2024-09-13T09:13:02.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944141+0000", 2024-09-13T09:13:02.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8920000000000001e-06 2024-09-13T09:13:02.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944148+0000", 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9709999999999998e-06 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944152+0000", 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5759999999999999e-06 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976714+0000", 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032561081999999998 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976756+0000", 2024-09-13T09:13:02.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2023000000000001e-05 2024-09-13T09:13:02.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976799+0000", 2024-09-13T09:13:02.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3600000000000003e-05 2024-09-13T09:13:02.267 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.267 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.267 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.267 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4934.0:10517 3.8 3:102dac2d:::rbd_data.11238aebf726.0000000000000005:15 [read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:02.267 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:57.944157+0000", 2024-09-13T09:13:02.267 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2704983319999998, 2024-09-13T09:13:02.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032741457000000002, 2024-09-13T09:13:02.268 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.268 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:02.268 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.268 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4934", 2024-09-13T09:13:02.268 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:02.268 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 10517 2024-09-13T09:13:02.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.268 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.268 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944157+0000", 2024-09-13T09:13:02.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944157+0000", 2024-09-13T09:13:02.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944157+0000", 2024-09-13T09:13:02.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944159+0000", 2024-09-13T09:13:02.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3180000000000001e-06 2024-09-13T09:13:02.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944173+0000", 2024-09-13T09:13:02.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3811e-05 2024-09-13T09:13:02.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.944177+0000", 2024-09-13T09:13:02.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9380000000000002e-06 2024-09-13T09:13:02.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976818+0000", 2024-09-13T09:13:02.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032640863999999999 2024-09-13T09:13:02.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976861+0000", 2024-09-13T09:13:02.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2967999999999999e-05 2024-09-13T09:13:02.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:57.976899+0000", 2024-09-13T09:13:02.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7948999999999999e-05 2024-09-13T09:13:02.271 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.271 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.272 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.272 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4991.0:2781 3.1e 3:7a934345:::rbd_object_map.1123b788e5d5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:13:02.272 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:01.466268+0000", 2024-09-13T09:13:02.272 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.74838714799999995, 2024-09-13T09:13:02.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032639467999999998, 2024-09-13T09:13:02.272 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.272 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:02.272 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.272 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4991", 2024-09-13T09:13:02.272 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2817394591", 2024-09-13T09:13:02.272 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2781 2024-09-13T09:13:02.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.273 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.273 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466268+0000", 2024-09-13T09:13:02.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.273 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466268+0000", 2024-09-13T09:13:02.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.274 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.274 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466268+0000", 2024-09-13T09:13:02.274 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.274 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.274 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466271+0000", 2024-09-13T09:13:02.274 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4070000000000002e-06 2024-09-13T09:13:02.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.274 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.274 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466278+0000", 2024-09-13T09:13:02.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2079999999999997e-06 2024-09-13T09:13:02.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466282+0000", 2024-09-13T09:13:02.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2699999999999998e-06 2024-09-13T09:13:02.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.493065+0000", 2024-09-13T09:13:02.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.026783409000000001 2024-09-13T09:13:02.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.493142+0000", 2024-09-13T09:13:02.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6959999999999995e-05 2024-09-13T09:13:02.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:02.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.493471+0000", 2024-09-13T09:13:02.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00032892700000000002 2024-09-13T09:13:02.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:02.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.495555+0000", 2024-09-13T09:13:02.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020841570000000001 2024-09-13T09:13:02.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:13:02.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.498853+0000", 2024-09-13T09:13:02.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003297492 2024-09-13T09:13:02.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:13:02.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.498884+0000", 2024-09-13T09:13:02.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0930999999999999e-05 2024-09-13T09:13:02.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.278 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.498908+0000", 2024-09-13T09:13:02.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4131999999999999e-05 2024-09-13T09:13:02.279 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.279 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.279 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.279 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4991.0:2782 3.1e 3:7a934345:::rbd_object_map.1123b788e5d5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:13:02.279 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:01.466287+0000", 2024-09-13T09:13:02.279 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.74836898100000004, 2024-09-13T09:13:02.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032707737000000001, 2024-09-13T09:13:02.280 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.280 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:02.280 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.280 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4991", 2024-09-13T09:13:02.280 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2817394591", 2024-09-13T09:13:02.280 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2782 2024-09-13T09:13:02.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.280 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.280 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.280 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466287+0000", 2024-09-13T09:13:02.280 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466287+0000", 2024-09-13T09:13:02.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466286+0000", 2024-09-13T09:13:02.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466289+0000", 2024-09-13T09:13:02.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.88e-06 2024-09-13T09:13:02.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.282 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.282 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466295+0000", 2024-09-13T09:13:02.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:13:02.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.282 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.282 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466300+0000", 2024-09-13T09:13:02.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5979999999999999e-06 2024-09-13T09:13:02.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.282 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.282 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.494026+0000", 2024-09-13T09:13:02.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.027725970999999999 2024-09-13T09:13:02.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.494110+0000", 2024-09-13T09:13:02.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4653000000000004e-05 2024-09-13T09:13:02.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:02.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.494491+0000", 2024-09-13T09:13:02.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00038088 2024-09-13T09:13:02.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:02.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.495591+0000", 2024-09-13T09:13:02.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001099318 2024-09-13T09:13:02.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:13:02.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.498951+0000", 2024-09-13T09:13:02.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033606339999999999 2024-09-13T09:13:02.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:13:02.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.498976+0000", 2024-09-13T09:13:02.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4927999999999999e-05 2024-09-13T09:13:02.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.498994+0000", 2024-09-13T09:13:02.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8070000000000001e-05 2024-09-13T09:13:02.285 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.285 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.285 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.285 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4991.0:2783 3.1e 3:7a934345:::rbd_object_map.1123b788e5d5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:13:02.285 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:01.466307+0000", 2024-09-13T09:13:02.285 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.74834828200000003, 2024-09-13T09:13:02.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032783095999999998, 2024-09-13T09:13:02.286 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.286 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:02.286 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.286 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4991", 2024-09-13T09:13:02.286 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2817394591", 2024-09-13T09:13:02.286 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2783 2024-09-13T09:13:02.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.286 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.287 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466307+0000", 2024-09-13T09:13:02.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.287 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466307+0000", 2024-09-13T09:13:02.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.287 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466307+0000", 2024-09-13T09:13:02.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466310+0000", 2024-09-13T09:13:02.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1219999999999999e-06 2024-09-13T09:13:02.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466322+0000", 2024-09-13T09:13:02.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1779e-05 2024-09-13T09:13:02.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466327+0000", 2024-09-13T09:13:02.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3229999999999997e-06 2024-09-13T09:13:02.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.494940+0000", 2024-09-13T09:13:02.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.028612552999999999 2024-09-13T09:13:02.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.494996+0000", 2024-09-13T09:13:02.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6904999999999998e-05 2024-09-13T09:13:02.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:02.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.495210+0000", 2024-09-13T09:13:02.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00021347900000000001 2024-09-13T09:13:02.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:02.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.497533+0000", 2024-09-13T09:13:02.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023229090000000002 2024-09-13T09:13:02.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:13:02.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499002+0000", 2024-09-13T09:13:02.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0014687789999999999 2024-09-13T09:13:02.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.291 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:13:02.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499043+0000", 2024-09-13T09:13:02.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1454e-05 2024-09-13T09:13:02.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.291 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499090+0000", 2024-09-13T09:13:02.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7199e-05 2024-09-13T09:13:02.291 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.291 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.291 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.292 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4991.0:2784 3.1e 3:7a934345:::rbd_object_map.1123b788e5d5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:13:02.292 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:01.466332+0000", 2024-09-13T09:13:02.292 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.74832376499999997, 2024-09-13T09:13:02.292 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032851261999999999, 2024-09-13T09:13:02.292 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.292 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:02.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4991", 2024-09-13T09:13:02.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2817394591", 2024-09-13T09:13:02.292 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2784 2024-09-13T09:13:02.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.293 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466332+0000", 2024-09-13T09:13:02.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466332+0000", 2024-09-13T09:13:02.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466331+0000", 2024-09-13T09:13:02.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466335+0000", 2024-09-13T09:13:02.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.444e-06 2024-09-13T09:13:02.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466342+0000", 2024-09-13T09:13:02.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2810000000000003e-06 2024-09-13T09:13:02.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466347+0000", 2024-09-13T09:13:02.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.113e-06 2024-09-13T09:13:02.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.495616+0000", 2024-09-13T09:13:02.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.029269238 2024-09-13T09:13:02.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.495686+0000", 2024-09-13T09:13:02.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9968999999999994e-05 2024-09-13T09:13:02.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:02.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.496043+0000", 2024-09-13T09:13:02.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0003569 2024-09-13T09:13:02.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:02.297 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.497575+0000", 2024-09-13T09:13:02.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015313359999999999 2024-09-13T09:13:02.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.297 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:13:02.297 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499104+0000", 2024-09-13T09:13:02.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001529599 2024-09-13T09:13:02.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.297 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:13:02.297 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499149+0000", 2024-09-13T09:13:02.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5036e-05 2024-09-13T09:13:02.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499183+0000", 2024-09-13T09:13:02.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3742999999999999e-05 2024-09-13T09:13:02.298 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.298 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.298 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.298 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4991.0:2785 3.1e 3:7a934345:::rbd_object_map.1123b788e5d5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:13:02.298 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:01.466352+0000", 2024-09-13T09:13:02.298 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.74830348300000005, 2024-09-13T09:13:02.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032916329000000001, 2024-09-13T09:13:02.298 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.298 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:02.299 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.299 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4991", 2024-09-13T09:13:02.299 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2817394591", 2024-09-13T09:13:02.299 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2785 2024-09-13T09:13:02.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.299 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466352+0000", 2024-09-13T09:13:02.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466352+0000", 2024-09-13T09:13:02.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466352+0000", 2024-09-13T09:13:02.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466355+0000", 2024-09-13T09:13:02.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9450000000000002e-06 2024-09-13T09:13:02.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466362+0000", 2024-09-13T09:13:02.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1949999999999997e-06 2024-09-13T09:13:02.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466367+0000", 2024-09-13T09:13:02.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8849999999999998e-06 2024-09-13T09:13:02.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.496463+0000", 2024-09-13T09:13:02.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.030096166000000001 2024-09-13T09:13:02.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.302 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.302 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.496538+0000", 2024-09-13T09:13:02.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5537999999999994e-05 2024-09-13T09:13:02.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.302 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:02.302 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.496859+0000", 2024-09-13T09:13:02.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00032110999999999999 2024-09-13T09:13:02.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:02.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.498944+0000", 2024-09-13T09:13:02.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020842489999999998 2024-09-13T09:13:02.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:13:02.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499195+0000", 2024-09-13T09:13:02.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00025119299999999998 2024-09-13T09:13:02.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:13:02.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499235+0000", 2024-09-13T09:13:02.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9625000000000001e-05 2024-09-13T09:13:02.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499268+0000", 2024-09-13T09:13:02.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3819000000000003e-05 2024-09-13T09:13:02.304 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.304 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.304 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.304 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4991.0:2786 3.1e 3:7a934345:::rbd_object_map.1123b788e5d5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:13:02.304 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:01.466372+0000", 2024-09-13T09:13:02.304 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.74828353299999995, 2024-09-13T09:13:02.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.033027191999999997, 2024-09-13T09:13:02.304 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.305 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:02.305 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.305 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4991", 2024-09-13T09:13:02.305 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2817394591", 2024-09-13T09:13:02.305 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2786 2024-09-13T09:13:02.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.305 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466372+0000", 2024-09-13T09:13:02.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.306 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466372+0000", 2024-09-13T09:13:02.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.306 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466372+0000", 2024-09-13T09:13:02.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.306 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466374+0000", 2024-09-13T09:13:02.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7410000000000001e-06 2024-09-13T09:13:02.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466381+0000", 2024-09-13T09:13:02.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6229999999999999e-06 2024-09-13T09:13:02.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466386+0000", 2024-09-13T09:13:02.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.95e-06 2024-09-13T09:13:02.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.497602+0000", 2024-09-13T09:13:02.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.031215989 2024-09-13T09:13:02.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.497685+0000", 2024-09-13T09:13:02.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3298000000000001e-05 2024-09-13T09:13:02.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:02.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.498035+0000", 2024-09-13T09:13:02.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035020599999999999 2024-09-13T09:13:02.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:13:02.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499277+0000", 2024-09-13T09:13:02.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0012420420000000001 2024-09-13T09:13:02.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:02.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499320+0000", 2024-09-13T09:13:02.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2598000000000001e-05 2024-09-13T09:13:02.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:13:02.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499365+0000", 2024-09-13T09:13:02.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5121999999999999e-05 2024-09-13T09:13:02.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499399+0000", 2024-09-13T09:13:02.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3966e-05 2024-09-13T09:13:02.310 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.310 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.310 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.310 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4991.0:2787 3.1e 3:7a934345:::rbd_object_map.1123b788e5d5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:13:02.310 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:01.466390+0000", 2024-09-13T09:13:02.310 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.74826523300000003, 2024-09-13T09:13:02.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.035053405000000003, 2024-09-13T09:13:02.311 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.311 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:02.311 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.311 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4991", 2024-09-13T09:13:02.311 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2817394591", 2024-09-13T09:13:02.311 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2787 2024-09-13T09:13:02.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.311 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.311 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.311 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466390+0000", 2024-09-13T09:13:02.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466390+0000", 2024-09-13T09:13:02.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466390+0000", 2024-09-13T09:13:02.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466393+0000", 2024-09-13T09:13:02.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0149999999999999e-06 2024-09-13T09:13:02.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466401+0000", 2024-09-13T09:13:02.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9799999999999998e-06 2024-09-13T09:13:02.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466406+0000", 2024-09-13T09:13:02.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3290000000000001e-06 2024-09-13T09:13:02.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.498346+0000", 2024-09-13T09:13:02.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.031939800999999997 2024-09-13T09:13:02.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.498393+0000", 2024-09-13T09:13:02.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6798000000000002e-05 2024-09-13T09:13:02.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:02.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.498592+0000", 2024-09-13T09:13:02.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00019930600000000001 2024-09-13T09:13:02.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:13:02.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499415+0000", 2024-09-13T09:13:02.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00082278799999999997 2024-09-13T09:13:02.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:02.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.501254+0000", 2024-09-13T09:13:02.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0018392689999999999 2024-09-13T09:13:02.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:13:02.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.501409+0000", 2024-09-13T09:13:02.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015475499999999999 2024-09-13T09:13:02.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.501444+0000", 2024-09-13T09:13:02.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4709000000000003e-05 2024-09-13T09:13:02.316 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.316 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.316 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.316 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4991.0:2788 3.1e 3:7a934345:::rbd_object_map.1123b788e5d5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:13:02.316 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:01.466411+0000", 2024-09-13T09:13:02.316 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.748244507, 2024-09-13T09:13:02.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.036222703000000002, 2024-09-13T09:13:02.316 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:02.316 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:02.316 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:02.317 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4991", 2024-09-13T09:13:02.317 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/2817394591", 2024-09-13T09:13:02.317 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 2788 2024-09-13T09:13:02.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.317 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:02.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:02.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466411+0000", 2024-09-13T09:13:02.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:02.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466411+0000", 2024-09-13T09:13:02.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:02.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:02.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466411+0000", 2024-09-13T09:13:02.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:02.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466414+0000", 2024-09-13T09:13:02.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9919999999999999e-06 2024-09-13T09:13:02.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:02.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466420+0000", 2024-09-13T09:13:02.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6529999999999997e-06 2024-09-13T09:13:02.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.466425+0000", 2024-09-13T09:13:02.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7249999999999997e-06 2024-09-13T09:13:02.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:02.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499456+0000", 2024-09-13T09:13:02.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.033031035 2024-09-13T09:13:02.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:02.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499553+0000", 2024-09-13T09:13:02.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6923999999999996e-05 2024-09-13T09:13:02.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:02.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.499906+0000", 2024-09-13T09:13:02.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000352803 2024-09-13T09:13:02.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:02.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.501693+0000", 2024-09-13T09:13:02.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001787194 2024-09-13T09:13:02.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:13:02.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.502560+0000", 2024-09-13T09:13:02.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00086665799999999997 2024-09-13T09:13:02.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:13:02.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.502602+0000", 2024-09-13T09:13:02.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2079000000000001e-05 2024-09-13T09:13:02.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:02.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:02.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:02.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:01.502634+0000", 2024-09-13T09:13:02.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2002999999999998e-05 2024-09-13T09:13:02.322 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.322 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.322 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.322 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:02.322 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:02.322 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:02.324 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:02.324 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:02.321+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:02.342 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-09-13T09:13:02.348 INFO:teuthology.orchestra.run.smithi044.stderr:set 2.7 pg_upmap mapping to [5,4] 2024-09-13T09:13:02.423 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:02.422+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:02.430 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:02.431 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:02.431 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:02.440 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:02.479 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone19 from 0xd682400 (overlap 0x123dc00) to 0xcc4c00 2024-09-13T09:13:02.526 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:02.523+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:02.602 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:02.602 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:02.602 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:02.602 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:02.602 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:02.611 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:02.655 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:02.653+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:02.716 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.193228928, 2024-09-13T09:13:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:13:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:13:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:13:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:13:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:13:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:13:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:13:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:13:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:13:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:13:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:13:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:13:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:13:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:13:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:13:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:13:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.193179872000002, 2024-09-13T09:13:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:13:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:13:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:13:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:13:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:13:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:13:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:13:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:13:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:13:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:13:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:13:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:13:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:13:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:13:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:13:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:13:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.193139797000001, 2024-09-13T09:13:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:13:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:13:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:13:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:13:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:13:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:13:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:13:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:13:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:13:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:13:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:13:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:13:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:13:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:13:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:13:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:13:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.193109981999999, 2024-09-13T09:13:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:13:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:13:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:13:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:13:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:13:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:13:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:13:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:13:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:13:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:13:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:13:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:13:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:13:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:13:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:13:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.193080321, 2024-09-13T09:13:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:13:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:13:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:13:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:13:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:13:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:13:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:13:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:13:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:13:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:13:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:13:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:13:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:13:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:13:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:13:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:13:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.193051113999999, 2024-09-13T09:13:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:13:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:13:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:13:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:13:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:13:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:13:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:13:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:13:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:13:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:13:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:13:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:13:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:13:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:13:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:13:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:13:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.192994933000001, 2024-09-13T09:13:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:13:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:13:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:13:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:13:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:13:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:13:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:13:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:13:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:13:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:13:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:13:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:13:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:13:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:13:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:13:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:13:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.192941472000001, 2024-09-13T09:13:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:13:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:13:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:13:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:13:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:13:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:13:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:13:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:13:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:13:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:13:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:13:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:13:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:13:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:13:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:13:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:13:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:13:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.192884641999999, 2024-09-13T09:13:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:13:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:13:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:13:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:13:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:13:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:13:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:13:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:13:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:13:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:13:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:13:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:13:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:13:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:13:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:13:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:13:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:13:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.192833615000001, 2024-09-13T09:13:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:13:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:13:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:13:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:13:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:13:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:13:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:13:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:13:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:13:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:13:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:13:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:13:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:13:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:13:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:13:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:13:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:13:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.192792425, 2024-09-13T09:13:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:13:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:13:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:13:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:13:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:13:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:13:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:13:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:13:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:13:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:13:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:13:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:13:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:13:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:13:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:13:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:13:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.192750309000001, 2024-09-13T09:13:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:13:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:13:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:13:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:13:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:13:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:13:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:13:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:13:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:13:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:13:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:13:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:13:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:13:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:13:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:13:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:13:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.192706430000001, 2024-09-13T09:13:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:13:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:13:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:13:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:13:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:13:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:13:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:13:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:13:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:13:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:13:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:13:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:13:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:13:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:13:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:13:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:13:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.192660513, 2024-09-13T09:13:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:13:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:13:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:13:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:13:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:13:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:13:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:13:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:13:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:13:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:13:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:13:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:13:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:13:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:13:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:13:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:13:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:13:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.192610741999999, 2024-09-13T09:13:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:13:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:13:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:13:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:13:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:13:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:13:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:13:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:13:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:13:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:13:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:13:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:13:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:13:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:13:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:13:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:13:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.192565338000001, 2024-09-13T09:13:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:13:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:13:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:13:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:13:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:13:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:13:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:13:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:13:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:13:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:13:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:13:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:13:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:13:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:13:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:13:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:13:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.1925147, 2024-09-13T09:13:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:13:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:13:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:13:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:13:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:13:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:13:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:13:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:13:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:13:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:13:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:13:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:13:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:13:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:13:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:13:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:13:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:13:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.192458856999998, 2024-09-13T09:13:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:13:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:13:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:13:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:13:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:13:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:13:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:13:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:13:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:13:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:13:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:13:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:13:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:13:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:13:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:13:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:13:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.192401934999999, 2024-09-13T09:13:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:13:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:13:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:13:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:13:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:13:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:13:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:13:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:13:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:13:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:13:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:13:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:13:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:13:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:13:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:13:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:13:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.192348372000001, 2024-09-13T09:13:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:13:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:13:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:13:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:13:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:13:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:13:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:13:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:13:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:13:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:13:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:13:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:13:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:13:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:13:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:13:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:13:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.838 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.838 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:02.838 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:02.838 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:02.839 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:02.840 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:02.837+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:02.905 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:13:02.949 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:02.948+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:02.986 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:02.986 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:02.986 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:02.986 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:02.994 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:03.030 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:03.031 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:03.041 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:03.039+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:03.048 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:03.048 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:03.063 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:13:03.064 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:13:03.080 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:03.080 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:03.097 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:03.097 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:03.101 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:03.101 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:03.110 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:03.113 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:03.113 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:03.116 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-13T09:13:03.167 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:03.166+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:3039 write 0x6132e00 thru 0x61380df (0x52e0 bytes) 2024-09-13T09:13:03.218 INFO:teuthology.orchestra.run.smithi157.stdout:3041 read 0x7021600 thru 0x702a3cb (0x8dcc bytes) 2024-09-13T09:13:03.219 INFO:teuthology.orchestra.run.smithi157.stdout:3043 write 0xaf0cc00 thru 0xaf11a31 (0x4e32 bytes) 2024-09-13T09:13:03.225 INFO:teuthology.orchestra.run.smithi157.stdout:3044 trunc from 0xd682400 to 0x8dc6e00 2024-09-13T09:13:03.243 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:03.242+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:03.259 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:03.259 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:03.268 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:03.277 INFO:teuthology.orchestra.run.smithi157.stdout:3049 write 0x9803800 thru 0x98126f7 (0xeef8 bytes) 2024-09-13T09:13:03.277 INFO:teuthology.orchestra.run.smithi157.stdout:3051 write 0x5118000 thru 0x5121397 (0x9398 bytes) 2024-09-13T09:13:03.278 INFO:teuthology.orchestra.run.smithi157.stdout:3054 write 0x32cf200 thru 0x32d9fb8 (0xadb9 bytes) 2024-09-13T09:13:03.278 INFO:teuthology.orchestra.run.smithi157.stdout:3055 read 0x48ca00 thru 0x48cd5e (0x35f bytes) 2024-09-13T09:13:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:3056 write 0xd352400 thru 0xd359ec7 (0x7ac8 bytes) 2024-09-13T09:13:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:3057 write 0x66b9200 thru 0x66bebf8 (0x59f9 bytes) 2024-09-13T09:13:03.302 INFO:teuthology.orchestra.run.smithi157.stdout:3058 write 0xb6b000 thru 0xb78f38 (0xdf39 bytes) 2024-09-13T09:13:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:3060 write 0x81ffe00 thru 0x8201754 (0x1955 bytes) 2024-09-13T09:13:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:3061 write 0xca5aa00 thru 0xca63b97 (0x9198 bytes) 2024-09-13T09:13:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:3062 write 0xcbf4a00 thru 0xcbf579d (0xd9e bytes) 2024-09-13T09:13:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:3064 write 0x4d3da00 thru 0x4d48afb (0xb0fc bytes) 2024-09-13T09:13:03.303 INFO:teuthology.orchestra.run.smithi157.stdout:3065 write 0x921e600 thru 0x922976a (0xb16b bytes) 2024-09-13T09:13:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:3066 read 0x691ce00 thru 0x692106f (0x4270 bytes) 2024-09-13T09:13:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:3067 read 0x7a1f400 thru 0x7a28467 (0x9068 bytes) 2024-09-13T09:13:03.304 INFO:teuthology.orchestra.run.smithi157.stdout:3070 read 0xd193600 thru 0xd19b1f1 (0x7bf2 bytes) 2024-09-13T09:13:03.318 INFO:teuthology.orchestra.run.smithi157.stdout:3074 trunc from 0xd35a000 to 0x1962000 2024-09-13T09:13:03.345 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:03.342+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:03.380 INFO:teuthology.orchestra.run.smithi157.stdout:3075 write 0x2a0400 thru 0x2a71d0 (0x6dd1 bytes) 2024-09-13T09:13:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:03.395 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:03.419 INFO:teuthology.orchestra.run.smithi157.stdout:3076 write 0xe035200 thru 0xe03c8ff (0x7700 bytes) 2024-09-13T09:13:03.419 INFO:teuthology.orchestra.run.smithi157.stdout:3077 punch from 0x23de800 to 0x23eb600, (0xce00 bytes) 2024-09-13T09:13:03.434 INFO:teuthology.orchestra.run.smithi157.stdout:3078 write 0xe764200 thru 0xe7692f8 (0x50f9 bytes) 2024-09-13T09:13:03.435 INFO:teuthology.orchestra.run.smithi157.stdout:3079 write 0x94a8800 thru 0x94a968b (0xe8c bytes) 2024-09-13T09:13:03.438 INFO:teuthology.orchestra.run.smithi157.stdout:3080 trunc from 0xe769400 to 0xc56c400 2024-09-13T09:13:03.455 INFO:teuthology.orchestra.run.smithi157.stdout:3081 write 0xbcba00 thru 0xbd399f (0x7fa0 bytes) 2024-09-13T09:13:03.457 INFO:teuthology.orchestra.run.smithi157.stdout:3082 read 0x7c0dc00 thru 0x7c1028e (0x268f bytes) 2024-09-13T09:13:03.457 INFO:teuthology.orchestra.run.smithi157.stdout:3083 punch from 0x836fc00 to 0x8376c00, (0x7000 bytes) 2024-09-13T09:13:03.458 INFO:teuthology.orchestra.run.smithi157.stdout:3084 clone 21 order 22 su 1048576 sc 9 2024-09-13T09:13:03.480 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:03.478+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:03.544 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:03.544 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:03.545 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:03.544+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:03.553 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:03.646 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:03.645+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:03.698 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:03.699 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:03.707 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:03.752 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:03.751+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:03.817 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:03.817 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:03.825 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:03.848 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:03.847+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:03.949 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:03.948+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:03.972 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:03.972 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:03.981 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:04.066 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:04.065+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:04.129 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:04.129 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:04.137 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:04.151 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:04.150+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:04.252 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:04.251+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:04.285 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:04.285 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:04.294 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:04.361 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:13:04.378 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:04.377+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:04.451 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:04.451 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:04.454 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:04.452+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:04.460 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:04.534 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone20 from 0xc56c400 (overlap 0x1962000) to 0xf3ac00 2024-09-13T09:13:04.555 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:04.554+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:04.607 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:04.608 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:04.619 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:13:04.656 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:04.654+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:04.756 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:04.755+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:04.768 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:04.768 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:04.777 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:13:04.857 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:04.856+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:04.929 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:04.929 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:04.939 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:13:04.959 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:04.957+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:05.049 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:05.049 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:05.058 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:05.060 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:05.059+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:05.127 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:13:05.164 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:05.164 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:05.173 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:05.171+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:05.174 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:05.208 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-13T09:13:05.262 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:05.260+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:05.277 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:05.277 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:05.302 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:05.302 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:05.309 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:05.310 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:05.312 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:05.337 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:13:05.337 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:13:05.363 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:05.362+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:05.364 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:05.364 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:05.379 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-09-13T09:13:05.393 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:05.393 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:05.412 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:05.412 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:05.461 INFO:teuthology.orchestra.run.smithi157.stdout:3085 punch from 0x6ca7e00 to 0x6cab800, (0x3a00 bytes) 2024-09-13T09:13:05.464 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:05.463+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:05.469 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:05.469 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:05.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:05.470 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:05.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.470 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.470 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:05.470 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.946396776, 2024-09-13T09:13:05.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:13:05.470 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.470 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.470 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:13:05.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.471 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:05.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:05.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:05.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:05.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:13:05.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:13:05.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:13:05.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:13:05.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:13:05.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:13:05.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:13:05.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:13:05.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:13:05.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:13:05.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.474 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:05.475 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.946347719999999, 2024-09-13T09:13:05.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:13:05.475 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.475 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.475 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.475 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.475 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.475 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:13:05.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.475 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:05.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:05.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:05.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:05.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:13:05.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:13:05.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:13:05.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:13:05.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:13:05.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:13:05.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:13:05.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:13:05.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:13:05.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:13:05.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:13:05.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:13:05.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:13:05.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:13:05.479 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.479 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.479 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.479 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.479 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:05.479 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.946307645000001, 2024-09-13T09:13:05.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:13:05.480 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.480 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.480 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:13:05.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.480 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:05.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:05.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:05.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:05.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:13:05.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:13:05.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:13:05.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:13:05.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:13:05.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:13:05.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:13:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:13:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.94627783, 2024-09-13T09:13:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:13:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:13:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:13:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:13:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:13:05.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:05.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:13:05.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:13:05.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:13:05.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:13:05.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:13:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:13:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:13:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:13:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:13:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:13:05.489 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.489 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.489 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.489 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.489 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:05.489 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.946248169, 2024-09-13T09:13:05.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:13:05.489 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.489 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.489 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.489 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.489 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.490 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:13:05.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.490 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:05.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:05.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:05.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:05.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:13:05.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:13:05.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:13:05.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:13:05.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:13:05.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:13:05.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:13:05.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:13:05.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:13:05.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:13:05.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:13:05.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:13:05.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:13:05.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:13:05.493 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.493 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.494 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.494 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.494 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:05.494 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.946218962, 2024-09-13T09:13:05.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:13:05.494 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.494 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.494 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.494 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.494 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.494 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:13:05.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.495 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:05.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:05.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:05.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:05.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:13:05.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:13:05.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:13:05.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:13:05.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:13:05.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:13:05.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:13:05.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:13:05.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:13:05.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:13:05.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:13:05.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:13:05.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:13:05.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:13:05.498 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.499 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.499 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.499 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.499 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:05.499 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.946162780999998, 2024-09-13T09:13:05.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:13:05.499 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.499 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.499 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:13:05.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.500 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:05.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:05.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:05.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:05.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:13:05.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:13:05.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:13:05.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:13:05.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:13:05.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:13:05.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:13:05.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:13:05.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:13:05.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:13:05.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:13:05.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:13:05.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:13:05.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:13:05.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.504 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.504 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.504 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:05.504 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.946109320000001, 2024-09-13T09:13:05.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:13:05.504 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.504 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.505 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:13:05.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.505 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:05.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:05.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:13:05.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:05.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:13:05.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:13:05.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:13:05.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:13:05.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:13:05.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:13:05.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:13:05.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:13:05.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:13:05.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:13:05.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:13:05.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:13:05.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:13:05.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:13:05.509 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.509 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.509 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.509 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.509 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:05.509 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.94605249, 2024-09-13T09:13:05.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:13:05.509 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.509 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.510 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.510 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:13:05.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.510 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:05.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:05.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:13:05.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:05.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:13:05.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:13:05.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:13:05.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:13:05.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:13:05.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:13:05.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:13:05.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:13:05.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:13:05.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:13:05.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:13:05.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:13:05.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:13:05.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:13:05.514 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.514 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.514 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.514 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.514 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:05.514 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.946001463000002, 2024-09-13T09:13:05.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:13:05.514 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.514 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.514 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.514 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.515 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.515 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:13:05.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.515 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:05.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:05.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:13:05.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:05.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:13:05.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:13:05.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:13:05.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:13:05.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:13:05.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:13:05.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:13:05.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:13:05.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:13:05.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:13:05.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:13:05.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:13:05.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:13:05.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:13:05.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.518 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.519 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.519 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:05.519 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.945960273000001, 2024-09-13T09:13:05.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:13:05.519 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.519 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.519 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:13:05.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.519 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:05.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:05.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:05.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:05.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:13:05.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:13:05.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:13:05.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:13:05.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:13:05.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:13:05.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:13:05.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:13:05.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:13:05.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:13:05.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:13:05.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:13:05.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:13:05.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:13:05.523 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.523 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.523 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.523 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.524 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:05.524 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.945918157000001, 2024-09-13T09:13:05.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:13:05.524 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.524 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.524 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.524 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.524 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.524 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:13:05.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.524 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:05.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:05.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:05.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:05.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:13:05.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:13:05.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:13:05.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:13:05.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:13:05.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:13:05.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:13:05.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:13:05.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:13:05.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:13:05.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:13:05.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:13:05.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:13:05.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:13:05.528 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.529 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.529 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.529 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.529 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:05.529 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.945874278000002, 2024-09-13T09:13:05.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:13:05.529 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.529 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.530 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:13:05.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.530 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:05.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:05.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:05.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:05.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:13:05.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:13:05.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:13:05.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:13:05.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:13:05.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:13:05.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:13:05.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:13:05.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:13:05.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:13:05.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:13:05.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:13:05.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:13:05.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:13:05.533 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.533 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.533 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.534 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.534 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:05.534 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.945828361, 2024-09-13T09:13:05.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:13:05.534 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.534 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.534 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.534 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.534 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.534 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:13:05.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.534 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:05.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:05.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:13:05.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:05.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:13:05.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:13:05.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:13:05.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:13:05.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:13:05.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:13:05.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:13:05.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:13:05.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:13:05.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:13:05.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:13:05.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:13:05.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:13:05.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:13:05.538 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.538 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.538 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.539 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.539 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:05.539 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.94577859, 2024-09-13T09:13:05.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:13:05.539 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.539 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.539 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.539 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.539 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.539 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:13:05.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.539 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:05.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:05.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:05.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:05.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:13:05.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:13:05.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:13:05.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:13:05.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:13:05.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:13:05.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:13:05.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:13:05.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:13:05.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:13:05.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:13:05.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:13:05.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:13:05.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:13:05.543 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.544 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.544 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.544 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.544 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:05.544 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.945733185999998, 2024-09-13T09:13:05.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:13:05.544 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.544 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.545 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:13:05.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.545 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:05.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:05.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:05.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:05.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:13:05.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:13:05.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:13:05.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:13:05.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:13:05.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:13:05.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:13:05.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:13:05.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:13:05.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:13:05.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:13:05.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:13:05.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:13:05.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:13:05.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.549 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.549 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.549 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:05.549 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.945682548000001, 2024-09-13T09:13:05.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:13:05.549 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.549 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.550 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:13:05.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.550 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:05.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:05.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:13:05.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:05.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:13:05.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:13:05.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:13:05.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:13:05.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:13:05.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:13:05.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:13:05.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:13:05.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:13:05.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:13:05.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:13:05.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:13:05.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:13:05.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:13:05.555 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.555 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.555 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.555 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.555 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:05.555 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.945626704999999, 2024-09-13T09:13:05.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:13:05.555 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.555 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.556 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.556 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.556 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.556 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:13:05.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.556 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:05.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:05.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:05.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:05.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:13:05.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:13:05.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:13:05.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:13:05.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:13:05.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:13:05.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:13:05.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:13:05.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:13:05.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:13:05.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:13:05.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:13:05.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:13:05.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:13:05.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.560 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.560 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.560 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:05.560 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.945569783, 2024-09-13T09:13:05.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:13:05.560 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.560 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.560 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.560 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.560 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.561 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:13:05.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.561 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:05.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:05.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:05.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:05.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:13:05.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:13:05.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:13:05.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:13:05.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:13:05.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:13:05.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:13:05.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:13:05.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:13:05.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:13:05.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:13:05.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:13:05.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:13:05.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:13:05.565 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.565 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.565 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.565 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:05.565 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:05.565 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.945516220000002, 2024-09-13T09:13:05.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:13:05.565 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:05.565 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:05.565 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:05.565 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:05.566 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:05.566 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:13:05.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.566 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:05.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:05.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:05.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:05.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:05.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:05.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:05.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:05.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:05.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:05.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:13:05.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:13:05.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:05.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:13:05.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:13:05.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:05.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:13:05.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:13:05.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:13:05.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:13:05.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:05.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:13:05.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:13:05.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:05.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:13:05.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:13:05.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:05.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:05.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:05.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:13:05.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:13:05.570 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.570 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.570 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.570 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:05.570 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:05.570 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:05.571 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:05.570+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:05.573 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:05.574 INFO:teuthology.orchestra.run.smithi157.stdout:3087 read 0x20d6200 thru 0x20dc51e (0x631f bytes) 2024-09-13T09:13:05.574 INFO:teuthology.orchestra.run.smithi157.stdout:3088 write 0x21e9600 thru 0x21e9d48 (0x749 bytes) 2024-09-13T09:13:05.574 INFO:teuthology.orchestra.run.smithi157.stdout:3089 trunc from 0xc56c400 to 0x5883600 2024-09-13T09:13:05.574 INFO:teuthology.orchestra.run.smithi157.stdout:3091 read 0x14c9e00 thru 0x14d8c11 (0xee12 bytes) 2024-09-13T09:13:05.574 INFO:teuthology.orchestra.run.smithi157.stdout:3092 write 0xbfd9c00 thru 0xbfe601c (0xc41d bytes) 2024-09-13T09:13:05.574 INFO:teuthology.orchestra.run.smithi157.stdout:3094 write 0xd55ea00 thru 0xd56dc15 (0xf216 bytes) 2024-09-13T09:13:05.574 INFO:teuthology.orchestra.run.smithi157.stdout:3095 write 0xa684a00 thru 0xa689e87 (0x5488 bytes) 2024-09-13T09:13:05.574 INFO:teuthology.orchestra.run.smithi157.stdout:3096 write 0x934b600 thru 0x935a31b (0xed1c bytes) 2024-09-13T09:13:05.574 INFO:teuthology.orchestra.run.smithi157.stdout:3097 punch from 0x5887800 to 0x588ba00, (0x4200 bytes) 2024-09-13T09:13:05.574 INFO:teuthology.orchestra.run.smithi157.stdout:3098 read 0x68bc200 thru 0x68c5cb6 (0x9ab7 bytes) 2024-09-13T09:13:05.574 INFO:teuthology.orchestra.run.smithi157.stdout:3099 write 0x6e84600 thru 0x6e84d16 (0x717 bytes) 2024-09-13T09:13:05.575 INFO:teuthology.orchestra.run.smithi157.stdout:3101 write 0x60b6800 thru 0x60ba8a1 (0x40a2 bytes) 2024-09-13T09:13:05.575 INFO:teuthology.orchestra.run.smithi157.stdout:3107 read 0x8201e00 thru 0x820b50e (0x970f bytes) 2024-09-13T09:13:05.575 INFO:teuthology.orchestra.run.smithi157.stdout:3108 trunc from 0xd56de00 to 0xc54ba00 2024-09-13T09:13:05.585 INFO:teuthology.orchestra.run.smithi157.stdout:3111 punch from 0x289e400 to 0x28a7c00, (0x9800 bytes) 2024-09-13T09:13:05.586 INFO:teuthology.orchestra.run.smithi157.stdout:3112 punch from 0x8e5a00 to 0x8e5c00, (0x200 bytes) 2024-09-13T09:13:05.600 INFO:teuthology.orchestra.run.smithi157.stdout:3115 trunc from 0xc54ba00 to 0x1732200 2024-09-13T09:13:05.672 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:05.671+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:05.751 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:05.751 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:05.751 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:05.751 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:05.762 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:05.773 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:05.772+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:05.786 INFO:teuthology.orchestra.run.smithi157.stdout:3117 write 0xaaf7a00 thru 0xab062a3 (0xe8a4 bytes) 2024-09-13T09:13:05.786 INFO:teuthology.orchestra.run.smithi157.stdout:3118 read 0x5fc4800 thru 0x5fd1581 (0xcd82 bytes) 2024-09-13T09:13:05.786 INFO:teuthology.orchestra.run.smithi157.stdout:3120 punch from 0x9705e00 to 0x970d800, (0x7a00 bytes) 2024-09-13T09:13:05.790 INFO:teuthology.orchestra.run.smithi157.stdout:3121 trunc from 0xab06400 to 0x7d3c000 2024-09-13T09:13:05.818 INFO:teuthology.orchestra.run.smithi157.stdout:3124 write 0x8af6800 thru 0x8b04902 (0xe103 bytes) 2024-09-13T09:13:05.823 INFO:teuthology.orchestra.run.smithi157.stdout:3127 trunc from 0x8b04a00 to 0xd598600 2024-09-13T09:13:05.837 INFO:teuthology.orchestra.run.smithi157.stdout:3129 write 0x4933a00 thru 0x4939fd5 (0x65d6 bytes) 2024-09-13T09:13:05.855 INFO:teuthology.orchestra.run.smithi157.stdout:3130 write 0xd9c8600 thru 0xd9d2c1f (0xa620 bytes) 2024-09-13T09:13:05.855 INFO:teuthology.orchestra.run.smithi157.stdout:3132 punch from 0x3432400 to 0x3438000, (0x5c00 bytes) 2024-09-13T09:13:05.855 INFO:teuthology.orchestra.run.smithi157.stdout:3133 read 0x81f6800 thru 0x81ff76f (0x8f70 bytes) 2024-09-13T09:13:05.869 INFO:teuthology.orchestra.run.smithi157.stdout:3134 trunc from 0xd9d2e00 to 0x1a84800 2024-09-13T09:13:05.874 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:05.873+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:05.909 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:05.909 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:05.909 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:05.910 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:05.910 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:05.918 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:06.002 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:06.000+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:06.061 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:06.062 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:06.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:06.062 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:06.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.062 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17752 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2125824~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.062 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138424+0000", 2024-09-13T09:13:06.062 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.921435461000002, 2024-09-13T09:13:06.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045520176000000002, 2024-09-13T09:13:06.062 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.062 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.063 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.063 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.063 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17752 2024-09-13T09:13:06.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.063 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138424+0000", 2024-09-13T09:13:06.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138424+0000", 2024-09-13T09:13:06.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138423+0000", 2024-09-13T09:13:06.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138425+0000", 2024-09-13T09:13:06.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0640000000000001e-06 2024-09-13T09:13:06.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138431+0000", 2024-09-13T09:13:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0900000000000001e-06 2024-09-13T09:13:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138437+0000", 2024-09-13T09:13:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0329999999999997e-06 2024-09-13T09:13:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183854+0000", 2024-09-13T09:13:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045416099000000001 2024-09-13T09:13:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183894+0000", 2024-09-13T09:13:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0935e-05 2024-09-13T09:13:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183944+0000", 2024-09-13T09:13:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9329999999999997e-05 2024-09-13T09:13:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.067 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17753 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2148352~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.067 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138442+0000", 2024-09-13T09:13:06.067 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.921416740000002, 2024-09-13T09:13:06.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045633606, 2024-09-13T09:13:06.067 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.067 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.067 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17753 2024-09-13T09:13:06.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.067 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138442+0000", 2024-09-13T09:13:06.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138442+0000", 2024-09-13T09:13:06.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138441+0000", 2024-09-13T09:13:06.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:13:06.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138444+0000", 2024-09-13T09:13:06.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0120000000000001e-06 2024-09-13T09:13:06.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138450+0000", 2024-09-13T09:13:06.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6180000000000003e-06 2024-09-13T09:13:06.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138458+0000", 2024-09-13T09:13:06.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9130000000000003e-06 2024-09-13T09:13:06.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183973+0000", 2024-09-13T09:13:06.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045514741999999997 2024-09-13T09:13:06.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184022+0000", 2024-09-13T09:13:06.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9521999999999998e-05 2024-09-13T09:13:06.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184076+0000", 2024-09-13T09:13:06.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3888000000000003e-05 2024-09-13T09:13:06.071 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.071 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.071 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.071 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17754 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2170880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.071 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:13:06.072 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.921387798000001, 2024-09-13T09:13:06.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045714312999999999, 2024-09-13T09:13:06.072 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.072 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.072 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17754 2024-09-13T09:13:06.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.072 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:13:06.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:13:06.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:13:06.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138474+0000", 2024-09-13T09:13:06.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8540000000000001e-06 2024-09-13T09:13:06.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138480+0000", 2024-09-13T09:13:06.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5139999999999998e-06 2024-09-13T09:13:06.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138485+0000", 2024-09-13T09:13:06.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.989e-06 2024-09-13T09:13:06.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184095+0000", 2024-09-13T09:13:06.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045609702000000002 2024-09-13T09:13:06.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184135+0000", 2024-09-13T09:13:06.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0510999999999999e-05 2024-09-13T09:13:06.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184186+0000", 2024-09-13T09:13:06.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0262000000000001e-05 2024-09-13T09:13:06.075 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.075 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.076 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.076 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17755 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2193408~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.076 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:13:06.076 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.921368610000002, 2024-09-13T09:13:06.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045899349999999998, 2024-09-13T09:13:06.076 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.076 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.077 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17755 2024-09-13T09:13:06.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.077 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:13:06.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:13:06.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:13:06.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138492+0000", 2024-09-13T09:13:06.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4310000000000001e-06 2024-09-13T09:13:06.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138498+0000", 2024-09-13T09:13:06.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7400000000000001e-06 2024-09-13T09:13:06.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138503+0000", 2024-09-13T09:13:06.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:13:06.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184222+0000", 2024-09-13T09:13:06.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045718996999999997 2024-09-13T09:13:06.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184306+0000", 2024-09-13T09:13:06.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4293000000000001e-05 2024-09-13T09:13:06.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184390+0000", 2024-09-13T09:13:06.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3718e-05 2024-09-13T09:13:06.080 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.080 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.080 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.080 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17756 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2215936~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.080 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138507+0000", 2024-09-13T09:13:06.080 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.921352063, 2024-09-13T09:13:06.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046004902, 2024-09-13T09:13:06.081 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.081 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.081 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.081 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.081 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.081 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17756 2024-09-13T09:13:06.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.081 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138507+0000", 2024-09-13T09:13:06.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138507+0000", 2024-09-13T09:13:06.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138506+0000", 2024-09-13T09:13:06.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138509+0000", 2024-09-13T09:13:06.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:13:06.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138514+0000", 2024-09-13T09:13:06.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.665e-06 2024-09-13T09:13:06.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138519+0000", 2024-09-13T09:13:06.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7530000000000001e-06 2024-09-13T09:13:06.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184415+0000", 2024-09-13T09:13:06.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045895575000000001 2024-09-13T09:13:06.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184459+0000", 2024-09-13T09:13:06.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4506999999999999e-05 2024-09-13T09:13:06.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184512+0000", 2024-09-13T09:13:06.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2685999999999997e-05 2024-09-13T09:13:06.084 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.084 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.084 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.085 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17766 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2441216~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.085 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140631+0000", 2024-09-13T09:13:06.085 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.919228505, 2024-09-13T09:13:06.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045443425000000003, 2024-09-13T09:13:06.085 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.085 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.085 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17766 2024-09-13T09:13:06.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.085 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140631+0000", 2024-09-13T09:13:06.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140631+0000", 2024-09-13T09:13:06.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140630+0000", 2024-09-13T09:13:06.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140632+0000", 2024-09-13T09:13:06.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1770000000000001e-06 2024-09-13T09:13:06.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140638+0000", 2024-09-13T09:13:06.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5550000000000002e-06 2024-09-13T09:13:06.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140643+0000", 2024-09-13T09:13:06.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5589999999999999e-06 2024-09-13T09:13:06.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.185929+0000", 2024-09-13T09:13:06.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045286356999999999 2024-09-13T09:13:06.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.185985+0000", 2024-09-13T09:13:06.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6042999999999999e-05 2024-09-13T09:13:06.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186074+0000", 2024-09-13T09:13:06.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9083000000000001e-05 2024-09-13T09:13:06.089 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.089 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.089 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.089 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17767 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2463744~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.089 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:13:06.089 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.919212712, 2024-09-13T09:13:06.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045589383999999997, 2024-09-13T09:13:06.089 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.089 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.090 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.090 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.090 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.090 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17767 2024-09-13T09:13:06.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.090 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:13:06.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:13:06.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:13:06.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140648+0000", 2024-09-13T09:13:06.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.001e-06 2024-09-13T09:13:06.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140654+0000", 2024-09-13T09:13:06.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6690000000000003e-06 2024-09-13T09:13:06.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140658+0000", 2024-09-13T09:13:06.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4220000000000002e-06 2024-09-13T09:13:06.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186115+0000", 2024-09-13T09:13:06.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045456435000000003 2024-09-13T09:13:06.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186172+0000", 2024-09-13T09:13:06.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7088e-05 2024-09-13T09:13:06.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186236+0000", 2024-09-13T09:13:06.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4082000000000001e-05 2024-09-13T09:13:06.093 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.093 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.093 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.094 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17768 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2486272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.094 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:13:06.094 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.919196745000001, 2024-09-13T09:13:06.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045865601999999998, 2024-09-13T09:13:06.094 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.094 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.094 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.094 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.094 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.094 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17768 2024-09-13T09:13:06.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.094 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:13:06.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:13:06.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:13:06.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140664+0000", 2024-09-13T09:13:06.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1449999999999998e-06 2024-09-13T09:13:06.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140670+0000", 2024-09-13T09:13:06.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9460000000000003e-06 2024-09-13T09:13:06.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140675+0000", 2024-09-13T09:13:06.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5770000000000004e-06 2024-09-13T09:13:06.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186267+0000", 2024-09-13T09:13:06.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045591957000000002 2024-09-13T09:13:06.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186322+0000", 2024-09-13T09:13:06.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5049000000000001e-05 2024-09-13T09:13:06.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186528+0000", 2024-09-13T09:13:06.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020626600000000001 2024-09-13T09:13:06.098 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.098 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.098 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.098 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17769 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2508800~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.098 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:13:06.098 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.919180684000001, 2024-09-13T09:13:06.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045981877999999997, 2024-09-13T09:13:06.098 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.098 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.099 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.099 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.099 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.099 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17769 2024-09-13T09:13:06.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.099 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:13:06.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:13:06.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:13:06.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140680+0000", 2024-09-13T09:13:06.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0209999999999998e-06 2024-09-13T09:13:06.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140686+0000", 2024-09-13T09:13:06.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2840000000000003e-06 2024-09-13T09:13:06.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140691+0000", 2024-09-13T09:13:06.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.369e-06 2024-09-13T09:13:06.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186562+0000", 2024-09-13T09:13:06.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045870834999999999 2024-09-13T09:13:06.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186606+0000", 2024-09-13T09:13:06.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4631e-05 2024-09-13T09:13:06.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186660+0000", 2024-09-13T09:13:06.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4042000000000002e-05 2024-09-13T09:13:06.102 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.102 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.102 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.103 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17885 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2217984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.103 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:13:06.103 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.917207585, 2024-09-13T09:13:06.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045427829000000003, 2024-09-13T09:13:06.103 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.103 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.103 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17885 2024-09-13T09:13:06.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.103 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:13:06.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:13:06.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:13:06.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142653+0000", 2024-09-13T09:13:06.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2500000000000001e-06 2024-09-13T09:13:06.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142659+0000", 2024-09-13T09:13:06.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.366e-06 2024-09-13T09:13:06.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142663+0000", 2024-09-13T09:13:06.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5190000000000003e-06 2024-09-13T09:13:06.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187980+0000", 2024-09-13T09:13:06.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045316233999999997 2024-09-13T09:13:06.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188032+0000", 2024-09-13T09:13:06.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1981000000000003e-05 2024-09-13T09:13:06.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188079+0000", 2024-09-13T09:13:06.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7803000000000003e-05 2024-09-13T09:13:06.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.107 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.107 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17886 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2240512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.107 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142686+0000", 2024-09-13T09:13:06.107 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.917173176999999, 2024-09-13T09:13:06.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045507535000000002, 2024-09-13T09:13:06.107 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.107 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.108 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17886 2024-09-13T09:13:06.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.108 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142686+0000", 2024-09-13T09:13:06.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142686+0000", 2024-09-13T09:13:06.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142685+0000", 2024-09-13T09:13:06.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142688+0000", 2024-09-13T09:13:06.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.83e-06 2024-09-13T09:13:06.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142694+0000", 2024-09-13T09:13:06.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2910000000000004e-06 2024-09-13T09:13:06.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142699+0000", 2024-09-13T09:13:06.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9599999999999999e-06 2024-09-13T09:13:06.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188102+0000", 2024-09-13T09:13:06.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045402459999999999 2024-09-13T09:13:06.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188143+0000", 2024-09-13T09:13:06.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1607000000000003e-05 2024-09-13T09:13:06.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188193+0000", 2024-09-13T09:13:06.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9947000000000001e-05 2024-09-13T09:13:06.111 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.111 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.111 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.112 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17887 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2263040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.112 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142704+0000", 2024-09-13T09:13:06.112 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.917155185999999, 2024-09-13T09:13:06.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045652102999999999, 2024-09-13T09:13:06.112 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.112 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.112 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.112 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.112 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.112 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17887 2024-09-13T09:13:06.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.112 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142704+0000", 2024-09-13T09:13:06.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142704+0000", 2024-09-13T09:13:06.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142703+0000", 2024-09-13T09:13:06.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142706+0000", 2024-09-13T09:13:06.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.351e-06 2024-09-13T09:13:06.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142712+0000", 2024-09-13T09:13:06.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8799999999999996e-06 2024-09-13T09:13:06.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142716+0000", 2024-09-13T09:13:06.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-13T09:13:06.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188230+0000", 2024-09-13T09:13:06.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045513801 2024-09-13T09:13:06.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188303+0000", 2024-09-13T09:13:06.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3032000000000001e-05 2024-09-13T09:13:06.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188356+0000", 2024-09-13T09:13:06.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2899000000000003e-05 2024-09-13T09:13:06.116 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.116 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.116 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.116 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17893 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2398208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.116 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:13:06.116 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.916103651, 2024-09-13T09:13:06.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045574960999999997, 2024-09-13T09:13:06.116 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.116 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.117 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.117 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17893 2024-09-13T09:13:06.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.117 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:13:06.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:13:06.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:13:06.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143757+0000", 2024-09-13T09:13:06.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2000000000000001e-06 2024-09-13T09:13:06.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143763+0000", 2024-09-13T09:13:06.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.553e-06 2024-09-13T09:13:06.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143768+0000", 2024-09-13T09:13:06.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0019999999999997e-06 2024-09-13T09:13:06.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189160+0000", 2024-09-13T09:13:06.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045392147000000001 2024-09-13T09:13:06.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189243+0000", 2024-09-13T09:13:06.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2594000000000002e-05 2024-09-13T09:13:06.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189330+0000", 2024-09-13T09:13:06.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7764999999999997e-05 2024-09-13T09:13:06.120 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.120 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.120 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.120 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17894 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2420736~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.121 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.143886+0000", 2024-09-13T09:13:06.121 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.915973168000001, 2024-09-13T09:13:06.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045562601000000001, 2024-09-13T09:13:06.121 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.121 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.121 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17894 2024-09-13T09:13:06.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.121 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143886+0000", 2024-09-13T09:13:06.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143886+0000", 2024-09-13T09:13:06.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143771+0000", 2024-09-13T09:13:06.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998856 2024-09-13T09:13:06.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143889+0000", 2024-09-13T09:13:06.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000117314 2024-09-13T09:13:06.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143896+0000", 2024-09-13T09:13:06.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5920000000000003e-06 2024-09-13T09:13:06.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143902+0000", 2024-09-13T09:13:06.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:13:06.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189359+0000", 2024-09-13T09:13:06.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045457414000000002 2024-09-13T09:13:06.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189399+0000", 2024-09-13T09:13:06.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9653999999999997e-05 2024-09-13T09:13:06.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189449+0000", 2024-09-13T09:13:06.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9456999999999998e-05 2024-09-13T09:13:06.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.125 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.125 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17895 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2443264~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.125 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.143990+0000", 2024-09-13T09:13:06.125 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.915869192999999, 2024-09-13T09:13:06.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045666642, 2024-09-13T09:13:06.125 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.125 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.125 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.125 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.126 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17895 2024-09-13T09:13:06.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.126 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143990+0000", 2024-09-13T09:13:06.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143990+0000", 2024-09-13T09:13:06.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143907+0000", 2024-09-13T09:13:06.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999166 2024-09-13T09:13:06.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143993+0000", 2024-09-13T09:13:06.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6049000000000002e-05 2024-09-13T09:13:06.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144000+0000", 2024-09-13T09:13:06.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6550000000000004e-06 2024-09-13T09:13:06.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144006+0000", 2024-09-13T09:13:06.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4450000000000004e-06 2024-09-13T09:13:06.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189483+0000", 2024-09-13T09:13:06.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045477228000000001 2024-09-13T09:13:06.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189567+0000", 2024-09-13T09:13:06.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3721999999999996e-05 2024-09-13T09:13:06.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189657+0000", 2024-09-13T09:13:06.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9764000000000003e-05 2024-09-13T09:13:06.129 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.129 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.129 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.129 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17896 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2465792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.129 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144260+0000", 2024-09-13T09:13:06.129 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.915599325999999, 2024-09-13T09:13:06.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045647222000000001, 2024-09-13T09:13:06.130 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.130 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.130 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.130 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.130 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.130 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17896 2024-09-13T09:13:06.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.130 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144260+0000", 2024-09-13T09:13:06.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144260+0000", 2024-09-13T09:13:06.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144010+0000", 2024-09-13T09:13:06.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9997506 2024-09-13T09:13:06.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144263+0000", 2024-09-13T09:13:06.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025252299999999999 2024-09-13T09:13:06.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144270+0000", 2024-09-13T09:13:06.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5259999999999996e-06 2024-09-13T09:13:06.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144276+0000", 2024-09-13T09:13:06.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:13:06.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189682+0000", 2024-09-13T09:13:06.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045406482999999997 2024-09-13T09:13:06.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189723+0000", 2024-09-13T09:13:06.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1059000000000001e-05 2024-09-13T09:13:06.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189907+0000", 2024-09-13T09:13:06.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018378099999999999 2024-09-13T09:13:06.133 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.133 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.133 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.134 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17897 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2488320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.134 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144430+0000", 2024-09-13T09:13:06.134 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.915429184000001, 2024-09-13T09:13:06.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045662734000000003, 2024-09-13T09:13:06.134 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.134 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.134 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.134 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.134 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.134 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17897 2024-09-13T09:13:06.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.134 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144430+0000", 2024-09-13T09:13:06.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144430+0000", 2024-09-13T09:13:06.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144281+0000", 2024-09-13T09:13:06.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998507 2024-09-13T09:13:06.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144433+0000", 2024-09-13T09:13:06.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00015261800000000001 2024-09-13T09:13:06.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144441+0000", 2024-09-13T09:13:06.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6790000000000005e-06 2024-09-13T09:13:06.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144446+0000", 2024-09-13T09:13:06.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5280000000000003e-06 2024-09-13T09:13:06.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189943+0000", 2024-09-13T09:13:06.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045496178999999998 2024-09-13T09:13:06.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190025+0000", 2024-09-13T09:13:06.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2325999999999997e-05 2024-09-13T09:13:06.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190093+0000", 2024-09-13T09:13:06.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7780000000000005e-05 2024-09-13T09:13:06.138 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.138 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.138 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.138 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17898 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2510848~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.138 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144655+0000", 2024-09-13T09:13:06.138 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.915204009, 2024-09-13T09:13:06.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045552875, 2024-09-13T09:13:06.138 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.138 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.138 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.138 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.139 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.139 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17898 2024-09-13T09:13:06.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.139 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144655+0000", 2024-09-13T09:13:06.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144655+0000", 2024-09-13T09:13:06.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144451+0000", 2024-09-13T09:13:06.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9997959 2024-09-13T09:13:06.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144658+0000", 2024-09-13T09:13:06.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000207188 2024-09-13T09:13:06.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144666+0000", 2024-09-13T09:13:06.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1380000000000007e-06 2024-09-13T09:13:06.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144672+0000", 2024-09-13T09:13:06.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7980000000000002e-06 2024-09-13T09:13:06.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190116+0000", 2024-09-13T09:13:06.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045443413000000002 2024-09-13T09:13:06.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190158+0000", 2024-09-13T09:13:06.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2055000000000003e-05 2024-09-13T09:13:06.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190208+0000", 2024-09-13T09:13:06.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0387999999999999e-05 2024-09-13T09:13:06.142 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.142 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.142 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.142 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17899 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2533376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.142 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144737+0000", 2024-09-13T09:13:06.142 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.915121866, 2024-09-13T09:13:06.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045677290000000002, 2024-09-13T09:13:06.143 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.143 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.143 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.143 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.143 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.143 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17899 2024-09-13T09:13:06.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.143 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144737+0000", 2024-09-13T09:13:06.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144737+0000", 2024-09-13T09:13:06.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144678+0000", 2024-09-13T09:13:06.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999409 2024-09-13T09:13:06.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144740+0000", 2024-09-13T09:13:06.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2116e-05 2024-09-13T09:13:06.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144748+0000", 2024-09-13T09:13:06.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2130000000000008e-06 2024-09-13T09:13:06.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144754+0000", 2024-09-13T09:13:06.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.66e-06 2024-09-13T09:13:06.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190238+0000", 2024-09-13T09:13:06.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045483824999999999 2024-09-13T09:13:06.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190322+0000", 2024-09-13T09:13:06.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4438999999999996e-05 2024-09-13T09:13:06.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190415+0000", 2024-09-13T09:13:06.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2227999999999999e-05 2024-09-13T09:13:06.146 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.146 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.146 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.147 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17900 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2555904~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:06.147 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144938+0000", 2024-09-13T09:13:06.147 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.914920965, 2024-09-13T09:13:06.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045612509000000002, 2024-09-13T09:13:06.147 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.147 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:06.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:06.147 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17900 2024-09-13T09:13:06.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.147 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144938+0000", 2024-09-13T09:13:06.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144938+0000", 2024-09-13T09:13:06.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144758+0000", 2024-09-13T09:13:06.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998202 2024-09-13T09:13:06.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144941+0000", 2024-09-13T09:13:06.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000182688 2024-09-13T09:13:06.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144949+0000", 2024-09-13T09:13:06.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7300000000000005e-06 2024-09-13T09:13:06.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144954+0000", 2024-09-13T09:13:06.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4090000000000002e-06 2024-09-13T09:13:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190448+0000", 2024-09-13T09:13:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045493901000000003 2024-09-13T09:13:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190499+0000", 2024-09-13T09:13:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0677999999999998e-05 2024-09-13T09:13:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190551+0000", 2024-09-13T09:13:06.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1820999999999997e-05 2024-09-13T09:13:06.151 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.151 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.151 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.151 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.151 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.151 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:06.152 INFO:teuthology.orchestra.run.smithi157.stdout:3135 trunc from 0x1a84800 to 0x1980400 2024-09-13T09:13:06.153 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:06.153 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:06.151+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:06.158 INFO:teuthology.orchestra.run.smithi157.stdout:3136 trunc from 0x1980400 to 0x9fc1000 2024-09-13T09:13:06.172 INFO:teuthology.orchestra.run.smithi157.stdout:3138 read 0x634f800 thru 0x635b230 (0xba31 bytes) 2024-09-13T09:13:06.175 INFO:teuthology.orchestra.run.smithi157.stdout:3139 trunc from 0x9fc1000 to 0x79b1800 2024-09-13T09:13:06.191 INFO:teuthology.orchestra.run.smithi157.stdout:3140 trunc from 0x79b1800 to 0x69a9e00 2024-09-13T09:13:06.208 INFO:teuthology.orchestra.run.smithi157.stdout:3141 punch from 0xbc9600 to 0xbca200, (0xc00 bytes) 2024-09-13T09:13:06.208 INFO:teuthology.orchestra.run.smithi157.stdout:3145 punch from 0x8ccc00 to 0x8db400, (0xe800 bytes) 2024-09-13T09:13:06.214 INFO:teuthology.orchestra.run.smithi157.stdout:3147 trunc from 0x69a9e00 to 0xc8f2c00 2024-09-13T09:13:06.229 INFO:teuthology.orchestra.run.smithi157.stdout:3152 punch from 0xbce7000 to 0xbce9400, (0x2400 bytes) 2024-09-13T09:13:06.229 INFO:teuthology.orchestra.run.smithi157.stdout:3153 write 0xb340e00 thru 0xb34103a (0x23b bytes) 2024-09-13T09:13:06.229 INFO:teuthology.orchestra.run.smithi157.stdout:3154 trunc from 0xc8f2c00 to 0xd00b600 2024-09-13T09:13:06.246 INFO:teuthology.orchestra.run.smithi157.stdout:3155 read 0x13d4a00 thru 0x13e3900 (0xef01 bytes) 2024-09-13T09:13:06.247 INFO:teuthology.orchestra.run.smithi157.stdout:3157 read 0xbd66400 thru 0xbd6df5c (0x7b5d bytes) 2024-09-13T09:13:06.247 INFO:teuthology.orchestra.run.smithi157.stdout:3162 punch from 0x5a2e00 to 0x5afa00, (0xcc00 bytes) 2024-09-13T09:13:06.247 INFO:teuthology.orchestra.run.smithi157.stdout:3163 read 0x952d000 thru 0x95366cf (0x96d0 bytes) 2024-09-13T09:13:06.253 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:06.252+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:06.256 INFO:teuthology.orchestra.run.smithi157.stdout:3164 trunc from 0xd00b600 to 0x62a9200 2024-09-13T09:13:06.278 INFO:teuthology.orchestra.run.smithi157.stdout:3165 read 0x2137a00 thru 0x2138eeb (0x14ec bytes) 2024-09-13T09:13:06.279 INFO:teuthology.orchestra.run.smithi157.stdout:3167 read 0x1459600 thru 0x1460951 (0x7352 bytes) 2024-09-13T09:13:06.280 INFO:teuthology.orchestra.run.smithi157.stdout:3172 read 0x1a04a00 thru 0x1a05583 (0xb84 bytes) 2024-09-13T09:13:06.280 INFO:teuthology.orchestra.run.smithi157.stdout:3174 write 0x4444c00 thru 0x445329a (0xe69b bytes) 2024-09-13T09:13:06.280 INFO:teuthology.orchestra.run.smithi157.stdout:3175 punch from 0x4ab5200 to 0x4abe400, (0x9200 bytes) 2024-09-13T09:13:06.299 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:06.300 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:06.300 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:06.300 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:06.305 INFO:teuthology.orchestra.run.smithi157.stdout:3176 write 0xe863800 thru 0xe86a3ab (0x6bac bytes) 2024-09-13T09:13:06.306 INFO:teuthology.orchestra.run.smithi157.stdout:3180 read 0x727c600 thru 0x7282f90 (0x6991 bytes) 2024-09-13T09:13:06.307 INFO:teuthology.orchestra.run.smithi157.stdout:3181 punch from 0xda14800 to 0xda22600, (0xde00 bytes) 2024-09-13T09:13:06.307 INFO:teuthology.orchestra.run.smithi157.stdout:3182 write 0xd35d800 thru 0xd36032b (0x2b2c bytes) 2024-09-13T09:13:06.307 INFO:teuthology.orchestra.run.smithi157.stdout:3184 read 0xca4b200 thru 0xca54133 (0x8f34 bytes) 2024-09-13T09:13:06.307 INFO:teuthology.orchestra.run.smithi157.stdout:3185 write 0x1651400 thru 0x165d5d3 (0xc1d4 bytes) 2024-09-13T09:13:06.309 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:06.309 INFO:teuthology.orchestra.run.smithi157.stdout:3189 read 0x711b600 thru 0x7129789 (0xe18a bytes) 2024-09-13T09:13:06.322 INFO:teuthology.orchestra.run.smithi157.stdout:3191 trunc from 0xe86a400 to 0x38ecc00 2024-09-13T09:13:06.351 INFO:teuthology.orchestra.run.smithi157.stdout:3193 trunc from 0x38ecc00 to 0x2732800 2024-09-13T09:13:06.354 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:06.352+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:06.368 INFO:teuthology.orchestra.run.smithi157.stdout:3197 read 0x817200 thru 0x823c17 (0xca18 bytes) 2024-09-13T09:13:06.384 INFO:teuthology.orchestra.run.smithi157.stdout:3200 write 0x6009600 thru 0x60182ea (0xeceb bytes) 2024-09-13T09:13:06.384 INFO:teuthology.orchestra.run.smithi157.stdout:3202 trunc from 0x6018400 to 0x651cc00 2024-09-13T09:13:06.415 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:06.416 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:06.416 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:06.419 INFO:teuthology.orchestra.run.smithi157.stdout:3203 write 0x83b6600 thru 0x83c2633 (0xc034 bytes) 2024-09-13T09:13:06.424 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:06.428 INFO:teuthology.orchestra.run.smithi157.stdout:3206 trunc from 0x83c2800 to 0x1546000 2024-09-13T09:13:06.465 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:06.464+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:06.474 INFO:teuthology.orchestra.run.smithi157.stdout:3210 write 0x1e57000 thru 0x1e5ced4 (0x5ed5 bytes) 2024-09-13T09:13:06.499 INFO:teuthology.orchestra.run.smithi157.stdout:3212 write 0xe9f6400 thru 0xe9fcfd4 (0x6bd5 bytes) 2024-09-13T09:13:06.517 INFO:teuthology.orchestra.run.smithi157.stdout:3214 trunc from 0xe9fd000 to 0xe33400 2024-09-13T09:13:06.539 INFO:teuthology.orchestra.run.smithi157.stdout:3215 read 0x607400 thru 0x607948 (0x549 bytes) 2024-09-13T09:13:06.540 INFO:teuthology.orchestra.run.smithi157.stdout:3216 read 0xe10000 thru 0xe11acf (0x1ad0 bytes) 2024-09-13T09:13:06.540 INFO:teuthology.orchestra.run.smithi157.stdout:3219 write 0xdc7c00 thru 0xdd16d1 (0x9ad2 bytes) 2024-09-13T09:13:06.543 INFO:teuthology.orchestra.run.smithi157.stdout:3221 read 0x132a00 thru 0x132c4e (0x24f bytes) 2024-09-13T09:13:06.543 INFO:teuthology.orchestra.run.smithi157.stdout:3222 read 0xd06600 thru 0xd0ccf0 (0x66f1 bytes) 2024-09-13T09:13:06.543 INFO:teuthology.orchestra.run.smithi157.stdout:3223 read 0xd52e00 thru 0xd5ed6a (0xbf6b bytes) 2024-09-13T09:13:06.552 INFO:teuthology.orchestra.run.smithi157.stdout:3225 write 0x1cc8200 thru 0x1ccbeae (0x3caf bytes) 2024-09-13T09:13:06.555 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:06.554+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:06.557 INFO:teuthology.orchestra.run.smithi157.stdout:3232 read 0xd5d400 thru 0xd5e220 (0xe21 bytes) 2024-09-13T09:13:06.574 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:06.574 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:06.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:06.574 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:06.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.574 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9035 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2342912~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.574 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:06.575 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8268276730000004, 2024-09-13T09:13:06.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070293606999999994, 2024-09-13T09:13:06.575 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.575 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.575 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.575 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.575 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.575 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9035 2024-09-13T09:13:06.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.575 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:06.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:06.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:06.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745602+0000", 2024-09-13T09:13:06.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-06 2024-09-13T09:13:06.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745607+0000", 2024-09-13T09:13:06.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0770000000000001e-06 2024-09-13T09:13:06.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745609+0000", 2024-09-13T09:13:06.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3499999999999999e-06 2024-09-13T09:13:06.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815626+0000", 2024-09-13T09:13:06.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070017398999999994 2024-09-13T09:13:06.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815672+0000", 2024-09-13T09:13:06.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5275999999999998e-05 2024-09-13T09:13:06.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815895+0000", 2024-09-13T09:13:06.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022328300000000001 2024-09-13T09:13:06.579 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.579 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.579 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.579 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.579 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:06.579 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8268151289999999, 2024-09-13T09:13:06.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:13:06.579 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.580 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.580 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:13:06.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.580 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:06.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:06.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:06.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:13:06.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:06.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:13:06.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:13:06.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:13:06.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:13:06.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:13:06.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:13:06.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:13:06.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:13:06.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:13:06.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:13:06.583 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.583 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.584 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.584 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.584 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:06.584 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8268064509999995, 2024-09-13T09:13:06.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:13:06.584 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.584 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.584 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.584 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.584 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.584 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:13:06.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.585 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:06.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:06.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:06.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:06.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:13:06.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:13:06.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:13:06.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:13:06.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:13:06.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:13:06.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:13:06.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:13:06.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:13:06.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:13:06.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:13:06.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:13:06.588 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.588 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.588 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.588 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.588 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:06.588 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8267974650000003, 2024-09-13T09:13:06.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:13:06.589 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.589 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.589 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:13:06.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.589 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:06.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:06.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:06.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:13:06.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:13:06.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:13:06.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:13:06.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:13:06.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:13:06.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:13:06.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:13:06.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:13:06.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:13:06.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:13:06.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:13:06.596 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.596 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.596 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.596 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.596 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:06.597 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8267894929999997, 2024-09-13T09:13:06.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:13:06.597 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.597 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.597 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.597 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.597 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.597 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:13:06.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.597 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:06.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:06.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:06.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:06.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:13:06.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:13:06.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:13:06.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:13:06.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:13:06.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:13:06.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:13:06.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:13:06.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:13:06.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:13:06.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:13:06.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:13:06.601 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.601 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.601 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.602 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.602 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:06.602 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8249146590000009, 2024-09-13T09:13:06.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:13:06.602 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.602 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.602 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.602 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.602 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.603 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:13:06.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.603 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:06.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:06.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:06.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:13:06.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:13:06.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:13:06.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:13:06.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:13:06.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:13:06.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:13:06.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:13:06.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:13:06.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:13:06.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:13:06.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:13:06.607 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.607 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.607 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.607 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.607 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:06.607 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8249039979999999, 2024-09-13T09:13:06.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:13:06.607 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.607 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.607 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.608 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.608 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.608 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:13:06.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.608 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:06.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:06.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:06.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:06.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:13:06.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:13:06.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:13:06.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:13:06.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:13:06.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:13:06.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:13:06.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:13:06.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:13:06.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:13:06.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:13:06.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:13:06.612 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.612 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.612 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.612 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.612 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:06.612 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8248820220000006, 2024-09-13T09:13:06.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:13:06.613 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.613 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.613 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.613 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.613 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.613 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:13:06.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.613 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:06.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:06.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:06.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:13:06.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:13:06.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:13:06.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:13:06.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:13:06.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:13:06.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:13:06.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:13:06.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:13:06.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:13:06.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:13:06.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:13:06.617 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.617 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.617 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.617 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.618 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:06.618 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8248685200000008, 2024-09-13T09:13:06.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:13:06.618 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.618 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.618 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.618 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.618 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.618 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:13:06.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.618 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:06.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:06.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:06.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:13:06.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:13:06.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:13:06.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:13:06.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:13:06.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:13:06.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:13:06.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:13:06.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:13:06.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:13:06.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:13:06.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:13:06.622 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.622 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.622 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.623 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9055 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 6930432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.623 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:06.623 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8228757400000006, 2024-09-13T09:13:06.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070177949000000003, 2024-09-13T09:13:06.623 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.623 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.623 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.623 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.623 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.624 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9055 2024-09-13T09:13:06.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.624 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:06.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:06.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:06.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:06.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749554+0000", 2024-09-13T09:13:06.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.11e-06 2024-09-13T09:13:06.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749557+0000", 2024-09-13T09:13:06.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7949999999999999e-06 2024-09-13T09:13:06.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749559+0000", 2024-09-13T09:13:06.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:13:06.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819633+0000", 2024-09-13T09:13:06.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073995 2024-09-13T09:13:06.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819679+0000", 2024-09-13T09:13:06.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6006e-05 2024-09-13T09:13:06.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819731+0000", 2024-09-13T09:13:06.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1934000000000001e-05 2024-09-13T09:13:06.628 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.628 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.628 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.628 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9056 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 7159808~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.628 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:06.628 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8228676910000008, 2024-09-13T09:13:06.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070283126000000001, 2024-09-13T09:13:06.629 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.629 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.629 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.629 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.629 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.629 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9056 2024-09-13T09:13:06.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.629 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:06.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:06.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:06.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:06.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749562+0000", 2024-09-13T09:13:06.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1340000000000001e-06 2024-09-13T09:13:06.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749565+0000", 2024-09-13T09:13:06.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8760000000000001e-06 2024-09-13T09:13:06.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749567+0000", 2024-09-13T09:13:06.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2340000000000001e-06 2024-09-13T09:13:06.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819750+0000", 2024-09-13T09:13:06.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070182537000000003 2024-09-13T09:13:06.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819792+0000", 2024-09-13T09:13:06.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2141999999999997e-05 2024-09-13T09:13:06.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819844+0000", 2024-09-13T09:13:06.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2428e-05 2024-09-13T09:13:06.633 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.633 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.633 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.634 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.634 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:06.634 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8228594220000005, 2024-09-13T09:13:06.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:13:06.634 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.634 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.634 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.634 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.634 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.634 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:13:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:13:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:13:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:13:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:13:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:13:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:13:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:13:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:13:06.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:13:06.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:13:06.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:13:06.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:13:06.638 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.639 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.639 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.639 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.639 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:06.639 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8228351000000007, 2024-09-13T09:13:06.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:13:06.639 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.639 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.639 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.639 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.640 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.640 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:13:06.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.640 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:06.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:06.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:06.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:13:06.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:13:06.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:13:06.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:13:06.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:13:06.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:13:06.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:13:06.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:13:06.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:13:06.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:13:06.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:13:06.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:13:06.644 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.644 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.644 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.644 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.644 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:06.644 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8228241260000004, 2024-09-13T09:13:06.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:13:06.645 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.645 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.645 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.645 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.645 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.645 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:13:06.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.645 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:06.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:06.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:13:06.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:13:06.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:13:06.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:13:06.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:13:06.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:13:06.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:13:06.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:13:06.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:13:06.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:13:06.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:13:06.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:13:06.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:13:06.649 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.649 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.649 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.650 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9063 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1589248~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.650 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:06.650 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8209204840000002, 2024-09-13T09:13:06.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070391924999999994, 2024-09-13T09:13:06.650 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.650 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.650 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.650 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.650 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.650 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9063 2024-09-13T09:13:06.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.651 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:06.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:06.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:06.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:06.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751509+0000", 2024-09-13T09:13:06.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.142e-06 2024-09-13T09:13:06.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751513+0000", 2024-09-13T09:13:06.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6669999999999998e-06 2024-09-13T09:13:06.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751520+0000", 2024-09-13T09:13:06.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5910000000000001e-06 2024-09-13T09:13:06.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821593+0000", 2024-09-13T09:13:06.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073519000000001 2024-09-13T09:13:06.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821648+0000", 2024-09-13T09:13:06.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:13:06.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821900+0000", 2024-09-13T09:13:06.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025263999999999999 2024-09-13T09:13:06.654 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.655 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.655 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.655 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.655 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:06.655 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8209065009999996, 2024-09-13T09:13:06.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:13:06.655 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.655 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.655 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.656 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:13:06.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.656 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:06.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:06.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:06.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:06.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:13:06.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:13:06.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:13:06.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:06.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:13:06.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:13:06.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:13:06.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:13:06.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:13:06.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:13:06.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:13:06.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:13:06.660 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.660 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.660 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.660 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.660 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:06.660 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8208971040000002, 2024-09-13T09:13:06.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:13:06.661 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.661 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.661 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:13:06.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.661 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:06.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:06.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:06.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:06.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:13:06.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:13:06.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:13:06.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:13:06.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:13:06.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:13:06.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:13:06.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:13:06.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:13:06.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:13:06.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:13:06.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:13:06.665 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.666 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.666 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.666 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9069 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2965504~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.666 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:06.666 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8189377249999996, 2024-09-13T09:13:06.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070184959000000005, 2024-09-13T09:13:06.667 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.667 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.667 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9069 2024-09-13T09:13:06.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.667 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:06.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:06.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:06.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:06.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753492+0000", 2024-09-13T09:13:06.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.403e-06 2024-09-13T09:13:06.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753495+0000", 2024-09-13T09:13:06.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9390000000000002e-06 2024-09-13T09:13:06.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753504+0000", 2024-09-13T09:13:06.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5129999999999996e-06 2024-09-13T09:13:06.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823390+0000", 2024-09-13T09:13:06.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.069886632000000004 2024-09-13T09:13:06.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823445+0000", 2024-09-13T09:13:06.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4880999999999999e-05 2024-09-13T09:13:06.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823676+0000", 2024-09-13T09:13:06.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000230812 2024-09-13T09:13:06.671 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.671 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.671 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.671 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9070 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3194880~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.671 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:06.672 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8189226339999998, 2024-09-13T09:13:06.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070280588000000005, 2024-09-13T09:13:06.672 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.672 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.672 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.672 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.672 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.672 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9070 2024-09-13T09:13:06.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.672 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:06.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:06.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:06.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:06.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753507+0000", 2024-09-13T09:13:06.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:06.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753510+0000", 2024-09-13T09:13:06.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0460000000000001e-06 2024-09-13T09:13:06.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753513+0000", 2024-09-13T09:13:06.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:13:06.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823689+0000", 2024-09-13T09:13:06.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070176083 2024-09-13T09:13:06.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823734+0000", 2024-09-13T09:13:06.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5056999999999999e-05 2024-09-13T09:13:06.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823787+0000", 2024-09-13T09:13:06.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3121000000000002e-05 2024-09-13T09:13:06.676 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.676 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.676 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.677 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9071 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:06.677 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:06.677 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8189140869999996, 2024-09-13T09:13:06.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070386377999999999, 2024-09-13T09:13:06.677 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:06.677 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:06.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:06.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:06.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:06.677 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9071 2024-09-13T09:13:06.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.678 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:06.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:06.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:06.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:06.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:06.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:06.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:06.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:06.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:06.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:06.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753516+0000", 2024-09-13T09:13:06.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1519999999999999e-06 2024-09-13T09:13:06.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:06.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753519+0000", 2024-09-13T09:13:06.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6559999999999998e-06 2024-09-13T09:13:06.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:06.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753522+0000", 2024-09-13T09:13:06.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:13:06.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:06.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823798+0000", 2024-09-13T09:13:06.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070275907999999998 2024-09-13T09:13:06.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:06.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823845+0000", 2024-09-13T09:13:06.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7593000000000003e-05 2024-09-13T09:13:06.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:06.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:06.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:06.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823901+0000", 2024-09-13T09:13:06.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5934999999999999e-05 2024-09-13T09:13:06.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.682 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.682 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.682 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:06.682 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:06.682 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:06.684 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:06.682+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:06.685 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:06.687 INFO:teuthology.orchestra.run.smithi157.stdout:3233 write 0xbb6aa00 thru 0xbb7766a (0xcc6b bytes) 2024-09-13T09:13:06.687 INFO:teuthology.orchestra.run.smithi157.stdout:3235 write 0x845c600 thru 0x845ffb7 (0x39b8 bytes) 2024-09-13T09:13:06.687 INFO:teuthology.orchestra.run.smithi157.stdout:3239 write 0x5d22400 thru 0x5d2add8 (0x89d9 bytes) 2024-09-13T09:13:06.687 INFO:teuthology.orchestra.run.smithi157.stdout:3240 read 0x1f28800 thru 0x1f37b7f (0xf380 bytes) 2024-09-13T09:13:06.687 INFO:teuthology.orchestra.run.smithi157.stdout:3241 write 0xaaad400 thru 0xaab4b65 (0x7766 bytes) 2024-09-13T09:13:06.687 INFO:teuthology.orchestra.run.smithi157.stdout:3242 read 0xa924200 thru 0xa9252ef (0x10f0 bytes) 2024-09-13T09:13:06.688 INFO:teuthology.orchestra.run.smithi157.stdout:3244 read 0x2aac00 thru 0x2b3f2d (0x932e bytes) 2024-09-13T09:13:06.688 INFO:teuthology.orchestra.run.smithi157.stdout:3245 write 0x698a200 thru 0x69990ca (0xeecb bytes) 2024-09-13T09:13:06.688 INFO:teuthology.orchestra.run.smithi157.stdout:3246 punch from 0x60fbc00 to 0x6104400, (0x8800 bytes) 2024-09-13T09:13:06.688 INFO:teuthology.orchestra.run.smithi157.stdout:3247 punch from 0x255da00 to 0x2566e00, (0x9400 bytes) 2024-09-13T09:13:06.688 INFO:teuthology.orchestra.run.smithi157.stdout:3249 write 0xaa2a200 thru 0xaa33d47 (0x9b48 bytes) 2024-09-13T09:13:06.688 INFO:teuthology.orchestra.run.smithi157.stdout:3253 write 0x5f36e00 thru 0x5f3c8eb (0x5aec bytes) 2024-09-13T09:13:06.688 INFO:teuthology.orchestra.run.smithi157.stdout:3254 read 0x26dfc00 thru 0x26e1e4c (0x224d bytes) 2024-09-13T09:13:06.688 INFO:teuthology.orchestra.run.smithi157.stdout:3258 read 0xa1b8000 thru 0xa1c0126 (0x8127 bytes) 2024-09-13T09:13:06.688 INFO:teuthology.orchestra.run.smithi157.stdout:3259 trunc from 0xbb77800 to 0x15f3400 2024-09-13T09:13:06.688 INFO:teuthology.orchestra.run.smithi157.stdout:3262 trunc from 0x15f3400 to 0xa620a00 2024-09-13T09:13:06.688 INFO:teuthology.orchestra.run.smithi157.stdout:3263 trunc from 0xa620a00 to 0x5534400 2024-09-13T09:13:06.689 INFO:teuthology.orchestra.run.smithi157.stdout:3270 punch from 0xe98600 to 0xea3e00, (0xb800 bytes) 2024-09-13T09:13:06.689 INFO:teuthology.orchestra.run.smithi157.stdout:3272 read 0x176b400 thru 0x1774b6a (0x976b bytes) 2024-09-13T09:13:06.689 INFO:teuthology.orchestra.run.smithi157.stdout:3275 read 0x363d000 thru 0x364b73b (0xe73c bytes) 2024-09-13T09:13:06.689 INFO:teuthology.orchestra.run.smithi157.stdout:3277 read 0x2f8a200 thru 0x2f8eed0 (0x4cd1 bytes) 2024-09-13T09:13:06.689 INFO:teuthology.orchestra.run.smithi157.stdout:3278 punch from 0x38cc600 to 0x38da400, (0xde00 bytes) 2024-09-13T09:13:06.689 INFO:teuthology.orchestra.run.smithi157.stdout:3279 write 0xa5b7c00 thru 0xa5b8070 (0x471 bytes) 2024-09-13T09:13:06.689 INFO:teuthology.orchestra.run.smithi157.stdout:3280 punch from 0xfd2400 to 0xfd8600, (0x6200 bytes) 2024-09-13T09:13:06.689 INFO:teuthology.orchestra.run.smithi157.stdout:3281 write 0x3225200 thru 0x322e992 (0x9793 bytes) 2024-09-13T09:13:06.689 INFO:teuthology.orchestra.run.smithi157.stdout:3283 write 0x2cd6e00 thru 0x2ce55bc (0xe7bd bytes) 2024-09-13T09:13:06.689 INFO:teuthology.orchestra.run.smithi157.stdout:3286 punch from 0x6bce600 to 0x6bdae00, (0xc800 bytes) 2024-09-13T09:13:06.702 INFO:teuthology.orchestra.run.smithi157.stdout:3287 write 0xed5f400 thru 0xed6dac3 (0xe6c4 bytes) 2024-09-13T09:13:06.709 INFO:teuthology.orchestra.run.smithi157.stdout:3288 write 0xc8bd600 thru 0xc8c96da (0xc0db bytes) 2024-09-13T09:13:06.709 INFO:teuthology.orchestra.run.smithi157.stdout:3289 write 0xb2bf400 thru 0xb2c42e0 (0x4ee1 bytes) 2024-09-13T09:13:06.716 INFO:teuthology.orchestra.run.smithi157.stdout:3291 trunc from 0xed6dc00 to 0x8b22a00 2024-09-13T09:13:06.737 INFO:teuthology.orchestra.run.smithi157.stdout:3292 read 0x149e200 thru 0x149f509 (0x130a bytes) 2024-09-13T09:13:06.752 INFO:teuthology.orchestra.run.smithi157.stdout:3293 write 0xbfbfc00 thru 0xbfbfe1b (0x21c bytes) 2024-09-13T09:13:06.756 INFO:teuthology.orchestra.run.smithi157.stdout:3294 write 0x656cc00 thru 0x657ab75 (0xdf76 bytes) 2024-09-13T09:13:06.766 INFO:teuthology.orchestra.run.smithi157.stdout:3295 write 0xd4d1400 thru 0xd4d4535 (0x3136 bytes) 2024-09-13T09:13:06.767 INFO:teuthology.orchestra.run.smithi157.stdout:3296 write 0x144a00 thru 0x152c78 (0xe279 bytes) 2024-09-13T09:13:06.767 INFO:teuthology.orchestra.run.smithi157.stdout:3299 clone 22 order 22 su 262144 sc 9 2024-09-13T09:13:06.784 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:06.782+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:06.803 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:06.804 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:06.804 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:06.804 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:06.812 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:06.885 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:06.884+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:06.918 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:06.918 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:06.918 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:06.918 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:06.918 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:06.927 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:06.986 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:06.984+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:07.036 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.036 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.504805272999999, 2024-09-13T09:13:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:13:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:13:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:13:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:13:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:13:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:13:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:13:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:13:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:13:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:13:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:13:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:13:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:13:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:13:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:13:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:13:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:13:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:13:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:13:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:13:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:13:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:13:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:13:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.042 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.504787237999999, 2024-09-13T09:13:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:13:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:13:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:13:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:13:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:13:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:13:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:13:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:13:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:13:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:13:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:13:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:13:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:13:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:13:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:13:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:13:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:13:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:13:07.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:13:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:13:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:13:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:13:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.504770568000001, 2024-09-13T09:13:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:13:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:13:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:13:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:13:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:13:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:13:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:13:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:13:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:13:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:13:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:13:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:13:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:13:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:13:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:13:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:13:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:13:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:13:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:13:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:13:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:13:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:13:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.057 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.504752935999999, 2024-09-13T09:13:07.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:13:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.059 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:13:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:07.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:13:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:13:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:13:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:13:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:13:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:13:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:13:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:13:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:13:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:13:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:13:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:13:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:13:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:13:07.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:13:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:13:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:13:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:13:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:13:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:13:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.504706631000001, 2024-09-13T09:13:07.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:13:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:13:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:13:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:13:07.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:13:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:13:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:13:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:13:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:13:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:13:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:13:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:13:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:13:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:13:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:13:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.070 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:13:07.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:13:07.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:13:07.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:13:07.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:13:07.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:13:07.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:13:07.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:13:07.072 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.072 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.073 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.073 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.073 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:07.073 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.50468133, 2024-09-13T09:13:07.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:13:07.073 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.074 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.074 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.074 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.074 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.074 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:13:07.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.074 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.074 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.075 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:07.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.075 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.075 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:07.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.076 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.076 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:13:07.076 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.076 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.076 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:13:07.076 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:13:07.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:13:07.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:13:07.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:13:07.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:13:07.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:13:07.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:13:07.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:13:07.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:13:07.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.080 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.080 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:13:07.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:13:07.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.080 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.080 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:13:07.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:13:07.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.081 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.081 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:13:07.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:13:07.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.081 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.081 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:13:07.082 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:13:07.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.082 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.082 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:13:07.082 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:13:07.082 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.082 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.082 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.083 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.083 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:07.083 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.504663100999998, 2024-09-13T09:13:07.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:13:07.083 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.083 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.084 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.084 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.084 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.084 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:13:07.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.084 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:07.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:07.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:13:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:13:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:13:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:13:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:13:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:13:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:13:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:13:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:13:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:13:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:13:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:13:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:13:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:13:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:13:07.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:13:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:13:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:13:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:13:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:13:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.504615781999998, 2024-09-13T09:13:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:13:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:13:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.090 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:07.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:13:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:13:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:13:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:13:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.092 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:13:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:13:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:13:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:13:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.093 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:13:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:13:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:13:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:13:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:13:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:13:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:13:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:13:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:13:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:13:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:13:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:13:07.095 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.504101544000001, 2024-09-13T09:13:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:13:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.096 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:13:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:13:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:13:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:13:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:13:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:13:07.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:13:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:13:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:13:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:13:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:13:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.099 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:13:07.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:13:07.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.100 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:13:07.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:13:07.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:13:07.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:13:07.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:13:07.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:13:07.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:13:07.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:13:07.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.103 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:13:07.104 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:13:07.104 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.104 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.104 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.105 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.105 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:07.105 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.504067014, 2024-09-13T09:13:07.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:13:07.105 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.105 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.106 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.106 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.106 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.106 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:13:07.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.106 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:07.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:07.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:07.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.109 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:13:07.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:13:07.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.110 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.110 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:13:07.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:13:07.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:13:07.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:13:07.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.112 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:13:07.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:13:07.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:13:07.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:13:07.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:13:07.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:13:07.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:13:07.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:13:07.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:13:07.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:13:07.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.116 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.116 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:13:07.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:13:07.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.117 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.117 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:13:07.117 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:13:07.117 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.117 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.117 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.118 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.118 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:07.118 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.504049107, 2024-09-13T09:13:07.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:13:07.118 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:13:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:07.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:13:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:13:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:13:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:13:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:13:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:13:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:13:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:13:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:13:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:13:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:13:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:13:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:13:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:13:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:13:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:13:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:13:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:13:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:13:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:13:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.494922258999999, 2024-09-13T09:13:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:13:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.124 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:13:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:07.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:13:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:13:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:13:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:13:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:13:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:13:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:13:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:13:07.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:13:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:13:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:13:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:13:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:13:07.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:13:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:13:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:13:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:13:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:13:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:13:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:13:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.494870679000002, 2024-09-13T09:13:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:13:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.130 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:13:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:13:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:13:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:13:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:13:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:13:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:13:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:13:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:13:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:13:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:13:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:13:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:13:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:13:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:13:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:13:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:13:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:13:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:13:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:13:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:13:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:13:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.135 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.494854403000001, 2024-09-13T09:13:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:13:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:13:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.136 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:13:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:13:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:13:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:13:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:13:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:13:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:13:07.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:13:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:13:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:13:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:13:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:13:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:13:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:13:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:13:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:13:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:13:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:13:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:13:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:13:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:13:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.494838629, 2024-09-13T09:13:07.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:13:07.142 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.142 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.142 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.142 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.142 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.142 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:13:07.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:13:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:13:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:13:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:13:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:13:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:13:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:13:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:13:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:13:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:13:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:13:07.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:13:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:13:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:13:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:13:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:13:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:13:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:13:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:13:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:13:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:13:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.147 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.494823753999999, 2024-09-13T09:13:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:13:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.148 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:13:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:13:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:13:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:13:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:13:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:13:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:13:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:13:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:13:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:13:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:13:07.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:13:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:13:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:13:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:13:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:13:07.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:13:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:13:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:13:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:13:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:13:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.494806292, 2024-09-13T09:13:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:13:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:13:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:13:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:13:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:13:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:13:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:13:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:13:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:13:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:13:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:13:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:13:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:13:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:13:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:13:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:13:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:13:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:13:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:13:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:13:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:13:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:13:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:13:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:07.159 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.493205660000001, 2024-09-13T09:13:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:13:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:13:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:13:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:13:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:13:07.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:13:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:13:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:13:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:13:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:13:07.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:13:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:13:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:13:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:13:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:13:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:13:07.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:13:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:13:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:13:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:13:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:13:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:13:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:13:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.493165608999998, 2024-09-13T09:13:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:13:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.165 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:13:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:13:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:13:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:13:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:13:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:13:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:13:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:13:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:13:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:13:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:13:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:13:07.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:13:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:13:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:13:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:13:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:13:07.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:13:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:13:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:13:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:13:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:13:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 24.493121846000001, 2024-09-13T09:13:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:13:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:13:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:07.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:07.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:13:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:13:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:13:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:13:07.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:13:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:13:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:13:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:13:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:13:07.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:13:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:13:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:13:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:13:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:13:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:13:07.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:13:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:13:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:13:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:13:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:13:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.177 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.177 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:07.178 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:07.178 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:07.177+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:07.279 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:07.278+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:07.325 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:07.325 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:07.325 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:07.325 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:07.334 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:07.366 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-13T09:13:07.366 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:13:07.366 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T09:13:07.367 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:13:07.397 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:13:07.418 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:07.417+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:07.423 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:13:07.500 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:07.500 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:07.500 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:07.500 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:07.500 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:07.501 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:07.499+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:07.510 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:07.560 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone21 from 0xd4d4600 (overlap 0xe33400) to 0x719a00 2024-09-13T09:13:07.595 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:07.594+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:07.634 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:07.653 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:07.670 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:13:07.670 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:13:07.677 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:07.677 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:07.681 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:07.680+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:07.687 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:07.687 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:07.688 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:13:07.708 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:07.709 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:07.726 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:07.726 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:07.738 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:13:07.738 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:07.738 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 69, 2024-09-13T09:13:07.738 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "06cda68b-dd73-4266-b5e6-93ae2b3f7bd5", 2024-09-13T09:13:07.739 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2024-09-13T09:11:45.458147+0000", 2024-09-13T09:13:07.739 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-09-13T09:13:07.379932+0000", 2024-09-13T09:13:07.739 INFO:teuthology.orchestra.run.smithi044.stdout: "last_up_change": "2024-09-13T09:11:56.271910+0000", 2024-09-13T09:13:07.739 INFO:teuthology.orchestra.run.smithi044.stdout: "last_in_change": "2024-09-13T09:12:42.513669+0000", 2024-09-13T09:13:07.739 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:13:07.739 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_num": 5806080, 2024-09-13T09:13:07.739 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_set": [ 2024-09-13T09:13:07.740 INFO:teuthology.orchestra.run.smithi044.stdout: "nodeep-scrub", 2024-09-13T09:13:07.740 INFO:teuthology.orchestra.run.smithi044.stdout: "noscrub", 2024-09-13T09:13:07.740 INFO:teuthology.orchestra.run.smithi044.stdout: "pglog_hardlimit", 2024-09-13T09:13:07.740 INFO:teuthology.orchestra.run.smithi044.stdout: "purged_snapdirs", 2024-09-13T09:13:07.740 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_deletes", 2024-09-13T09:13:07.740 INFO:teuthology.orchestra.run.smithi044.stdout: "sortbitwise" 2024-09-13T09:13:07.740 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:13:07.741 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_version": 4, 2024-09-13T09:13:07.741 INFO:teuthology.orchestra.run.smithi044.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:13:07.741 INFO:teuthology.orchestra.run.smithi044.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:13:07.741 INFO:teuthology.orchestra.run.smithi044.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:13:07.741 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_snapshot": "", 2024-09-13T09:13:07.741 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_max": 3, 2024-09-13T09:13:07.741 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd": 6, 2024-09-13T09:13:07.742 INFO:teuthology.orchestra.run.smithi044.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:13:07.742 INFO:teuthology.orchestra.run.smithi044.stdout: "min_compat_client": "luminous", 2024-09-13T09:13:07.742 INFO:teuthology.orchestra.run.smithi044.stdout: "require_osd_release": "reef", 2024-09-13T09:13:07.742 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_crimson": false, 2024-09-13T09:13:07.742 INFO:teuthology.orchestra.run.smithi044.stdout: "pools": [ 2024-09-13T09:13:07.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.743 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 1, 2024-09-13T09:13:07.743 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": ".mgr", 2024-09-13T09:13:07.743 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-09-13T09:11:57.562930+0000", 2024-09-13T09:13:07.743 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-09-13T09:13:07.743 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool", 2024-09-13T09:13:07.743 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-09-13T09:13:07.743 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-09-13T09:13:07.743 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-09-13T09:13:07.743 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-09-13T09:13:07.743 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:13:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:13:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:13:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:13:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-09-13T09:13:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:13:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 1, 2024-09-13T09:13:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 1, 2024-09-13T09:13:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 1, 2024-09-13T09:13:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 1, 2024-09-13T09:13:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 1, 2024-09-13T09:13:07.744 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-09-13T09:13:07.745 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-09-13T09:13:07.745 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-09-13T09:13:07.745 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-09-13T09:13:07.745 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-09-13T09:13:07.745 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-09-13T09:13:07.745 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-09-13T09:13:07.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.745 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "15", 2024-09-13T09:13:07.745 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-09-13T09:13:07.745 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:13:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:13:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-09-13T09:13:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:13:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 0, 2024-09-13T09:13:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 0, 2024-09-13T09:13:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-09-13T09:13:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-09-13T09:13:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-09-13T09:13:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-09-13T09:13:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-09-13T09:13:07.746 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-09-13T09:13:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-09-13T09:13:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-09-13T09:13:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-09-13T09:13:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-09-13T09:13:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-09-13T09:13:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:13:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:13:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:13:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-09-13T09:13:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-09-13T09:13:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-09-13T09:13:07.747 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-09-13T09:13:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-09-13T09:13:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-09-13T09:13:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-09-13T09:13:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-09-13T09:13:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:13:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:13:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:13:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:13:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-09-13T09:13:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-09-13T09:13:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-09-13T09:13:07.748 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-09-13T09:13:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: "options": { 2024-09-13T09:13:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_max": 32, 2024-09-13T09:13:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_min": 1 2024-09-13T09:13:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-09-13T09:13:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": {} 2024-09-13T09:13:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-09-13T09:13:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 6.059999942779541, 2024-09-13T09:13:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 6.059999942779541, 2024-09-13T09:13:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 0.33000001311302185, 2024-09-13T09:13:07.749 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 2, 2024-09-13T09:13:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 2, 2024-09-13T09:13:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:13:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 1, 2024-09-13T09:13:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:13:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 2, 2024-09-13T09:13:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": "rbd", 2024-09-13T09:13:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-09-13T09:12:03.214789+0000", 2024-09-13T09:13:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 8193, 2024-09-13T09:13:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:13:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-09-13T09:13:07.750 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-09-13T09:13:07.751 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-09-13T09:13:07.751 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-09-13T09:13:07.751 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:13:07.751 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:13:07.751 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:13:07.751 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:13:07.751 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-09-13T09:13:07.751 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:13:07.751 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 8, 2024-09-13T09:13:07.752 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 8, 2024-09-13T09:13:07.752 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 8, 2024-09-13T09:13:07.752 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 8, 2024-09-13T09:13:07.752 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 8, 2024-09-13T09:13:07.752 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-09-13T09:13:07.752 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-09-13T09:13:07.752 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-09-13T09:13:07.752 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-09-13T09:13:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-09-13T09:13:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-09-13T09:13:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-09-13T09:13:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "19", 2024-09-13T09:13:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-09-13T09:13:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:13:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:13:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-09-13T09:13:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:13:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 3, 2024-09-13T09:13:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 19, 2024-09-13T09:13:07.753 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-09-13T09:13:07.754 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-09-13T09:13:07.754 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-09-13T09:13:07.754 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-09-13T09:13:07.754 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-09-13T09:13:07.754 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-09-13T09:13:07.754 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-09-13T09:13:07.754 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-09-13T09:13:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-09-13T09:13:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-09-13T09:13:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-09-13T09:13:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:13:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:13:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:13:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-09-13T09:13:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-09-13T09:13:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-09-13T09:13:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-09-13T09:13:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-09-13T09:13:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-09-13T09:13:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-09-13T09:13:07.755 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-09-13T09:13:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:13:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:13:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:13:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:13:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-09-13T09:13:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-09-13T09:13:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-09-13T09:13:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-09-13T09:13:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "options": {}, 2024-09-13T09:13:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-09-13T09:13:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd": {} 2024-09-13T09:13:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.756 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-09-13T09:13:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 2.25, 2024-09-13T09:13:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 2.25, 2024-09-13T09:13:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 1, 2024-09-13T09:13:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 2.25, 2024-09-13T09:13:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 2.25, 2024-09-13T09:13:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:13:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 1, 2024-09-13T09:13:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:13:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 3, 2024-09-13T09:13:07.757 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": "pool_client.0", 2024-09-13T09:13:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-09-13T09:12:12.120436+0000", 2024-09-13T09:13:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 8193, 2024-09-13T09:13:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:13:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-09-13T09:13:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-09-13T09:13:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-09-13T09:13:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-09-13T09:13:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:13:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:13:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:13:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:13:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-09-13T09:13:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:13:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 32, 2024-09-13T09:13:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 32, 2024-09-13T09:13:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 32, 2024-09-13T09:13:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 32, 2024-09-13T09:13:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 32, 2024-09-13T09:13:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-09-13T09:13:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-09-13T09:13:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-09-13T09:13:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-09-13T09:13:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-09-13T09:13:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-09-13T09:13:07.759 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-09-13T09:13:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "69", 2024-09-13T09:13:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-09-13T09:13:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:13:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:13:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-09-13T09:13:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:13:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 25, 2024-09-13T09:13:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 69, 2024-09-13T09:13:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-09-13T09:13:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-09-13T09:13:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-09-13T09:13:07.760 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-09-13T09:13:07.761 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-09-13T09:13:07.761 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-09-13T09:13:07.761 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-09-13T09:13:07.761 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-09-13T09:13:07.761 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-09-13T09:13:07.761 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-09-13T09:13:07.761 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-09-13T09:13:07.761 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:13:07.761 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:13:07.761 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:13:07.761 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-09-13T09:13:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-09-13T09:13:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-09-13T09:13:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-09-13T09:13:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-09-13T09:13:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-09-13T09:13:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-09-13T09:13:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-09-13T09:13:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:13:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:13:07.762 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:13:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:13:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-09-13T09:13:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-09-13T09:13:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-09-13T09:13:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-09-13T09:13:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: "options": {}, 2024-09-13T09:13:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-09-13T09:13:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd": {} 2024-09-13T09:13:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-09-13T09:13:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 1.5, 2024-09-13T09:13:07.763 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 1.5, 2024-09-13T09:13:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 1, 2024-09-13T09:13:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 1.5, 2024-09-13T09:13:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 1.5, 2024-09-13T09:13:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:13:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 1, 2024-09-13T09:13:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:13:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:13:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osds": [ 2024-09-13T09:13:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.764 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 0, 2024-09-13T09:13:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "156c6730-f804-4806-bc2d-3a7d0856c2b2", 2024-09-13T09:13:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:13:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:13:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:13:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:13:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:13:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:13:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:13:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 61, 2024-09-13T09:13:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:13:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:13:07.765 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:13:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6808", 2024-09-13T09:13:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6810", 2024-09-13T09:13:07.766 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:13:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6813", 2024-09-13T09:13:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6815", 2024-09-13T09:13:07.767 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:13:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6821", 2024-09-13T09:13:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.768 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.769 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.769 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6823", 2024-09-13T09:13:07.769 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:07.769 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.769 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.769 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:13:07.769 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.769 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6817", 2024-09-13T09:13:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6819", 2024-09-13T09:13:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2268459392 2024-09-13T09:13:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6810/2268459392", 2024-09-13T09:13:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6815/2268459392", 2024-09-13T09:13:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6823/2268459392", 2024-09-13T09:13:07.770 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6819/2268459392", 2024-09-13T09:13:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:13:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:13:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:13:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 1, 2024-09-13T09:13:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "00735025-3bf9-4dd0-8b42-f08f1bd1c421", 2024-09-13T09:13:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:13:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:13:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:13:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:13:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:13:07.771 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:13:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:13:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 48, 2024-09-13T09:13:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:13:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:13:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:13:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6800", 2024-09-13T09:13:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.772 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6801", 2024-09-13T09:13:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:13:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6802", 2024-09-13T09:13:07.773 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6803", 2024-09-13T09:13:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:13:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.775 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.775 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6806", 2024-09-13T09:13:07.775 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:07.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.775 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6807", 2024-09-13T09:13:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:13:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6804", 2024-09-13T09:13:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.776 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.777 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6805", 2024-09-13T09:13:07.777 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2209612586 2024-09-13T09:13:07.777 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.777 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.777 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6801/2209612586", 2024-09-13T09:13:07.777 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6803/2209612586", 2024-09-13T09:13:07.777 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6807/2209612586", 2024-09-13T09:13:07.778 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6805/2209612586", 2024-09-13T09:13:07.778 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:13:07.778 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:13:07.778 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:13:07.778 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.778 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 2, 2024-09-13T09:13:07.778 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "395516ac-33b6-4f43-8333-dcb1f4cded32", 2024-09-13T09:13:07.778 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:13:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:13:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:13:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:13:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:13:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:13:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:13:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 62, 2024-09-13T09:13:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:13:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:13:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:13:07.779 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6809", 2024-09-13T09:13:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6811", 2024-09-13T09:13:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.780 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:13:07.781 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.781 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.781 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6812", 2024-09-13T09:13:07.781 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:07.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.781 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.781 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6814", 2024-09-13T09:13:07.781 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:07.782 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.782 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.782 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:13:07.782 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.782 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.782 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6820", 2024-09-13T09:13:07.782 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6822", 2024-09-13T09:13:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:13:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.784 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.784 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6816", 2024-09-13T09:13:07.784 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:07.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.784 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.784 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6818", 2024-09-13T09:13:07.784 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 770926265 2024-09-13T09:13:07.784 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.784 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6811/770926265", 2024-09-13T09:13:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6814/770926265", 2024-09-13T09:13:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6822/770926265", 2024-09-13T09:13:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6818/770926265", 2024-09-13T09:13:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:13:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:13:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:13:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 3, 2024-09-13T09:13:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "429e877f-5462-402a-80e5-92a7b17a2eaa", 2024-09-13T09:13:07.785 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:13:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:13:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:13:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:13:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:13:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:13:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:13:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 49, 2024-09-13T09:13:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:13:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:13:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:13:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.786 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6808", 2024-09-13T09:13:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6809", 2024-09-13T09:13:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:13:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.787 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6810", 2024-09-13T09:13:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6811", 2024-09-13T09:13:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:13:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.788 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6814", 2024-09-13T09:13:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6815", 2024-09-13T09:13:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:13:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6812", 2024-09-13T09:13:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6813", 2024-09-13T09:13:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2132151186 2024-09-13T09:13:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.129:6809/2132151186", 2024-09-13T09:13:07.790 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.129:6811/2132151186", 2024-09-13T09:13:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.129:6815/2132151186", 2024-09-13T09:13:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.129:6813/2132151186", 2024-09-13T09:13:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:13:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:13:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:13:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 4, 2024-09-13T09:13:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "9b3be757-15ee-4157-80a8-0537cd1bcf28", 2024-09-13T09:13:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:13:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:13:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:13:07.791 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:13:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:13:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:13:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:13:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 48, 2024-09-13T09:13:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:13:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:13:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:13:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6800", 2024-09-13T09:13:07.792 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:07.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.793 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.793 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6801", 2024-09-13T09:13:07.793 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:07.793 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.793 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.794 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:13:07.794 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.794 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.794 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6802", 2024-09-13T09:13:07.794 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:07.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.794 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6803", 2024-09-13T09:13:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:13:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6806", 2024-09-13T09:13:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.795 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6807", 2024-09-13T09:13:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:13:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6804", 2024-09-13T09:13:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6805", 2024-09-13T09:13:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3393542416 2024-09-13T09:13:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.129:6801/3393542416", 2024-09-13T09:13:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.129:6803/3393542416", 2024-09-13T09:13:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.129:6807/3393542416", 2024-09-13T09:13:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.129:6805/3393542416", 2024-09-13T09:13:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:13:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:13:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:13:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 5, 2024-09-13T09:13:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "c7601d5e-2164-45e1-ab21-368d9e3af27e", 2024-09-13T09:13:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-09-13T09:13:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-09-13T09:13:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-09-13T09:13:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-09-13T09:13:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-09-13T09:13:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-09-13T09:13:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 11, 2024-09-13T09:13:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 65, 2024-09-13T09:13:07.798 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-09-13T09:13:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-09-13T09:13:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-09-13T09:13:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6816", 2024-09-13T09:13:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6817", 2024-09-13T09:13:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.799 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.800 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-09-13T09:13:07.800 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.800 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.800 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6818", 2024-09-13T09:13:07.800 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:07.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6819", 2024-09-13T09:13:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-09-13T09:13:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6822", 2024-09-13T09:13:07.801 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6823", 2024-09-13T09:13:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-09-13T09:13:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-09-13T09:13:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-09-13T09:13:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6820", 2024-09-13T09:13:07.802 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-09-13T09:13:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.129:6821", 2024-09-13T09:13:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1803198188 2024-09-13T09:13:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.129:6817/1803198188", 2024-09-13T09:13:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.129:6819/1803198188", 2024-09-13T09:13:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.129:6823/1803198188", 2024-09-13T09:13:07.803 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.129:6821/1803198188", 2024-09-13T09:13:07.804 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-09-13T09:13:07.804 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-09-13T09:13:07.804 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-09-13T09:13:07.804 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.804 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.804 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:13:07.804 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_xinfo": [ 2024-09-13T09:13:07.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 0, 2024-09-13T09:13:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:13:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:13:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:13:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:13:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:13:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.230474+0000", 2024-09-13T09:13:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:13:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 1, 2024-09-13T09:13:07.805 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:13:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:13:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:13:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:13:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:13:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.195476+0000", 2024-09-13T09:13:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:13:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 2, 2024-09-13T09:13:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:13:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:13:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:13:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:13:07.806 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:13:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.248729+0000", 2024-09-13T09:13:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:13:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 3, 2024-09-13T09:13:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:13:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:13:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:13:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:13:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:13:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.195482+0000", 2024-09-13T09:13:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:13:07.807 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 4, 2024-09-13T09:13:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:13:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:13:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:13:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:13:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:13:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.198746+0000", 2024-09-13T09:13:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:13:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 5, 2024-09-13T09:13:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-09-13T09:13:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-09-13T09:13:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-09-13T09:13:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-09-13T09:13:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-09-13T09:13:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-09-13T09:11:53.279957+0000", 2024-09-13T09:13:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-09-13T09:13:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:13:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap": [ 2024-09-13T09:13:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: "pgid": "2.7", 2024-09-13T09:13:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: "osds": [ 2024-09-13T09:13:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: 5, 2024-09-13T09:13:07.809 INFO:teuthology.orchestra.run.smithi044.stdout: 4 2024-09-13T09:13:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: "pgid": "3.1b", 2024-09-13T09:13:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: "osds": [ 2024-09-13T09:13:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: 2, 2024-09-13T09:13:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: 3 2024-09-13T09:13:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:13:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap_items": [], 2024-09-13T09:13:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap_primaries": [], 2024-09-13T09:13:07.810 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_temp": [], 2024-09-13T09:13:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_temp": [], 2024-09-13T09:13:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklist": {}, 2024-09-13T09:13:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: "range_blocklist": {}, 2024-09-13T09:13:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profiles": { 2024-09-13T09:13:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: "default": { 2024-09-13T09:13:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-failure-domain": "osd", 2024-09-13T09:13:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: "k": "2", 2024-09-13T09:13:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: "m": "1", 2024-09-13T09:13:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: "plugin": "jerasure", 2024-09-13T09:13:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: "technique": "reed_sol_van" 2024-09-13T09:13:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps_queue": [], 2024-09-13T09:13:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: "new_removed_snaps": [], 2024-09-13T09:13:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: "new_purged_snaps": [], 2024-09-13T09:13:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_node_flags": {}, 2024-09-13T09:13:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: "device_class_flags": {}, 2024-09-13T09:13:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": { 2024-09-13T09:13:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode_enabled": false, 2024-09-13T09:13:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_bucket_count": 0, 2024-09-13T09:13:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:13:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:13:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode_bucket": 0 2024-09-13T09:13:07.812 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:07.812 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:07.815 INFO:tasks.thrashosds.thrasher:j is {'epoch': 69, 'fsid': '06cda68b-dd73-4266-b5e6-93ae2b3f7bd5', 'created': '2024-09-13T09:11:45.458147+0000', 'modified': '2024-09-13T09:13:07.379932+0000', 'last_up_change': '2024-09-13T09:11:56.271910+0000', 'last_in_change': '2024-09-13T09:12:42.513669+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:11:57.562930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:12:03.214789+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:12:12.120436+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 69, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '156c6730-f804-4806-bc2d-3a7d0856c2b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6808', 'nonce': 2268459392}, {'type': 'v1', 'addr': '172.21.15.44:6810', 'nonce': 2268459392}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6813', 'nonce': 2268459392}, {'type': 'v1', 'addr': '172.21.15.44:6815', 'nonce': 2268459392}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6821', 'nonce': 2268459392}, {'type': 'v1', 'addr': '172.21.15.44:6823', 'nonce': 2268459392}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6817', 'nonce': 2268459392}, {'type': 'v1', 'addr': '172.21.15.44:6819', 'nonce': 2268459392}]}, 'public_addr': '172.21.15.44:6810/2268459392', 'cluster_addr': '172.21.15.44:6815/2268459392', 'heartbeat_back_addr': '172.21.15.44:6823/2268459392', 'heartbeat_front_addr': '172.21.15.44:6819/2268459392', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '00735025-3bf9-4dd0-8b42-f08f1bd1c421', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6800', 'nonce': 2209612586}, {'type': 'v1', 'addr': '172.21.15.44:6801', 'nonce': 2209612586}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6802', 'nonce': 2209612586}, {'type': 'v1', 'addr': '172.21.15.44:6803', 'nonce': 2209612586}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6806', 'nonce': 2209612586}, {'type': 'v1', 'addr': '172.21.15.44:6807', 'nonce': 2209612586}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6804', 'nonce': 2209612586}, {'type': 'v1', 'addr': '172.21.15.44:6805', 'nonce': 2209612586}]}, 'public_addr': '172.21.15.44:6801/2209612586', 'cluster_addr': '172.21.15.44:6803/2209612586', 'heartbeat_back_addr': '172.21.15.44:6807/2209612586', 'heartbeat_front_addr': '172.21.15.44:6805/2209612586', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '395516ac-33b6-4f43-8333-dcb1f4cded32', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6809', 'nonce': 770926265}, {'type': 'v1', 'addr': '172.21.15.44:6811', 'nonce': 770926265}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6812', 'nonce': 770926265}, {'type': 'v1', 'addr': '172.21.15.44:6814', 'nonce': 770926265}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6820', 'nonce': 770926265}, {'type': 'v1', 'addr': '172.21.15.44:6822', 'nonce': 770926265}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6816', 'nonce': 770926265}, {'type': 'v1', 'addr': '172.21.15.44:6818', 'nonce': 770926265}]}, 'public_addr': '172.21.15.44:6811/770926265', 'cluster_addr': '172.21.15.44:6814/770926265', 'heartbeat_back_addr': '172.21.15.44:6822/770926265', 'heartbeat_front_addr': '172.21.15.44:6818/770926265', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '429e877f-5462-402a-80e5-92a7b17a2eaa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6808', 'nonce': 2132151186}, {'type': 'v1', 'addr': '172.21.15.129:6809', 'nonce': 2132151186}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6810', 'nonce': 2132151186}, {'type': 'v1', 'addr': '172.21.15.129:6811', 'nonce': 2132151186}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6814', 'nonce': 2132151186}, {'type': 'v1', 'addr': '172.21.15.129:6815', 'nonce': 2132151186}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6812', 'nonce': 2132151186}, {'type': 'v1', 'addr': '172.21.15.129:6813', 'nonce': 2132151186}]}, 'public_addr': '172.21.15.129:6809/2132151186', 'cluster_addr': '172.21.15.129:6811/2132151186', 'heartbeat_back_addr': '172.21.15.129:6815/2132151186', 'heartbeat_front_addr': '172.21.15.129:6813/2132151186', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '9b3be757-15ee-4157-80a8-0537cd1bcf28', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6800', 'nonce': 3393542416}, {'type': 'v1', 'addr': '172.21.15.129:6801', 'nonce': 3393542416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6802', 'nonce': 3393542416}, {'type': 'v1', 'addr': '172.21.15.129:6803', 'nonce': 3393542416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6806', 'nonce': 3393542416}, {'type': 'v1', 'addr': '172.21.15.129:6807', 'nonce': 3393542416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6804', 'nonce': 3393542416}, {'type': 'v1', 'addr': '172.21.15.129:6805', 'nonce': 3393542416}]}, 'public_addr': '172.21.15.129:6801/3393542416', 'cluster_addr': '172.21.15.129:6803/3393542416', 'heartbeat_back_addr': '172.21.15.129:6807/3393542416', 'heartbeat_front_addr': '172.21.15.129:6805/3393542416', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c7601d5e-2164-45e1-ab21-368d9e3af27e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 65, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6816', 'nonce': 1803198188}, {'type': 'v1', 'addr': '172.21.15.129:6817', 'nonce': 1803198188}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6818', 'nonce': 1803198188}, {'type': 'v1', 'addr': '172.21.15.129:6819', 'nonce': 1803198188}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6822', 'nonce': 1803198188}, {'type': 'v1', 'addr': '172.21.15.129:6823', 'nonce': 1803198188}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6820', 'nonce': 1803198188}, {'type': 'v1', 'addr': '172.21.15.129:6821', 'nonce': 1803198188}]}, 'public_addr': '172.21.15.129:6817/1803198188', 'cluster_addr': '172.21.15.129:6819/1803198188', 'heartbeat_back_addr': '172.21.15.129:6823/1803198188', 'heartbeat_front_addr': '172.21.15.129:6821/1803198188', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.230474+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.195476+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.248729+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.195482+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.198746+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:11:53.279957+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.7', 'osds': [5, 4]}, {'pgid': '3.1b', 'osds': [2, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:13:07.815 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:13:07.820 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:07.820 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:07.820 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:07.819+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:07.828 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:13:07.916 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:07.915+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:07.982 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:07.983 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:07.993 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:13:08.017 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:08.016+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:08.102 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:08.102 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:08.111 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:08.123 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:13:08.123 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:13:08.135 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-13T09:13:07.527892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":834669809,"num_objects":755,"num_object_clones":244,"num_object_copies":1510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":755,"num_whiteouts":235,"num_read":157369,"num_read_kb":683073,"num_write":10956,"num_write_kb":191798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":11324303,"num_keys_recovered":33,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9313,"ondisk_log_size":9313,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":409808,"kb_used_data":169764,"kb_used_omap":9,"kb_used_meta":239862,"kb_avail":565821232,"statfs":{"total":579820584960,"available":579400941568,"internally_reserved":0,"allocated":173838336,"data_stored":186169008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9563,"internal_metadata":245619365},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20678028,"num_objects":79,"num_object_clones":31,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":53,"num_read":24351,"num_read_kb":86235,"num_write":1055,"num_write_kb":18080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.169628"},"pg_stats":[{"pgid":"3.1f","version":"68'339","reported_seq":3093,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.542750+0000","last_change":"2024-09-13T09:12:43.692870+0000","last_active":"2024-09-13T09:13:06.542750+0000","last_peered":"2024-09-13T09:13:06.542750+0000","last_clean":"2024-09-13T09:13:06.542750+0000","last_became_active":"2024-09-13T09:12:43.534990+0000","last_became_peered":"2024-09-13T09:12:43.534990+0000","last_unstale":"2024-09-13T09:13:06.542750+0000","last_undegraded":"2024-09-13T09:13:06.542750+0000","last_fullsized":"2024-09-13T09:13:06.542750+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:45.810610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058630900000000003,"stat_sum":{"num_bytes":9588789,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":3894,"num_read_kb":5653,"num_write":341,"num_write_kb":1862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":848912,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"68'371","reported_seq":4010,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.311459+0000","last_change":"2024-09-13T09:12:43.652049+0000","last_active":"2024-09-13T09:13:06.311459+0000","last_peered":"2024-09-13T09:13:06.311459+0000","last_clean":"2024-09-13T09:13:06.311459+0000","last_became_active":"2024-09-13T09:12:43.532920+0000","last_became_peered":"2024-09-13T09:12:43.532920+0000","last_unstale":"2024-09-13T09:13:06.311459+0000","last_undegraded":"2024-09-13T09:13:06.311459+0000","last_fullsized":"2024-09-13T09:13:06.311459+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:10:38.824649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088145900000000002,"stat_sum":{"num_bytes":28026432,"num_objects":27,"num_object_clones":11,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":5633,"num_read_kb":21053,"num_write":554,"num_write_kb":9865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":311808,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"68'732","reported_seq":3207,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.531340+0000","last_change":"2024-09-13T09:12:16.388369+0000","last_active":"2024-09-13T09:13:06.531340+0000","last_peered":"2024-09-13T09:13:06.531340+0000","last_clean":"2024-09-13T09:13:06.531340+0000","last_became_active":"2024-09-13T09:12:13.362666+0000","last_became_peered":"2024-09-13T09:12:13.362666+0000","last_unstale":"2024-09-13T09:13:06.531340+0000","last_undegraded":"2024-09-13T09:13:06.531340+0000","last_fullsized":"2024-09-13T09:13:06.531340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:44:04.594826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039115800000000001,"stat_sum":{"num_bytes":8567963,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":5393,"num_read_kb":5900,"num_write":1096,"num_write_kb":4188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"68'286","reported_seq":3945,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.505095+0000","last_change":"2024-09-13T09:12:16.388972+0000","last_active":"2024-09-13T09:13:06.505095+0000","last_peered":"2024-09-13T09:13:06.505095+0000","last_clean":"2024-09-13T09:13:06.505095+0000","last_became_active":"2024-09-13T09:12:13.364622+0000","last_became_peered":"2024-09-13T09:12:13.364622+0000","last_unstale":"2024-09-13T09:13:06.505095+0000","last_undegraded":"2024-09-13T09:13:06.505095+0000","last_fullsized":"2024-09-13T09:13:06.505095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:34:20.301837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088725000000000002,"stat_sum":{"num_bytes":47518310,"num_objects":32,"num_object_clones":11,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":9,"num_read":5219,"num_read_kb":26840,"num_write":249,"num_write_kb":11743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"68'318","reported_seq":5923,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.538629+0000","last_change":"2024-09-13T09:12:59.335769+0000","last_active":"2024-09-13T09:13:06.538629+0000","last_peered":"2024-09-13T09:13:06.538629+0000","last_clean":"2024-09-13T09:13:06.538629+0000","last_became_active":"2024-09-13T09:12:59.334125+0000","last_became_peered":"2024-09-13T09:12:59.334125+0000","last_unstale":"2024-09-13T09:13:06.538629+0000","last_undegraded":"2024-09-13T09:13:06.538629+0000","last_fullsized":"2024-09-13T09:13:06.538629+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:12.239435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082447100000000001,"stat_sum":{"num_bytes":35335281,"num_objects":33,"num_object_clones":11,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":10,"num_read":6379,"num_read_kb":38175,"num_write":370,"num_write_kb":8047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1403976,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"68'214","reported_seq":6164,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.538277+0000","last_change":"2024-09-13T09:12:16.380731+0000","last_active":"2024-09-13T09:13:06.538277+0000","last_peered":"2024-09-13T09:13:06.538277+0000","last_clean":"2024-09-13T09:13:06.538277+0000","last_became_active":"2024-09-13T09:12:13.362533+0000","last_became_peered":"2024-09-13T09:12:13.362533+0000","last_unstale":"2024-09-13T09:13:06.538277+0000","last_undegraded":"2024-09-13T09:13:06.538277+0000","last_fullsized":"2024-09-13T09:13:06.538277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:50.399315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072061100000000002,"stat_sum":{"num_bytes":23726627,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":6855,"num_read_kb":22506,"num_write":171,"num_write_kb":1279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"68'169","reported_seq":4881,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:05.574221+0000","last_change":"2024-09-13T09:12:16.376447+0000","last_active":"2024-09-13T09:13:05.574221+0000","last_peered":"2024-09-13T09:13:05.574221+0000","last_clean":"2024-09-13T09:13:05.574221+0000","last_became_active":"2024-09-13T09:12:13.368221+0000","last_became_peered":"2024-09-13T09:12:13.368221+0000","last_unstale":"2024-09-13T09:13:05.574221+0000","last_undegraded":"2024-09-13T09:13:05.574221+0000","last_fullsized":"2024-09-13T09:13:05.574221+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:10.966835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000740483,"stat_sum":{"num_bytes":51753027,"num_objects":23,"num_object_clones":8,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":4884,"num_read_kb":9762,"num_write":190,"num_write_kb":1550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"68'190","reported_seq":2603,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:05.709412+0000","last_change":"2024-09-13T09:12:16.377490+0000","last_active":"2024-09-13T09:13:05.709412+0000","last_peered":"2024-09-13T09:13:05.709412+0000","last_clean":"2024-09-13T09:13:05.709412+0000","last_became_active":"2024-09-13T09:12:13.367773+0000","last_became_peered":"2024-09-13T09:12:13.367773+0000","last_unstale":"2024-09-13T09:13:05.709412+0000","last_undegraded":"2024-09-13T09:13:05.709412+0000","last_fullsized":"2024-09-13T09:13:05.709412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:34.023964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000479879,"stat_sum":{"num_bytes":6144528,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":3558,"num_read_kb":9073,"num_write":155,"num_write_kb":4856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"67'213","reported_seq":3211,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:05.394608+0000","last_change":"2024-09-13T09:12:16.381209+0000","last_active":"2024-09-13T09:13:05.394608+0000","last_peered":"2024-09-13T09:13:05.394608+0000","last_clean":"2024-09-13T09:13:05.394608+0000","last_became_active":"2024-09-13T09:12:13.365820+0000","last_became_peered":"2024-09-13T09:12:13.365820+0000","last_unstale":"2024-09-13T09:13:05.394608+0000","last_undegraded":"2024-09-13T09:13:05.394608+0000","last_fullsized":"2024-09-13T09:13:05.394608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:48.073570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097766600000000008,"stat_sum":{"num_bytes":12229120,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":3844,"num_read_kb":22167,"num_write":219,"num_write_kb":5139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"68'353","reported_seq":2466,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:05.797579+0000","last_change":"2024-09-13T09:12:46.251509+0000","last_active":"2024-09-13T09:13:05.797579+0000","last_peered":"2024-09-13T09:13:05.797579+0000","last_clean":"2024-09-13T09:13:05.797579+0000","last_became_active":"2024-09-13T09:12:46.250870+0000","last_became_peered":"2024-09-13T09:12:46.250870+0000","last_unstale":"2024-09-13T09:13:05.797579+0000","last_undegraded":"2024-09-13T09:13:05.797579+0000","last_fullsized":"2024-09-13T09:13:05.797579+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:33.138405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040009000000000003,"stat_sum":{"num_bytes":22950515,"num_objects":27,"num_object_clones":9,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":2987,"num_read_kb":14090,"num_write":487,"num_write_kb":1777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":706121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"67'595","reported_seq":5870,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:05.617969+0000","last_change":"2024-09-13T09:12:46.255856+0000","last_active":"2024-09-13T09:13:05.617969+0000","last_peered":"2024-09-13T09:13:05.617969+0000","last_clean":"2024-09-13T09:13:05.617969+0000","last_became_active":"2024-09-13T09:12:46.250696+0000","last_became_peered":"2024-09-13T09:12:46.250696+0000","last_unstale":"2024-09-13T09:13:05.617969+0000","last_undegraded":"2024-09-13T09:13:05.617969+0000","last_fullsized":"2024-09-13T09:13:05.617969+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:43.821997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069172600000000004,"stat_sum":{"num_bytes":16830501,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":7823,"num_read_kb":21035,"num_write":1023,"num_write_kb":2985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1136450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"68'282","reported_seq":3521,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.444892+0000","last_change":"2024-09-13T09:12:46.254606+0000","last_active":"2024-09-13T09:13:06.444892+0000","last_peered":"2024-09-13T09:13:06.444892+0000","last_clean":"2024-09-13T09:13:06.444892+0000","last_became_active":"2024-09-13T09:12:46.254016+0000","last_became_peered":"2024-09-13T09:12:46.254016+0000","last_unstale":"2024-09-13T09:13:06.444892+0000","last_undegraded":"2024-09-13T09:13:06.444892+0000","last_fullsized":"2024-09-13T09:13:06.444892+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:50.193567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074689299999999999,"stat_sum":{"num_bytes":46676853,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":4352,"num_read_kb":18029,"num_write":237,"num_write_kb":3991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1110608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:05.380505+0000","last_change":"2024-09-13T09:13:03.364523+0000","last_active":"2024-09-13T09:13:05.380505+0000","last_peered":"2024-09-13T09:13:05.380505+0000","last_clean":"2024-09-13T09:13:05.380505+0000","last_became_active":"2024-09-13T09:13:03.364099+0000","last_became_peered":"2024-09-13T09:13:03.364099+0000","last_unstale":"2024-09-13T09:13:05.380505+0000","last_undegraded":"2024-09-13T09:13:05.380505+0000","last_fullsized":"2024-09-13T09:13:05.380505+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:00:04.139675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"68'450","reported_seq":5445,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.286321+0000","last_change":"2024-09-13T09:12:16.380593+0000","last_active":"2024-09-13T09:13:06.286321+0000","last_peered":"2024-09-13T09:13:06.286321+0000","last_clean":"2024-09-13T09:13:06.286321+0000","last_became_active":"2024-09-13T09:12:13.365496+0000","last_became_peered":"2024-09-13T09:12:13.365496+0000","last_unstale":"2024-09-13T09:13:06.286321+0000","last_undegraded":"2024-09-13T09:13:06.286321+0000","last_fullsized":"2024-09-13T09:13:06.286321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:29:49.115977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061775199999999995,"stat_sum":{"num_bytes":21520526,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":2,"num_read":6350,"num_read_kb":44991,"num_write":675,"num_write_kb":7335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":81,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:13:04.382007+0000","last_change":"2024-09-13T09:12:43.531266+0000","last_active":"2024-09-13T09:13:04.382007+0000","last_peered":"2024-09-13T09:13:04.382007+0000","last_clean":"2024-09-13T09:13:04.382007+0000","last_became_active":"2024-09-13T09:12:43.530941+0000","last_became_peered":"2024-09-13T09:12:43.530941+0000","last_unstale":"2024-09-13T09:13:04.382007+0000","last_undegraded":"2024-09-13T09:13:04.382007+0000","last_fullsized":"2024-09-13T09:13:04.382007+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:27.261467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"67'424","reported_seq":7181,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:05.709266+0000","last_change":"2024-09-13T09:12:16.378483+0000","last_active":"2024-09-13T09:13:05.709266+0000","last_peered":"2024-09-13T09:13:05.709266+0000","last_clean":"2024-09-13T09:13:05.709266+0000","last_became_active":"2024-09-13T09:12:13.367146+0000","last_became_peered":"2024-09-13T09:12:13.367146+0000","last_unstale":"2024-09-13T09:13:05.709266+0000","last_undegraded":"2024-09-13T09:13:05.709266+0000","last_fullsized":"2024-09-13T09:13:05.709266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:49.370653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032368300000000001,"stat_sum":{"num_bytes":14565376,"num_objects":27,"num_object_clones":9,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":10461,"num_read_kb":30893,"num_write":326,"num_write_kb":6402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:13:04.372465+0000","last_change":"2024-09-13T09:12:43.533829+0000","last_active":"2024-09-13T09:13:04.372465+0000","last_peered":"2024-09-13T09:13:04.372465+0000","last_clean":"2024-09-13T09:13:04.372465+0000","last_became_active":"2024-09-13T09:12:43.533557+0000","last_became_peered":"2024-09-13T09:12:43.533557+0000","last_unstale":"2024-09-13T09:13:04.372465+0000","last_undegraded":"2024-09-13T09:13:04.372465+0000","last_fullsized":"2024-09-13T09:13:04.372465+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:51.939164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"67'200","reported_seq":4209,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:05.697530+0000","last_change":"2024-09-13T09:12:16.376528+0000","last_active":"2024-09-13T09:13:05.697530+0000","last_peered":"2024-09-13T09:13:05.697530+0000","last_clean":"2024-09-13T09:13:05.697530+0000","last_became_active":"2024-09-13T09:12:13.367848+0000","last_became_peered":"2024-09-13T09:12:13.367848+0000","last_unstale":"2024-09-13T09:13:05.697530+0000","last_undegraded":"2024-09-13T09:13:05.697530+0000","last_fullsized":"2024-09-13T09:13:05.697530+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:40:21.943266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067051299999999997,"stat_sum":{"num_bytes":43766784,"num_objects":31,"num_object_clones":11,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":5224,"num_read_kb":19079,"num_write":179,"num_write_kb":5414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":65,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:13:04.381921+0000","last_change":"2024-09-13T09:12:06.328091+0000","last_active":"2024-09-13T09:13:04.381921+0000","last_peered":"2024-09-13T09:13:04.381921+0000","last_clean":"2024-09-13T09:13:04.381921+0000","last_became_active":"2024-09-13T09:12:04.446508+0000","last_became_peered":"2024-09-13T09:12:04.446508+0000","last_unstale":"2024-09-13T09:13:04.381921+0000","last_undegraded":"2024-09-13T09:13:04.381921+0000","last_fullsized":"2024-09-13T09:13:04.381921+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:33.410873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042514000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"65'262","reported_seq":648,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:13:04.393381+0000","last_change":"2024-09-13T09:12:16.389006+0000","last_active":"2024-09-13T09:13:04.393381+0000","last_peered":"2024-09-13T09:13:04.393381+0000","last_clean":"2024-09-13T09:13:04.393381+0000","last_became_active":"2024-09-13T09:12:13.369489+0000","last_became_peered":"2024-09-13T09:12:13.369489+0000","last_unstale":"2024-09-13T09:13:04.393381+0000","last_undegraded":"2024-09-13T09:13:04.393381+0000","last_fullsized":"2024-09-13T09:13:04.393381+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:33.879162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088595999999999998,"stat_sum":{"num_bytes":3100815,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":1226,"num_read_kb":2430,"num_write":380,"num_write_kb":1246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"68'232","reported_seq":8556,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.530468+0000","last_change":"2024-09-13T09:12:16.380981+0000","last_active":"2024-09-13T09:13:06.530468+0000","last_peered":"2024-09-13T09:13:06.530468+0000","last_clean":"2024-09-13T09:13:06.530468+0000","last_became_active":"2024-09-13T09:12:13.367770+0000","last_became_peered":"2024-09-13T09:12:13.367770+0000","last_unstale":"2024-09-13T09:13:06.530468+0000","last_undegraded":"2024-09-13T09:13:06.530468+0000","last_fullsized":"2024-09-13T09:13:06.530468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:21:50.690425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083150800000000001,"stat_sum":{"num_bytes":51234596,"num_objects":42,"num_object_clones":16,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":13,"num_read":8397,"num_read_kb":55412,"num_write":237,"num_write_kb":9820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":66,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:13:04.393222+0000","last_change":"2024-09-13T09:12:06.330649+0000","last_active":"2024-09-13T09:13:04.393222+0000","last_peered":"2024-09-13T09:13:04.393222+0000","last_clean":"2024-09-13T09:13:04.393222+0000","last_became_active":"2024-09-13T09:12:04.325470+0000","last_became_peered":"2024-09-13T09:12:04.325470+0000","last_unstale":"2024-09-13T09:13:04.393222+0000","last_undegraded":"2024-09-13T09:13:04.393222+0000","last_fullsized":"2024-09-13T09:13:04.393222+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:37.971615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093634899999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"67'193","reported_seq":4410,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:05.726779+0000","last_change":"2024-09-13T09:12:16.378750+0000","last_active":"2024-09-13T09:13:05.726779+0000","last_peered":"2024-09-13T09:13:05.726779+0000","last_clean":"2024-09-13T09:13:05.726779+0000","last_became_active":"2024-09-13T09:12:13.367656+0000","last_became_peered":"2024-09-13T09:12:13.367656+0000","last_unstale":"2024-09-13T09:13:05.726779+0000","last_undegraded":"2024-09-13T09:13:05.726779+0000","last_fullsized":"2024-09-13T09:13:05.726779+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:11.942294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003044836,"stat_sum":{"num_bytes":21523488,"num_objects":30,"num_object_clones":11,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":9,"num_read":4230,"num_read_kb":27367,"num_write":186,"num_write_kb":8806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":65,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:13:04.371291+0000","last_change":"2024-09-13T09:12:06.845125+0000","last_active":"2024-09-13T09:13:04.371291+0000","last_peered":"2024-09-13T09:13:04.371291+0000","last_clean":"2024-09-13T09:13:04.371291+0000","last_became_active":"2024-09-13T09:12:04.321941+0000","last_became_peered":"2024-09-13T09:12:04.321941+0000","last_unstale":"2024-09-13T09:13:04.371291+0000","last_undegraded":"2024-09-13T09:13:04.371291+0000","last_fullsized":"2024-09-13T09:13:04.371291+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:57.706060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031569600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"68'112","reported_seq":1272,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.336458+0000","last_change":"2024-09-13T09:12:16.380222+0000","last_active":"2024-09-13T09:13:06.336458+0000","last_peered":"2024-09-13T09:13:06.336458+0000","last_clean":"2024-09-13T09:13:06.336458+0000","last_became_active":"2024-09-13T09:12:13.367633+0000","last_became_peered":"2024-09-13T09:12:13.367633+0000","last_unstale":"2024-09-13T09:13:06.336458+0000","last_undegraded":"2024-09-13T09:13:06.336458+0000","last_fullsized":"2024-09-13T09:13:06.336458+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:34.734632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438009,"stat_sum":{"num_bytes":6547456,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":1116,"num_read_kb":8935,"num_write":111,"num_write_kb":1112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":31,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:13:04.372280+0000","last_change":"2024-09-13T09:12:43.532201+0000","last_active":"2024-09-13T09:13:04.372280+0000","last_peered":"2024-09-13T09:13:04.372280+0000","last_clean":"2024-09-13T09:13:04.372280+0000","last_became_active":"2024-09-13T09:12:43.531942+0000","last_became_peered":"2024-09-13T09:12:43.531942+0000","last_unstale":"2024-09-13T09:13:04.372280+0000","last_undegraded":"2024-09-13T09:13:04.372280+0000","last_fullsized":"2024-09-13T09:13:04.372280+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:26.408883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"67'270","reported_seq":1106,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:05.626326+0000","last_change":"2024-09-13T09:12:16.377544+0000","last_active":"2024-09-13T09:13:05.626326+0000","last_peered":"2024-09-13T09:13:05.626326+0000","last_clean":"2024-09-13T09:13:05.626326+0000","last_became_active":"2024-09-13T09:12:13.361909+0000","last_became_peered":"2024-09-13T09:12:13.361909+0000","last_unstale":"2024-09-13T09:13:05.626326+0000","last_undegraded":"2024-09-13T09:13:05.626326+0000","last_fullsized":"2024-09-13T09:13:05.626326+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:34.570438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001546782,"stat_sum":{"num_bytes":3917451,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":1612,"num_read_kb":4027,"num_write":376,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":65,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:13:04.393284+0000","last_change":"2024-09-13T09:12:06.330140+0000","last_active":"2024-09-13T09:13:04.393284+0000","last_peered":"2024-09-13T09:13:04.393284+0000","last_clean":"2024-09-13T09:13:04.393284+0000","last_became_active":"2024-09-13T09:12:04.324857+0000","last_became_peered":"2024-09-13T09:12:04.324857+0000","last_unstale":"2024-09-13T09:13:04.393284+0000","last_undegraded":"2024-09-13T09:13:04.393284+0000","last_fullsized":"2024-09-13T09:13:04.393284+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:57.400286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038911000000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":234,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:13:04.372285+0000","last_change":"2024-09-13T09:12:46.248714+0000","last_active":"2024-09-13T09:13:04.372285+0000","last_peered":"2024-09-13T09:13:04.372285+0000","last_clean":"2024-09-13T09:13:04.372285+0000","last_became_active":"2024-09-13T09:12:46.248451+0000","last_became_peered":"2024-09-13T09:12:46.248451+0000","last_unstale":"2024-09-13T09:13:04.372285+0000","last_undegraded":"2024-09-13T09:13:04.372285+0000","last_fullsized":"2024-09-13T09:13:04.372285+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:27.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"68'236","reported_seq":2061,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.323096+0000","last_change":"2024-09-13T09:12:16.377642+0000","last_active":"2024-09-13T09:13:06.323096+0000","last_peered":"2024-09-13T09:13:06.323096+0000","last_clean":"2024-09-13T09:13:06.323096+0000","last_became_active":"2024-09-13T09:12:13.369742+0000","last_became_peered":"2024-09-13T09:12:13.369742+0000","last_unstale":"2024-09-13T09:13:06.323096+0000","last_undegraded":"2024-09-13T09:13:06.323096+0000","last_fullsized":"2024-09-13T09:13:06.323096+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:27:48.361252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060355399999999996,"stat_sum":{"num_bytes":26963562,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":9,"num_read":2376,"num_read_kb":11440,"num_write":322,"num_write_kb":10253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"68'367","reported_seq":7850,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.327565+0000","last_change":"2024-09-13T09:12:16.380989+0000","last_active":"2024-09-13T09:13:06.327565+0000","last_peered":"2024-09-13T09:13:06.327565+0000","last_clean":"2024-09-13T09:13:06.327565+0000","last_became_active":"2024-09-13T09:12:13.370096+0000","last_became_peered":"2024-09-13T09:12:13.370096+0000","last_unstale":"2024-09-13T09:13:06.327565+0000","last_undegraded":"2024-09-13T09:13:06.327565+0000","last_fullsized":"2024-09-13T09:13:06.327565+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:14.224546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000903458,"stat_sum":{"num_bytes":43394576,"num_objects":30,"num_object_clones":12,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":9,"num_read":10520,"num_read_kb":26078,"num_write":257,"num_write_kb":5245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"68'193","reported_seq":3018,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.432515+0000","last_change":"2024-09-13T09:12:46.253134+0000","last_active":"2024-09-13T09:13:06.432515+0000","last_peered":"2024-09-13T09:13:06.432515+0000","last_clean":"2024-09-13T09:13:06.432515+0000","last_became_active":"2024-09-13T09:12:46.252578+0000","last_became_peered":"2024-09-13T09:12:46.252578+0000","last_unstale":"2024-09-13T09:13:06.432515+0000","last_undegraded":"2024-09-13T09:13:06.432515+0000","last_fullsized":"2024-09-13T09:13:06.432515+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:03.931492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041459800000000002,"stat_sum":{"num_bytes":56534596,"num_objects":29,"num_object_clones":11,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":7,"num_read":2811,"num_read_kb":23728,"num_write":204,"num_write_kb":7174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2590736,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"68'163","reported_seq":5419,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.244066+0000","last_change":"2024-09-13T09:12:16.377864+0000","last_active":"2024-09-13T09:13:06.244066+0000","last_peered":"2024-09-13T09:13:06.244066+0000","last_clean":"2024-09-13T09:13:06.244066+0000","last_became_active":"2024-09-13T09:12:13.362409+0000","last_became_peered":"2024-09-13T09:12:13.362409+0000","last_unstale":"2024-09-13T09:13:06.244066+0000","last_undegraded":"2024-09-13T09:13:06.244066+0000","last_fullsized":"2024-09-13T09:13:06.244066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:37.023594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001713904,"stat_sum":{"num_bytes":69169184,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":5272,"num_read_kb":59814,"num_write":198,"num_write_kb":14179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"65'449","reported_seq":8048,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:13:04.393462+0000","last_change":"2024-09-13T09:12:16.389470+0000","last_active":"2024-09-13T09:13:04.393462+0000","last_peered":"2024-09-13T09:13:04.393462+0000","last_clean":"2024-09-13T09:13:04.393462+0000","last_became_active":"2024-09-13T09:12:13.365758+0000","last_became_peered":"2024-09-13T09:12:13.365758+0000","last_unstale":"2024-09-13T09:13:04.393462+0000","last_undegraded":"2024-09-13T09:13:04.393462+0000","last_fullsized":"2024-09-13T09:13:04.393462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:46.649879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013123500000000001,"stat_sum":{"num_bytes":93873226,"num_objects":32,"num_object_clones":13,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":9067,"num_read_kb":70605,"num_write":693,"num_write_kb":24992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"64'136","reported_seq":1514,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:05.684737+0000","last_change":"2024-09-13T09:12:43.630169+0000","last_active":"2024-09-13T09:13:05.684737+0000","last_peered":"2024-09-13T09:13:05.684737+0000","last_clean":"2024-09-13T09:13:05.684737+0000","last_became_active":"2024-09-13T09:12:43.533321+0000","last_became_peered":"2024-09-13T09:12:43.533321+0000","last_unstale":"2024-09-13T09:13:05.684737+0000","last_undegraded":"2024-09-13T09:13:05.684737+0000","last_fullsized":"2024-09-13T09:13:05.684737+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:43.973948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000679308,"stat_sum":{"num_bytes":6726699,"num_objects":16,"num_object_clones":7,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":1509,"num_read_kb":7325,"num_write":139,"num_write_kb":4910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"68'140","reported_seq":1876,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:05.827499+0000","last_change":"2024-09-13T09:12:16.389243+0000","last_active":"2024-09-13T09:13:05.827499+0000","last_peered":"2024-09-13T09:13:05.827499+0000","last_clean":"2024-09-13T09:13:05.827499+0000","last_became_active":"2024-09-13T09:12:13.362547+0000","last_became_peered":"2024-09-13T09:12:13.362547+0000","last_unstale":"2024-09-13T09:13:05.827499+0000","last_undegraded":"2024-09-13T09:13:05.827499+0000","last_fullsized":"2024-09-13T09:13:05.827499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:24:24.557400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011202860000000001,"stat_sum":{"num_bytes":2537984,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":2277,"num_read_kb":3622,"num_write":120,"num_write_kb":809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"67'284","reported_seq":1050,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:05.619189+0000","last_change":"2024-09-13T09:12:43.623612+0000","last_active":"2024-09-13T09:13:05.619189+0000","last_peered":"2024-09-13T09:13:05.619189+0000","last_clean":"2024-09-13T09:13:05.619189+0000","last_became_active":"2024-09-13T09:12:43.533088+0000","last_became_peered":"2024-09-13T09:12:43.533088+0000","last_unstale":"2024-09-13T09:13:05.619189+0000","last_undegraded":"2024-09-13T09:13:05.619189+0000","last_fullsized":"2024-09-13T09:13:05.619189+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:43:49.678463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059255799999999997,"stat_sum":{"num_bytes":3072162,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":1465,"num_read_kb":2215,"num_write":396,"num_write_kb":1672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":635004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"68'145","reported_seq":3405,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.232273+0000","last_change":"2024-09-13T09:12:16.389277+0000","last_active":"2024-09-13T09:13:06.232273+0000","last_peered":"2024-09-13T09:13:06.232273+0000","last_clean":"2024-09-13T09:13:06.232273+0000","last_became_active":"2024-09-13T09:12:13.367757+0000","last_became_peered":"2024-09-13T09:12:13.367757+0000","last_unstale":"2024-09-13T09:13:06.232273+0000","last_undegraded":"2024-09-13T09:13:06.232273+0000","last_fullsized":"2024-09-13T09:13:06.232273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:19:33.596156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011263219999999999,"stat_sum":{"num_bytes":19280988,"num_objects":19,"num_object_clones":8,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":3286,"num_read_kb":18892,"num_write":139,"num_write_kb":12161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"68'138","reported_seq":569,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.520920+0000","last_change":"2024-09-13T09:12:43.614382+0000","last_active":"2024-09-13T09:13:06.520920+0000","last_peered":"2024-09-13T09:13:06.520920+0000","last_clean":"2024-09-13T09:13:06.520920+0000","last_became_active":"2024-09-13T09:12:43.532504+0000","last_became_peered":"2024-09-13T09:12:43.532504+0000","last_unstale":"2024-09-13T09:13:06.520920+0000","last_undegraded":"2024-09-13T09:13:06.520920+0000","last_fullsized":"2024-09-13T09:13:06.520920+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:25:04.225563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034391499999999998,"stat_sum":{"num_bytes":11360807,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":1040,"num_read_kb":12205,"num_write":142,"num_write_kb":1684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":694311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"68'376","reported_seq":10615,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.539267+0000","last_change":"2024-09-13T09:12:46.248982+0000","last_active":"2024-09-13T09:13:06.539267+0000","last_peered":"2024-09-13T09:13:06.539267+0000","last_clean":"2024-09-13T09:13:06.539267+0000","last_became_active":"2024-09-13T09:12:46.248261+0000","last_became_peered":"2024-09-13T09:12:46.248261+0000","last_unstale":"2024-09-13T09:13:06.539267+0000","last_undegraded":"2024-09-13T09:13:06.539267+0000","last_fullsized":"2024-09-13T09:13:06.539267+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:44.056762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049261300000000004,"stat_sum":{"num_bytes":20207751,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":11,"num_read":12568,"num_read_kb":33637,"num_write":290,"num_write_kb":4801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":676864,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"68'361","reported_seq":4436,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-13T09:13:06.530563+0000","last_change":"2024-09-13T09:12:46.251913+0000","last_active":"2024-09-13T09:13:06.530563+0000","last_peered":"2024-09-13T09:13:06.530563+0000","last_clean":"2024-09-13T09:13:06.530563+0000","last_became_active":"2024-09-13T09:12:46.250552+0000","last_became_peered":"2024-09-13T09:12:46.250552+0000","last_unstale":"2024-09-13T09:13:06.530563+0000","last_undegraded":"2024-09-13T09:13:06.530563+0000","last_fullsized":"2024-09-13T09:13:06.530563+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:47.142721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059370899999999999,"stat_sum":{"num_bytes":5433449,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":5625,"num_read_kb":5995,"num_write":333,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":28777,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":834079422,"num_objects":751,"num_object_clones":244,"num_object_copies":1502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":751,"num_whiteouts":235,"num_read":157253,"num_read_kb":682973,"num_write":10795,"num_write_kb":189380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":10143567,"num_keys_recovered":33,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":157515776,"data_stored":172667260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9193,"ondisk_log_size":9193,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640273,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76932,"kb_used_data":37620,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94294908,"statfs":{"total":96636764160,"available":96557985792,"internally_reserved":0,"allocated":38522880,"data_stored":38710814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.41799999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.439}]},{"osd":2,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.38500000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.39400000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.40400000000000003}]}]},{"osd":4,"up_from":11,"seq":47244640274,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66184,"kb_used_data":26824,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94305656,"statfs":{"total":96636764160,"available":96568991744,"internally_reserved":0,"allocated":27467776,"data_stored":29587527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.505},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.46300000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.28100000000000003}]},{"osd":5,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.31}]}]},{"osd":3,"up_from":11,"seq":47244640275,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59724,"kb_used_data":20420,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94312116,"statfs":{"total":96636764160,"available":96575606784,"internally_reserved":0,"allocated":20910080,"data_stored":22784269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":2.0310000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":4.3819999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.38100000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":2.9129999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":1.819}]},{"osd":4,"last update":"Fri Sep 13 09:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.312},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.41599999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.32200000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640273,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56988,"kb_used_data":17680,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94314852,"statfs":{"total":96636764160,"available":96578408448,"internally_reserved":0,"allocated":18104320,"data_stored":21402593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:13:01 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.17499999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"last":0.36699999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:13:01 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.38500000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:13:01 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:13:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.55900000000000005}]},{"osd":5,"last update":"Fri Sep 13 09:13:01 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59199999999999997}]}]},{"osd":1,"up_from":11,"seq":47244640274,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60976,"kb_used_data":21668,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94310864,"statfs":{"total":96636764160,"available":96574324736,"internally_reserved":0,"allocated":22188032,"data_stored":24839688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40237509},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.249}]},{"osd":2,"last update":"Fri Sep 13 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.27100000000000002}]},{"osd":3,"last update":"Fri Sep 13 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.45000000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.437}]}]},{"osd":0,"up_from":11,"seq":47244640273,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89004,"kb_used_data":45552,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94282836,"statfs":{"total":96636764160,"available":96545624064,"internally_reserved":0,"allocated":46645248,"data_stored":48844117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44431814},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.30499999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.36799999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.438}]},{"osd":4,"last update":"Fri Sep 13 09:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.48899999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.5}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":43216896,"data_stored":45889897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":21401600,"data_stored":24527913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17321984,"data_stored":21090837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16269312,"data_stored":18620699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":23592960,"data_stored":26173563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35713024,"data_stored":36364351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:13:08.138 INFO:tasks.thrashosds.thrasher:Setting 3.0 to [5, 1] 2024-09-13T09:13:08.138 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.0', '5', '1'] 2024-09-13T09:13:08.138 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.0 5 1 2024-09-13T09:13:08.154 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:08.153+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:08.217 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:08.217 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:08.222 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:08.220+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:08.224 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-13T09:13:08.226 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:08.325 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:08.323+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:08.387 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:08.387 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:08.394 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-09-13T09:13:08.397 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:08.425 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:08.424+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:08.527 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:08.526+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:08.539 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:08.539 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:08.548 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:08.628 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:08.627+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:08.658 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:08.658 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:08.667 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:08.671 INFO:teuthology.orchestra.run.smithi157.stdout:3302 trunc from 0xd4d4600 to 0x8abc800 2024-09-13T09:13:08.699 INFO:teuthology.orchestra.run.smithi157.stdout:3306 punch from 0x3d7ac00 to 0x3d82e00, (0x8200 bytes) 2024-09-13T09:13:08.700 INFO:teuthology.orchestra.run.smithi157.stdout:3307 write 0x5b37a00 thru 0x5b396d6 (0x1cd7 bytes) 2024-09-13T09:13:08.701 INFO:teuthology.orchestra.run.smithi157.stdout:3310 trunc from 0x8abc800 to 0x8526800 2024-09-13T09:13:08.715 INFO:teuthology.orchestra.run.smithi157.stdout:3311 write 0x688d000 thru 0x688e7fd (0x17fe bytes) 2024-09-13T09:13:08.721 INFO:teuthology.orchestra.run.smithi157.stdout:3312 read 0x70b6c00 thru 0x70bdb4a (0x6f4b bytes) 2024-09-13T09:13:08.721 INFO:teuthology.orchestra.run.smithi157.stdout:3314 punch from 0x6b47600 to 0x6b4d400, (0x5e00 bytes) 2024-09-13T09:13:08.722 INFO:teuthology.orchestra.run.smithi157.stdout:3315 write 0x15e2600 thru 0x15e3452 (0xe53 bytes) 2024-09-13T09:13:08.728 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:08.727+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:08.729 INFO:teuthology.orchestra.run.smithi157.stdout:3316 trunc from 0x8526800 to 0xd8f9a00 2024-09-13T09:13:08.759 INFO:teuthology.orchestra.run.smithi157.stdout:3317 write 0xad2cc00 thru 0xad337ef (0x6bf0 bytes) 2024-09-13T09:13:08.759 INFO:teuthology.orchestra.run.smithi157.stdout:3318 punch from 0x8980400 to 0x8983000, (0x2c00 bytes) 2024-09-13T09:13:08.759 INFO:teuthology.orchestra.run.smithi157.stdout:3319 read 0x7bdfa00 thru 0x7be3499 (0x3a9a bytes) 2024-09-13T09:13:08.772 INFO:teuthology.orchestra.run.smithi157.stdout:3321 write 0xe234c00 thru 0xe237ccb (0x30cc bytes) 2024-09-13T09:13:08.782 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:08.782 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:08.786 INFO:teuthology.orchestra.run.smithi157.stdout:3322 trunc from 0xe237e00 to 0x2c6d800 2024-09-13T09:13:08.791 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:08.818 INFO:teuthology.orchestra.run.smithi157.stdout:3323 punch from 0x25a6400 to 0x25af200, (0x8e00 bytes) 2024-09-13T09:13:08.818 INFO:teuthology.orchestra.run.smithi157.stdout:3325 punch from 0x26f1200 to 0x26f5600, (0x4400 bytes) 2024-09-13T09:13:08.819 INFO:teuthology.orchestra.run.smithi157.stdout:3326 read 0x942400 thru 0x95199d (0xf59e bytes) 2024-09-13T09:13:08.829 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:08.828+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:08.837 INFO:teuthology.orchestra.run.smithi157.stdout:3327 write 0xa458200 thru 0xa465799 (0xd59a bytes) 2024-09-13T09:13:08.837 INFO:teuthology.orchestra.run.smithi157.stdout:3328 punch from 0x75ac00 to 0x75e800, (0x3c00 bytes) 2024-09-13T09:13:08.842 INFO:teuthology.orchestra.run.smithi157.stdout:3331 read 0x3467800 thru 0x346a65e (0x2e5f bytes) 2024-09-13T09:13:08.842 INFO:teuthology.orchestra.run.smithi157.stdout:3333 read 0x519f400 thru 0x51a3601 (0x4202 bytes) 2024-09-13T09:13:08.842 INFO:teuthology.orchestra.run.smithi157.stdout:3334 read 0x545fe00 thru 0x54609f6 (0xbf7 bytes) 2024-09-13T09:13:08.842 INFO:teuthology.orchestra.run.smithi157.stdout:3336 read 0x326b600 thru 0x32797d0 (0xe1d1 bytes) 2024-09-13T09:13:08.859 INFO:teuthology.orchestra.run.smithi157.stdout:3337 write 0xb960800 thru 0xb9650df (0x48e0 bytes) 2024-09-13T09:13:08.866 INFO:teuthology.orchestra.run.smithi157.stdout:3338 punch from 0xb82c800 to 0xb831400, (0x4c00 bytes) 2024-09-13T09:13:08.866 INFO:teuthology.orchestra.run.smithi157.stdout:3339 read 0x4410000 thru 0x441815e (0x815f bytes) 2024-09-13T09:13:08.877 INFO:teuthology.orchestra.run.smithi157.stdout:3343 trunc from 0xb965200 to 0x329b200 2024-09-13T09:13:08.906 INFO:teuthology.orchestra.run.smithi157.stdout:3344 trunc from 0x329b200 to 0x9c7b400 2024-09-13T09:13:08.908 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:08.908 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:08.915 INFO:teuthology.orchestra.run.smithi157.stdout:3345 write 0x10e4e00 thru 0x10e52c0 (0x4c1 bytes) 2024-09-13T09:13:08.916 INFO:teuthology.orchestra.run.smithi157.stdout:3346 read 0x8df3c00 thru 0x8dfaa38 (0x6e39 bytes) 2024-09-13T09:13:08.917 INFO:teuthology.orchestra.run.smithi157.stdout:3347 read 0x9a1a400 thru 0x9a1fcb9 (0x58ba bytes) 2024-09-13T09:13:08.919 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:08.921 INFO:teuthology.orchestra.run.smithi157.stdout:3349 trunc from 0x9c7b400 to 0xd147200 2024-09-13T09:13:08.949 INFO:teuthology.orchestra.run.smithi157.stdout:3350 trunc from 0xd147200 to 0x611c800 2024-09-13T09:13:08.959 INFO:teuthology.orchestra.run.smithi157.stdout:3351 read 0x2ce1200 thru 0x2ceab32 (0x9933 bytes) 2024-09-13T09:13:08.960 INFO:teuthology.orchestra.run.smithi157.stdout:3352 read 0x19c8400 thru 0x19cbe9b (0x3a9c bytes) 2024-09-13T09:13:08.963 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:08.961+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:08.968 INFO:teuthology.orchestra.run.smithi157.stdout:3357 trunc from 0x611c800 to 0x112400 2024-09-13T09:13:09.003 INFO:teuthology.orchestra.run.smithi157.stdout:3358 punch from 0xaec00 to 0xb8000, (0x9400 bytes) 2024-09-13T09:13:09.029 INFO:teuthology.orchestra.run.smithi157.stdout:3361 write 0xd7f3200 thru 0xd7ffb38 (0xc939 bytes) 2024-09-13T09:13:09.030 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:09.029+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:09.031 INFO:teuthology.orchestra.run.smithi157.stdout:3362 read 0xbe3ee00 thru 0xbe4b59a (0xc79b bytes) 2024-09-13T09:13:09.031 INFO:teuthology.orchestra.run.smithi157.stdout:3363 write 0x95eea00 thru 0x95f5433 (0x6a34 bytes) 2024-09-13T09:13:09.031 INFO:teuthology.orchestra.run.smithi157.stdout:3369 write 0x8ce7e00 thru 0x8cf66a2 (0xe8a3 bytes) 2024-09-13T09:13:09.031 INFO:teuthology.orchestra.run.smithi157.stdout:3373 read 0x9aff000 thru 0x9b0ae0a (0xbe0b bytes) 2024-09-13T09:13:09.032 INFO:teuthology.orchestra.run.smithi157.stdout:3375 trunc from 0xd7ffc00 to 0xc144000 2024-09-13T09:13:09.037 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:09.037 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:09.046 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:09.050 INFO:teuthology.orchestra.run.smithi157.stdout:3376 punch from 0x9e8fa00 to 0x9e95600, (0x5c00 bytes) 2024-09-13T09:13:09.052 INFO:teuthology.orchestra.run.smithi157.stdout:3379 read 0xa849c00 thru 0xa84ac2a (0x102b bytes) 2024-09-13T09:13:09.052 INFO:teuthology.orchestra.run.smithi157.stdout:3381 punch from 0x14fee00 to 0x150bc00, (0xce00 bytes) 2024-09-13T09:13:09.055 INFO:teuthology.orchestra.run.smithi157.stdout:3382 trunc from 0xc144000 to 0xea65600 2024-09-13T09:13:09.063 INFO:teuthology.orchestra.run.smithi157.stdout:3384 read 0x6c76c00 thru 0x6c77c44 (0x1045 bytes) 2024-09-13T09:13:09.064 INFO:teuthology.orchestra.run.smithi157.stdout:3385 punch from 0x967d200 to 0x967da00, (0x800 bytes) 2024-09-13T09:13:09.065 INFO:teuthology.orchestra.run.smithi157.stdout:3386 trunc from 0xea65600 to 0xdc8a600 2024-09-13T09:13:09.076 INFO:teuthology.orchestra.run.smithi157.stdout:3387 write 0x121e800 thru 0x122dbf3 (0xf3f4 bytes) 2024-09-13T09:13:09.077 INFO:teuthology.orchestra.run.smithi157.stdout:3390 write 0x7962e00 thru 0x796ebec (0xbded bytes) 2024-09-13T09:13:09.078 INFO:teuthology.orchestra.run.smithi157.stdout:3391 trunc from 0xdc8a600 to 0xd986200 2024-09-13T09:13:09.089 INFO:teuthology.orchestra.run.smithi157.stdout:3392 read 0x5519000 thru 0x551ea10 (0x5a11 bytes) 2024-09-13T09:13:09.091 INFO:teuthology.orchestra.run.smithi157.stdout:3395 read 0xb55c600 thru 0xb56855e (0xbf5f bytes) 2024-09-13T09:13:09.091 INFO:teuthology.orchestra.run.smithi157.stdout:3398 punch from 0x7c54800 to 0x7c5da00, (0x9200 bytes) 2024-09-13T09:13:09.091 INFO:teuthology.orchestra.run.smithi157.stdout:3399 write 0x85ebc00 thru 0x85fae2b (0xf22c bytes) 2024-09-13T09:13:09.091 INFO:teuthology.orchestra.run.smithi157.stdout:3400 punch from 0x3dce00 to 0x3e2600, (0x5800 bytes) 2024-09-13T09:13:09.091 INFO:teuthology.orchestra.run.smithi157.stdout:3401 read 0x316ea00 thru 0x317c632 (0xdc33 bytes) 2024-09-13T09:13:09.101 INFO:teuthology.orchestra.run.smithi157.stdout:3403 trunc from 0xd986200 to 0x50bcc00 2024-09-13T09:13:09.128 INFO:teuthology.orchestra.run.smithi157.stdout:3407 trunc from 0x50bcc00 to 0xb0b7200 2024-09-13T09:13:09.130 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:09.129+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:09.144 INFO:teuthology.orchestra.run.smithi157.stdout:3408 trunc from 0xb0b7200 to 0xda43e00 2024-09-13T09:13:09.158 INFO:teuthology.orchestra.run.smithi157.stdout:3409 trunc from 0xda43e00 to 0x5e5d000 2024-09-13T09:13:09.175 INFO:teuthology.orchestra.run.smithi157.stdout:3410 trunc from 0x5e5d000 to 0x4f73e00 2024-09-13T09:13:09.194 INFO:teuthology.orchestra.run.smithi157.stdout:3411 write 0x5fd4400 thru 0x5fdfc85 (0xb886 bytes) 2024-09-13T09:13:09.197 INFO:teuthology.orchestra.run.smithi157.stdout:3415 trunc from 0x5fdfe00 to 0x821fe00 2024-09-13T09:13:09.200 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:09.200 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:09.211 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:09.231 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:09.230+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:09.312 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:09.312 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:09.322 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:09.367 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:09.366+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:09.397 INFO:teuthology.orchestra.run.smithi044.stderr:set 3.0 pg_upmap mapping to [5,1] 2024-09-13T09:13:09.417 INFO:teuthology.orchestra.run.smithi157.stdout:3417 write 0x22fd400 thru 0x2307c6a (0xa86b bytes) 2024-09-13T09:13:09.433 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:09.434 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:09.434 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:09.433+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:09.442 INFO:teuthology.orchestra.run.smithi157.stdout:3418 write 0xc6e4800 thru 0xc6edce4 (0x94e5 bytes) 2024-09-13T09:13:09.443 INFO:teuthology.orchestra.run.smithi157.stdout:3419 write 0x11a8200 thru 0x11b733e (0xf13f bytes) 2024-09-13T09:13:09.443 INFO:teuthology.orchestra.run.smithi157.stdout:3420 read 0x5e11c00 thru 0x5e1a99d (0x8d9e bytes) 2024-09-13T09:13:09.443 INFO:teuthology.orchestra.run.smithi157.stdout:3421 read 0xb13e800 thru 0xb142e01 (0x4602 bytes) 2024-09-13T09:13:09.443 INFO:teuthology.orchestra.run.smithi157.stdout:3423 read 0x4163400 thru 0x41646b4 (0x12b5 bytes) 2024-09-13T09:13:09.443 INFO:teuthology.orchestra.run.smithi157.stdout:3424 write 0x7d92400 thru 0x7d993a5 (0x6fa6 bytes) 2024-09-13T09:13:09.443 INFO:teuthology.orchestra.run.smithi157.stdout:3425 read 0xae3c200 thru 0xae41851 (0x5652 bytes) 2024-09-13T09:13:09.443 INFO:teuthology.orchestra.run.smithi157.stdout:3427 read 0x109f800 thru 0x10a2787 (0x2f88 bytes) 2024-09-13T09:13:09.443 INFO:teuthology.orchestra.run.smithi157.stdout:3429 read 0x51f1400 thru 0x51f8e9c (0x7a9d bytes) 2024-09-13T09:13:09.443 INFO:teuthology.orchestra.run.smithi157.stdout:3434 read 0x25ec00 thru 0x26a3fe (0xb7ff bytes) 2024-09-13T09:13:09.443 INFO:teuthology.orchestra.run.smithi157.stdout:3436 read 0x6635600 thru 0x6639083 (0x3a84 bytes) 2024-09-13T09:13:09.444 INFO:teuthology.orchestra.run.smithi157.stdout:3437 write 0x2a7d000 thru 0x2a7eb49 (0x1b4a bytes) 2024-09-13T09:13:09.444 INFO:teuthology.orchestra.run.smithi157.stdout:3438 read 0x63d400 thru 0x648ce7 (0xb8e8 bytes) 2024-09-13T09:13:09.445 INFO:teuthology.orchestra.run.smithi157.stdout:3440 read 0xac6e00 thru 0xad3a67 (0xcc68 bytes) 2024-09-13T09:13:09.445 INFO:teuthology.orchestra.run.smithi157.stdout:3441 punch from 0x3cd2000 to 0x3cd8000, (0x6000 bytes) 2024-09-13T09:13:09.446 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:09.457 INFO:teuthology.orchestra.run.smithi157.stdout:3442 trunc from 0xc6ede00 to 0x13cf600 2024-09-13T09:13:09.534 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:09.533+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:09.593 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:09.593 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:09.601 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:09.642 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:09.641+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:09.736 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:09.735+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:09.747 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:09.747 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:09.755 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:09.814 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:13:09.840 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:09.839+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:09.905 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:09.905 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:09.914 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:09.938 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:09.936+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:09.963 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:09.963 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:09.980 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:09.981 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:09.996 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:13:09.997 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:13:10.013 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:10.013 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:10.023 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:10.023 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:10.023 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:10.023 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:10.023 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:10.030 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:10.030 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:10.031 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:10.039 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:10.038+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:10.047 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:10.047 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:10.139 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.634013066000001, 2024-09-13T09:13:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:13:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:13:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:13:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:13:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:13:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:13:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:13:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:13:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:13:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:13:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:13:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:13:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:13:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:13:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:13:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.633985381999999, 2024-09-13T09:13:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:13:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:13:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:13:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:13:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:13:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:13:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:13:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:13:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:13:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:13:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:13:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:13:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:13:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.633961040999999, 2024-09-13T09:13:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:13:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:13:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:13:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:13:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:13:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:13:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:13:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:13:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:13:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:13:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:13:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:13:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:13:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:13:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:13:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.633944720999999, 2024-09-13T09:13:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:13:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:13:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:13:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:13:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:13:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:13:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:13:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:13:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:13:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:13:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:13:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:13:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:13:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.633928306000001, 2024-09-13T09:13:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:13:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:13:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:13:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:13:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:13:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:13:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:13:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:13:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:13:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:13:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:13:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:13:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:13:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:13:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.633912198000001, 2024-09-13T09:13:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:13:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:13:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:13:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:13:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:13:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:13:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:13:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:13:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:13:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:13:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:13:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:13:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:13:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:13:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.633896636999999, 2024-09-13T09:13:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:13:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:13:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:13:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:13:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:13:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:13:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:13:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:13:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:13:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:13:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:13:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:13:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:13:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:13:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.633879798999999, 2024-09-13T09:13:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:13:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:13:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:13:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:13:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:13:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:13:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:13:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:13:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:13:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:13:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:13:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:13:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:13:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:13:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:13:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.633863807000001, 2024-09-13T09:13:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:13:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:13:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:13:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:13:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:13:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:13:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:13:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:13:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:13:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:13:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:13:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:13:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:13:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:13:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:13:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.633836308999999, 2024-09-13T09:13:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:13:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:13:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:13:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:13:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:13:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:13:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:13:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:13:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:13:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:13:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:13:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:13:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:13:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:13:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.633810449999999, 2024-09-13T09:13:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:13:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:13:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:13:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:13:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:13:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:13:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:13:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:13:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:13:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:13:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:13:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:13:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:13:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:13:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.633768960000001, 2024-09-13T09:13:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:13:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:13:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:13:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:13:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:13:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:13:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:13:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:13:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:13:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:13:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:13:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:13:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:13:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:13:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:13:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.633747721999999, 2024-09-13T09:13:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:13:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:13:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:13:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:13:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:13:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:13:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:13:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:13:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:13:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:13:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:13:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:13:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:13:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:13:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.633724921999999, 2024-09-13T09:13:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:13:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:13:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:13:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:13:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:13:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:13:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:13:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:13:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:13:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:13:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:13:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:13:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:13:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:13:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.633708466000002, 2024-09-13T09:13:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:13:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:13:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:13:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:13:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:13:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:13:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:13:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:13:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:13:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:13:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:13:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:13:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:13:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:13:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.633463329000001, 2024-09-13T09:13:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:13:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:13:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.212 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:10.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.212 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:10.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.212 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:13:10.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:13:10.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.213 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:13:10.213 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:13:10.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.213 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:13:10.213 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:13:10.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.213 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:13:10.213 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:10.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:13:10.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:13:10.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:13:10.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:13:10.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:13:10.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:13:10.214 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.215 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.215 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20408 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5582848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.215 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.505992+0000", 2024-09-13T09:13:10.215 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.631598748999998, 2024-09-13T09:13:10.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050463591000000002, 2024-09-13T09:13:10.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.215 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.215 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.215 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.215 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.215 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20408 2024-09-13T09:13:10.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.216 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505992+0000", 2024-09-13T09:13:10.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505992+0000", 2024-09-13T09:13:10.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505991+0000", 2024-09-13T09:13:10.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.217 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.217 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.505994+0000", 2024-09-13T09:13:10.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6259999999999999e-06 2024-09-13T09:13:10.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.217 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.217 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506002+0000", 2024-09-13T09:13:10.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1550000000000007e-06 2024-09-13T09:13:10.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.217 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506008+0000", 2024-09-13T09:13:10.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.553e-06 2024-09-13T09:13:10.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556375+0000", 2024-09-13T09:13:10.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050366850999999997 2024-09-13T09:13:10.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556412+0000", 2024-09-13T09:13:10.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7138999999999998e-05 2024-09-13T09:13:10.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556455+0000", 2024-09-13T09:13:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3733999999999998e-05 2024-09-13T09:13:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.63157777, 2024-09-13T09:13:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:13:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.220 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.220 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.220 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:13:10.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.220 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:10.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:10.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:13:10.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:13:10.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:13:10.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:13:10.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:13:10.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.222 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:13:10.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:13:10.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.222 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:13:10.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:13:10.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.222 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:13:10.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:13:10.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.223 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:13:10.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:13:10.223 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.223 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.223 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.223 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.223 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:10.223 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.631556404000001, 2024-09-13T09:13:10.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:13:10.224 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.224 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.224 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.224 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.224 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.224 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:13:10.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.224 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:10.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:10.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:10.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:13:10.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:13:10.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:13:10.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:13:10.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:13:10.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:13:10.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:13:10.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:13:10.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.227 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.227 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:13:10.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:13:10.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.227 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.227 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:13:10.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:13:10.227 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.227 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.227 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.228 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:10.228 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:10.228 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.631351036000002, 2024-09-13T09:13:10.228 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:13:10.228 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.228 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.228 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.228 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.228 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.228 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:13:10.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.228 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.229 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.229 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:10.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.229 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.229 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:10.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.229 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.229 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:13:10.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:13:10.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.230 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.230 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:13:10.230 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:13:10.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.230 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.230 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:13:10.230 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:13:10.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.230 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.230 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:13:10.230 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:13:10.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:13:10.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:13:10.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:13:10.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:13:10.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:13:10.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:13:10.232 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.232 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.232 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.232 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.232 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.232 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:10.232 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:10.138+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:10.233 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:13:10.240 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:10.239+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:10.341 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:10.339+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:10.380 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:10.380 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:10.381 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:10.381 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:10.388 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:13:10.412 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:13:10.442 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:10.439+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:10.496 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:10.496 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:10.496 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:10.496 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:10.496 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:10.505 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:13:10.549 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:10.548+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:10.615 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:6867 3.13 3:cd293326:::rbd_header.112326fb43d3:head [watch reconnect cookie 140383087120544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T09:13:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:13:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.084506628, 2024-09-13T09:13:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0277181360000001, 2024-09-13T09:13:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:13:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:13:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6867 2024-09-13T09:13:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:13:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:13:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529067+0000", 2024-09-13T09:13:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-09-13T09:13:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529073+0000", 2024-09-13T09:13:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:13:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529087+0000", 2024-09-13T09:13:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3825000000000001e-05 2024-09-13T09:13:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529097+0000", 2024-09-13T09:13:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9559999999999992e-06 2024-09-13T09:13:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529149+0000", 2024-09-13T09:13:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1668000000000001e-05 2024-09-13T09:13:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529161+0000", 2024-09-13T09:13:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1831000000000001e-05 2024-09-13T09:13:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.554941+0000", 2024-09-13T09:13:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0257799729999999 2024-09-13T09:13:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.555239+0000", 2024-09-13T09:13:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00029829100000000001 2024-09-13T09:13:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.556787+0000", 2024-09-13T09:13:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001547617 2024-09-13T09:13:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20898 3.13 3:cd293326:::rbd_header.112326fb43d3:head [watch reconnect cookie 140642395970896 gen 16] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T09:13:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:13:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.07984381, 2024-09-13T09:13:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0236151950000001, 2024-09-13T09:13:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20898 2024-09-13T09:13:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:13:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:13:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533729+0000", 2024-09-13T09:13:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-09-13T09:13:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533736+0000", 2024-09-13T09:13:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9670000000000004e-06 2024-09-13T09:13:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533749+0000", 2024-09-13T09:13:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2449000000000001e-05 2024-09-13T09:13:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533757+0000", 2024-09-13T09:13:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:13:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533813+0000", 2024-09-13T09:13:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5683000000000002e-05 2024-09-13T09:13:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533823+0000", 2024-09-13T09:13:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0168e-05 2024-09-13T09:13:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.555940+0000", 2024-09-13T09:13:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0221170559999999 2024-09-13T09:13:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.556047+0000", 2024-09-13T09:13:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000106881 2024-09-13T09:13:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.557347+0000", 2024-09-13T09:13:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001299323 2024-09-13T09:13:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5385 3.1e 3:79106327:::rbd_data.1123b428d19a.0000000000000009:16 [read 1216512~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.706521+0000", 2024-09-13T09:13:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9070545689999996, 2024-09-13T09:13:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037021376000000002, 2024-09-13T09:13:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5385 2024-09-13T09:13:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706521+0000", 2024-09-13T09:13:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706521+0000", 2024-09-13T09:13:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706520+0000", 2024-09-13T09:13:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706522+0000", 2024-09-13T09:13:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0140000000000001e-06 2024-09-13T09:13:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706527+0000", 2024-09-13T09:13:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1959999999999999e-06 2024-09-13T09:13:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706533+0000", 2024-09-13T09:13:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2839999999999997e-06 2024-09-13T09:13:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.743365+0000", 2024-09-13T09:13:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.036832070000000001 2024-09-13T09:13:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.743451+0000", 2024-09-13T09:13:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6601000000000007e-05 2024-09-13T09:13:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.743542+0000", 2024-09-13T09:13:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0556999999999994e-05 2024-09-13T09:13:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5393 3.1e 3:7c501324:::rbd_data.1123b428d19a.000000000000000d:16 [read 1224704~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.706575+0000", 2024-09-13T09:13:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9069997560000003, 2024-09-13T09:13:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037089817999999997, 2024-09-13T09:13:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5393 2024-09-13T09:13:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706575+0000", 2024-09-13T09:13:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706575+0000", 2024-09-13T09:13:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706575+0000", 2024-09-13T09:13:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706577+0000", 2024-09-13T09:13:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1600000000000001e-06 2024-09-13T09:13:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706583+0000", 2024-09-13T09:13:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4280000000000004e-06 2024-09-13T09:13:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706588+0000", 2024-09-13T09:13:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5720000000000002e-06 2024-09-13T09:13:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.743571+0000", 2024-09-13T09:13:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.036982899 2024-09-13T09:13:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.743615+0000", 2024-09-13T09:13:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4252999999999998e-05 2024-09-13T09:13:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.743665+0000", 2024-09-13T09:13:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9923999999999998e-05 2024-09-13T09:13:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5394 3.1e 3:79106327:::rbd_data.1123b428d19a.0000000000000009:16 [read 1228800~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.706593+0000", 2024-09-13T09:13:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9069825659999999, 2024-09-13T09:13:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037289958999999998, 2024-09-13T09:13:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5394 2024-09-13T09:13:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706593+0000", 2024-09-13T09:13:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706593+0000", 2024-09-13T09:13:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706592+0000", 2024-09-13T09:13:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706594+0000", 2024-09-13T09:13:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.153e-06 2024-09-13T09:13:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706600+0000", 2024-09-13T09:13:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8050000000000003e-06 2024-09-13T09:13:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706605+0000", 2024-09-13T09:13:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4850000000000003e-06 2024-09-13T09:13:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.743695+0000", 2024-09-13T09:13:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037090426000000003 2024-09-13T09:13:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.743782+0000", 2024-09-13T09:13:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6426000000000002e-05 2024-09-13T09:13:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.743883+0000", 2024-09-13T09:13:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010100200000000001 2024-09-13T09:13:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5399 3.1e 3:7c501324:::rbd_data.1123b428d19a.000000000000000d:16 [read 1236992~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.706620+0000", 2024-09-13T09:13:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9069548330000003, 2024-09-13T09:13:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037379786999999998, 2024-09-13T09:13:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5399 2024-09-13T09:13:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706620+0000", 2024-09-13T09:13:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706620+0000", 2024-09-13T09:13:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706620+0000", 2024-09-13T09:13:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706621+0000", 2024-09-13T09:13:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.173e-06 2024-09-13T09:13:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706624+0000", 2024-09-13T09:13:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:13:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706627+0000", 2024-09-13T09:13:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.345e-06 2024-09-13T09:13:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.743909+0000", 2024-09-13T09:13:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037282514000000003 2024-09-13T09:13:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.743951+0000", 2024-09-13T09:13:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2329000000000001e-05 2024-09-13T09:13:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744000+0000", 2024-09-13T09:13:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8696000000000003e-05 2024-09-13T09:13:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5406 3.1e 3:79106327:::rbd_data.1123b428d19a.0000000000000009:16 [read 1232896~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.706637+0000", 2024-09-13T09:13:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9069380730000001, 2024-09-13T09:13:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037571879000000002, 2024-09-13T09:13:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5406 2024-09-13T09:13:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706637+0000", 2024-09-13T09:13:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706637+0000", 2024-09-13T09:13:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706637+0000", 2024-09-13T09:13:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706638+0000", 2024-09-13T09:13:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.181e-06 2024-09-13T09:13:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706641+0000", 2024-09-13T09:13:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7839999999999999e-06 2024-09-13T09:13:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706645+0000", 2024-09-13T09:13:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7900000000000001e-06 2024-09-13T09:13:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744030+0000", 2024-09-13T09:13:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037385319 2024-09-13T09:13:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744117+0000", 2024-09-13T09:13:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6981e-05 2024-09-13T09:13:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744209+0000", 2024-09-13T09:13:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2026000000000003e-05 2024-09-13T09:13:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5409 3.1e 3:79106327:::rbd_data.1123b428d19a.0000000000000009:16 [read 1236992~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.706648+0000", 2024-09-13T09:13:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9069269709999999, 2024-09-13T09:13:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037676726000000001, 2024-09-13T09:13:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5409 2024-09-13T09:13:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706648+0000", 2024-09-13T09:13:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706648+0000", 2024-09-13T09:13:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706648+0000", 2024-09-13T09:13:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706650+0000", 2024-09-13T09:13:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:13:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706655+0000", 2024-09-13T09:13:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3739999999999997e-06 2024-09-13T09:13:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706660+0000", 2024-09-13T09:13:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8060000000000002e-06 2024-09-13T09:13:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744234+0000", 2024-09-13T09:13:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037574112 2024-09-13T09:13:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744276+0000", 2024-09-13T09:13:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1715000000000003e-05 2024-09-13T09:13:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744325+0000", 2024-09-13T09:13:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9075e-05 2024-09-13T09:13:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5412 3.1e 3:7c501324:::rbd_data.1123b428d19a.000000000000000d:16 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.706666+0000", 2024-09-13T09:13:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9069089769999996, 2024-09-13T09:13:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037868800000000001, 2024-09-13T09:13:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5412 2024-09-13T09:13:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706666+0000", 2024-09-13T09:13:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706666+0000", 2024-09-13T09:13:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706666+0000", 2024-09-13T09:13:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706668+0000", 2024-09-13T09:13:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0760000000000001e-06 2024-09-13T09:13:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706674+0000", 2024-09-13T09:13:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9689999999999999e-06 2024-09-13T09:13:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706680+0000", 2024-09-13T09:13:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9839999999999998e-06 2024-09-13T09:13:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744356+0000", 2024-09-13T09:13:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037675820999999998 2024-09-13T09:13:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744441+0000", 2024-09-13T09:13:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5864000000000003e-05 2024-09-13T09:13:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744535+0000", 2024-09-13T09:13:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3471e-05 2024-09-13T09:13:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5419 3.1e 3:79106327:::rbd_data.1123b428d19a.0000000000000009:16 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.706701+0000", 2024-09-13T09:13:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.906873794, 2024-09-13T09:13:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037962826999999998, 2024-09-13T09:13:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5419 2024-09-13T09:13:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706701+0000", 2024-09-13T09:13:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706701+0000", 2024-09-13T09:13:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706701+0000", 2024-09-13T09:13:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706703+0000", 2024-09-13T09:13:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9470000000000002e-06 2024-09-13T09:13:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706709+0000", 2024-09-13T09:13:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:13:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706714+0000", 2024-09-13T09:13:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:13:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744560+0000", 2024-09-13T09:13:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037845573 2024-09-13T09:13:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744602+0000", 2024-09-13T09:13:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2026e-05 2024-09-13T09:13:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744664+0000", 2024-09-13T09:13:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2611000000000001e-05 2024-09-13T09:13:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5445 3.1e 3:7c501324:::rbd_data.1123b428d19a.000000000000000d:16 [read 1228800~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.706777+0000", 2024-09-13T09:13:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9067976389999997, 2024-09-13T09:13:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.038105026, 2024-09-13T09:13:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5445 2024-09-13T09:13:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706777+0000", 2024-09-13T09:13:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706777+0000", 2024-09-13T09:13:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706777+0000", 2024-09-13T09:13:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706779+0000", 2024-09-13T09:13:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0200000000000001e-06 2024-09-13T09:13:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706784+0000", 2024-09-13T09:13:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3859999999999998e-06 2024-09-13T09:13:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706791+0000", 2024-09-13T09:13:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4450000000000001e-06 2024-09-13T09:13:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744695+0000", 2024-09-13T09:13:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037904265999999999 2024-09-13T09:13:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744783+0000", 2024-09-13T09:13:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7703999999999999e-05 2024-09-13T09:13:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744883+0000", 2024-09-13T09:13:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9610000000000003e-05 2024-09-13T09:13:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5455 3.1e 3:7c501324:::rbd_data.1123b428d19a.000000000000000d:16 [read 1232896~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.706939+0000", 2024-09-13T09:13:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9066356669999998, 2024-09-13T09:13:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.038058952, 2024-09-13T09:13:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5455 2024-09-13T09:13:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706939+0000", 2024-09-13T09:13:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706939+0000", 2024-09-13T09:13:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706838+0000", 2024-09-13T09:13:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998989 2024-09-13T09:13:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706943+0000", 2024-09-13T09:13:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000104493 2024-09-13T09:13:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706951+0000", 2024-09-13T09:13:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3100000000000001e-06 2024-09-13T09:13:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.706957+0000", 2024-09-13T09:13:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4e-06 2024-09-13T09:13:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744908+0000", 2024-09-13T09:13:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037950976999999997 2024-09-13T09:13:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744950+0000", 2024-09-13T09:13:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2224000000000001e-05 2024-09-13T09:13:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.744998+0000", 2024-09-13T09:13:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8640999999999997e-05 2024-09-13T09:13:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5485 3.1e 3:79106327:::rbd_data.1123b428d19a.0000000000000009:16 [read 1245184~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.708956+0000", 2024-09-13T09:13:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9046188810000002, 2024-09-13T09:13:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.036900167999999997, 2024-09-13T09:13:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5485 2024-09-13T09:13:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708956+0000", 2024-09-13T09:13:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708956+0000", 2024-09-13T09:13:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708956+0000", 2024-09-13T09:13:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708957+0000", 2024-09-13T09:13:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.19e-06 2024-09-13T09:13:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708960+0000", 2024-09-13T09:13:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7829999999999999e-06 2024-09-13T09:13:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708962+0000", 2024-09-13T09:13:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4260000000000002e-06 2024-09-13T09:13:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.745670+0000", 2024-09-13T09:13:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.036707904 2024-09-13T09:13:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.745757+0000", 2024-09-13T09:13:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6681999999999994e-05 2024-09-13T09:13:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.745856+0000", 2024-09-13T09:13:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9380000000000001e-05 2024-09-13T09:13:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5486 3.1e 3:79106327:::rbd_data.1123b428d19a.0000000000000009:16 [read 1249280~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.708966+0000", 2024-09-13T09:13:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9046095000000003, 2024-09-13T09:13:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037013163000000002, 2024-09-13T09:13:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5486 2024-09-13T09:13:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708966+0000", 2024-09-13T09:13:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708966+0000", 2024-09-13T09:13:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708965+0000", 2024-09-13T09:13:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708967+0000", 2024-09-13T09:13:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1289999999999999e-06 2024-09-13T09:13:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708971+0000", 2024-09-13T09:13:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9910000000000004e-06 2024-09-13T09:13:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708973+0000", 2024-09-13T09:13:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.351e-06 2024-09-13T09:13:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.745883+0000", 2024-09-13T09:13:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.036909810000000001 2024-09-13T09:13:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.745927+0000", 2024-09-13T09:13:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4233999999999997e-05 2024-09-13T09:13:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.745979+0000", 2024-09-13T09:13:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1853e-05 2024-09-13T09:13:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5491 3.1e 3:7c501324:::rbd_data.1123b428d19a.000000000000000d:16 [read 1253376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.708977+0000", 2024-09-13T09:13:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.904598494, 2024-09-13T09:13:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037212332000000001, 2024-09-13T09:13:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5491 2024-09-13T09:13:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708977+0000", 2024-09-13T09:13:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708977+0000", 2024-09-13T09:13:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708976+0000", 2024-09-13T09:13:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708977+0000", 2024-09-13T09:13:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.102e-06 2024-09-13T09:13:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708980+0000", 2024-09-13T09:13:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8880000000000001e-06 2024-09-13T09:13:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708983+0000", 2024-09-13T09:13:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9019999999999999e-06 2024-09-13T09:13:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746010+0000", 2024-09-13T09:13:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037026934999999997 2024-09-13T09:13:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746097+0000", 2024-09-13T09:13:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6970000000000002e-05 2024-09-13T09:13:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746189+0000", 2024-09-13T09:13:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1768000000000006e-05 2024-09-13T09:13:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5492 3.1e 3:7c501324:::rbd_data.1123b428d19a.000000000000000d:16 [read 1257472~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.708985+0000", 2024-09-13T09:13:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9045897390000004, 2024-09-13T09:13:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037320814000000001, 2024-09-13T09:13:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5492 2024-09-13T09:13:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708985+0000", 2024-09-13T09:13:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708985+0000", 2024-09-13T09:13:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708985+0000", 2024-09-13T09:13:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708986+0000", 2024-09-13T09:13:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1209999999999999e-06 2024-09-13T09:13:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708989+0000", 2024-09-13T09:13:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7410000000000001e-06 2024-09-13T09:13:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.708991+0000", 2024-09-13T09:13:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:13:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746213+0000", 2024-09-13T09:13:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037221716000000002 2024-09-13T09:13:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746255+0000", 2024-09-13T09:13:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2336999999999998e-05 2024-09-13T09:13:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746306+0000", 2024-09-13T09:13:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0816999999999998e-05 2024-09-13T09:13:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5499 3.1e 3:79106327:::rbd_data.1123b428d19a.0000000000000009:16 [read 1253376~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.709010+0000", 2024-09-13T09:13:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9045648010000003, 2024-09-13T09:13:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037505745, 2024-09-13T09:13:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5499 2024-09-13T09:13:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709010+0000", 2024-09-13T09:13:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709010+0000", 2024-09-13T09:13:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709010+0000", 2024-09-13T09:13:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709011+0000", 2024-09-13T09:13:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1000000000000001e-06 2024-09-13T09:13:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709015+0000", 2024-09-13T09:13:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.569e-06 2024-09-13T09:13:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709017+0000", 2024-09-13T09:13:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:13:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746337+0000", 2024-09-13T09:13:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037319934999999999 2024-09-13T09:13:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746424+0000", 2024-09-13T09:13:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6469000000000002e-05 2024-09-13T09:13:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746516+0000", 2024-09-13T09:13:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2540999999999994e-05 2024-09-13T09:13:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5507 3.1e 3:79106327:::rbd_data.1123b428d19a.0000000000000009:16 [read 1257472~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.709019+0000", 2024-09-13T09:13:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9045559159999996, 2024-09-13T09:13:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037614994999999998, 2024-09-13T09:13:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5507 2024-09-13T09:13:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709019+0000", 2024-09-13T09:13:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709019+0000", 2024-09-13T09:13:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709019+0000", 2024-09-13T09:13:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709023+0000", 2024-09-13T09:13:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7890000000000001e-06 2024-09-13T09:13:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709026+0000", 2024-09-13T09:13:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.117e-06 2024-09-13T09:13:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709028+0000", 2024-09-13T09:13:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4420000000000001e-06 2024-09-13T09:13:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746540+0000", 2024-09-13T09:13:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037511418999999997 2024-09-13T09:13:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746584+0000", 2024-09-13T09:13:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4702e-05 2024-09-13T09:13:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746634+0000", 2024-09-13T09:13:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9725999999999998e-05 2024-09-13T09:13:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5516 3.1e 3:79106327:::rbd_data.1123b428d19a.0000000000000009:16 [read 1261568~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.709030+0000", 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9045446400000001, 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037820399999999997, 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5516 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709030+0000", 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709030+0000", 2024-09-13T09:13:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709030+0000", 2024-09-13T09:13:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-09-13T09:13:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709031+0000", 2024-09-13T09:13:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-06 2024-09-13T09:13:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709034+0000", 2024-09-13T09:13:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7819999999999998e-06 2024-09-13T09:13:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709038+0000", 2024-09-13T09:13:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2470000000000002e-06 2024-09-13T09:13:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746665+0000", 2024-09-13T09:13:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037626661999999998 2024-09-13T09:13:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746751+0000", 2024-09-13T09:13:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6321000000000002e-05 2024-09-13T09:13:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746851+0000", 2024-09-13T09:13:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9457999999999995e-05 2024-09-13T09:13:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5036.0:5519 3.1e 3:79106327:::rbd_data.1123b428d19a.0000000000000009:16 [read 1265664~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-13T09:13:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:05.709044+0000", 2024-09-13T09:13:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.9045311710000004, 2024-09-13T09:13:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037918970000000003, 2024-09-13T09:13:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5036", 2024-09-13T09:13:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4010752148", 2024-09-13T09:13:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 5519 2024-09-13T09:13:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709044+0000", 2024-09-13T09:13:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709044+0000", 2024-09-13T09:13:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709043+0000", 2024-09-13T09:13:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709046+0000", 2024-09-13T09:13:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2189999999999998e-06 2024-09-13T09:13:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709050+0000", 2024-09-13T09:13:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7620000000000003e-06 2024-09-13T09:13:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.709055+0000", 2024-09-13T09:13:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:13:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746868+0000", 2024-09-13T09:13:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.037812748 2024-09-13T09:13:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746911+0000", 2024-09-13T09:13:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2926999999999997e-05 2024-09-13T09:13:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:05.746963+0000", 2024-09-13T09:13:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2182000000000003e-05 2024-09-13T09:13:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:10.710 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:10.712 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:10.712 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:10.710+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:10.813 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:10.811+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:10.827 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:10.827 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:10.827 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:10.827 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:10.836 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:10.914 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:10.913+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:10.980 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:10.980 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:10.980 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:10.980 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:10.980 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:10.988 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:11.016 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:11.015+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:11.092 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:11.092 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:11.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:11.092 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:11.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.092 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.093 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:11.093 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568949607, 2024-09-13T09:13:11.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:13:11.093 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.093 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.093 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.093 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.093 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.093 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:13:11.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.094 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:11.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:11.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:11.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:13:11.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:13:11.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:13:11.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:13:11.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:13:11.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:13:11.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:13:11.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:13:11.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:13:11.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:13:11.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:13:11.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:13:11.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:13:11.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:13:11.099 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.099 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.099 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.100 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.100 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:11.100 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568900550999999, 2024-09-13T09:13:11.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:13:11.100 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.100 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.100 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.100 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.100 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.100 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:13:11.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.101 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:11.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:11.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:11.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:13:11.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:13:11.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:13:11.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:13:11.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:13:11.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:13:11.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:13:11.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:13:11.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:13:11.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:13:11.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:13:11.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:13:11.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:13:11.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:13:11.105 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.105 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.105 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.105 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.105 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:11.105 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568860476000001, 2024-09-13T09:13:11.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:13:11.106 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.106 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.106 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.106 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.106 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.106 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:13:11.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.106 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:11.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:11.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:11.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:13:11.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:13:11.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:13:11.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:13:11.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:13:11.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:13:11.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:13:11.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:13:11.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:13:11.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:13:11.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:13:11.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:13:11.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:13:11.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:13:11.110 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.110 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.110 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.110 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.110 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:11.110 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568830661, 2024-09-13T09:13:11.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:13:11.111 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.111 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.111 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.111 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.111 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.111 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:13:11.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.111 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:11.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:11.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:11.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:13:11.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:13:11.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:13:11.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:11.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:13:11.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:13:11.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:13:11.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:13:11.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:13:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:13:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:13:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:13:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:13:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:13:11.115 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.116 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.116 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.116 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.116 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:11.116 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568801000000001, 2024-09-13T09:13:11.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:13:11.116 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.116 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.117 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:13:11.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.117 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:11.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:11.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:11.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:13:11.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:13:11.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:13:11.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:13:11.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:13:11.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:13:11.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:13:11.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:13:11.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:13:11.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:13:11.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:13:11.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:13:11.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:13:11.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:13:11.120 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.121 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.121 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.121 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.121 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:11.121 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568771793, 2024-09-13T09:13:11.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:13:11.121 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.121 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.122 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.122 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:13:11.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.122 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:11.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:11.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:11.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:13:11.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:13:11.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:13:11.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:13:11.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:13:11.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:13:11.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:13:11.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:13:11.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:13:11.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:13:11.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:13:11.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:13:11.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:13:11.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:13:11.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.126 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.126 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.126 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.126 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:11.126 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568715611999998, 2024-09-13T09:13:11.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:13:11.126 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.126 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.126 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:13:11.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.127 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:11.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:11.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:11.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:13:11.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:13:11.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:13:11.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:13:11.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:13:11.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:13:11.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:13:11.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:13:11.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:13:11.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:13:11.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:13:11.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:13:11.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:13:11.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:13:11.131 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.131 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.131 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.131 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.132 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:11.132 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568662151000002, 2024-09-13T09:13:11.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:13:11.132 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.132 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.132 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.132 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.132 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.132 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:13:11.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.132 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:11.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:11.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:13:11.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:13:11.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:13:11.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:13:11.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:13:11.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:13:11.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:13:11.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:13:11.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:13:11.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:13:11.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:13:11.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:13:11.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:13:11.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:13:11.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:13:11.137 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.137 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.137 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.137 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.137 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:11.137 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568605321, 2024-09-13T09:13:11.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:13:11.138 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.138 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.138 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.138 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.138 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.139 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:13:11.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.139 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:11.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:11.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:13:11.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:13:11.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:13:11.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:13:11.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:13:11.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:13:11.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:13:11.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:13:11.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:13:11.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:13:11.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:13:11.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:13:11.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:13:11.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:13:11.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:13:11.143 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.143 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.143 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.143 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.143 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:11.143 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568554293999998, 2024-09-13T09:13:11.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:13:11.144 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.144 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.144 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:13:11.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.144 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:11.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:11.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:13:11.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:13:11.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:13:11.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:13:11.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:13:11.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:13:11.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:13:11.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:13:11.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:13:11.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:13:11.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:13:11.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:13:11.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:13:11.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:13:11.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:13:11.148 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.148 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.148 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.148 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.148 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:11.148 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568513104000001, 2024-09-13T09:13:11.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:13:11.148 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.149 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.149 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.149 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.149 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.149 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:13:11.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.149 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:11.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:11.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:11.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:13:11.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:13:11.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:13:11.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:13:11.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:13:11.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:13:11.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:13:11.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:13:11.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:13:11.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:13:11.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:13:11.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:13:11.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:13:11.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:13:11.153 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.153 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.153 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.153 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.153 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:11.153 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568470988000001, 2024-09-13T09:13:11.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:13:11.153 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.153 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.153 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.153 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.154 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.154 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:13:11.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.154 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:11.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:11.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:13:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:13:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:13:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:13:11.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:13:11.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:13:11.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:13:11.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:13:11.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:13:11.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:13:11.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:13:11.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:13:11.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:13:11.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:13:11.157 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.158 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.158 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.158 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.158 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:11.158 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568427109000002, 2024-09-13T09:13:11.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:13:11.158 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.158 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.158 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.158 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.158 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.158 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:13:11.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.159 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:11.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:11.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:11.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:11.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:13:11.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:13:11.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:13:11.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:13:11.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:13:11.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:13:11.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:13:11.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:13:11.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:13:11.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:13:11.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:13:11.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:13:11.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:13:11.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:13:11.162 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.162 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.162 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.163 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.163 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:11.163 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568381192, 2024-09-13T09:13:11.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:13:11.163 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.163 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.163 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.163 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.164 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.164 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:13:11.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.164 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:11.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:11.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:13:11.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:13:11.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:13:11.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:13:11.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:13:11.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:13:11.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:13:11.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:13:11.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:13:11.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:13:11.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:13:11.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:13:11.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:13:11.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:13:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:13:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568331421, 2024-09-13T09:13:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:13:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.168 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.169 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:13:11.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.169 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:11.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:11.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:11.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:11.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:13:11.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:13:11.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:13:11.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:13:11.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:13:11.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:13:11.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:13:11.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:13:11.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:13:11.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:13:11.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:13:11.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:13:11.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:13:11.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:13:11.172 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.172 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.173 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.173 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.173 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:11.173 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568286016999998, 2024-09-13T09:13:11.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:13:11.173 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.173 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.173 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.173 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.173 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.173 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:13:11.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.174 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:11.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:11.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:11.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:11.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:13:11.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:13:11.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:13:11.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:13:11.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:13:11.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:13:11.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:13:11.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:13:11.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:13:11.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:13:11.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:13:11.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:13:11.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:13:11.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:13:11.177 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.177 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.177 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.178 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.178 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:11.178 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568235379000001, 2024-09-13T09:13:11.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:13:11.178 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.178 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.178 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.178 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.178 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.178 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:13:11.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.178 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:11.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:11.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:13:11.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:13:11.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:13:11.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:13:11.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:13:11.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:13:11.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:13:11.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:13:11.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:13:11.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:13:11.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:13:11.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:13:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:13:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:13:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:13:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568179535999999, 2024-09-13T09:13:11.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:13:11.183 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.183 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.183 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.183 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.183 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.183 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:13:11.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.183 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:11.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:11.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:11.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:13:11.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:13:11.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:13:11.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:13:11.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:13:11.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:13:11.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:13:11.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:13:11.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:13:11.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:13:11.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:13:11.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:13:11.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:13:11.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:13:11.187 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.187 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.187 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.187 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.187 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:11.187 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568122614, 2024-09-13T09:13:11.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:13:11.187 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.187 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.188 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.188 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:13:11.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.188 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:11.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:11.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:11.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:11.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:13:11.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:13:11.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:13:11.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:13:11.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:13:11.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:13:11.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:13:11.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:13:11.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:13:11.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:13:11.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:13:11.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:13:11.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:13:11.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:13:11.192 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.192 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.192 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.192 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:11.192 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:11.192 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.568069050999998, 2024-09-13T09:13:11.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:13:11.192 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.192 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:11.192 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.192 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.193 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.193 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:13:11.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.193 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:11.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:11.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:11.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:13:11.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:13:11.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:13:11.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:13:11.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:13:11.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:13:11.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:13:11.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:13:11.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:11.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:13:11.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:13:11.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:13:11.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:13:11.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:13:11.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:13:11.196 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.196 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.196 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.196 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.197 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.197 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:11.198 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:11.199 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:11.197+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:11.246 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-09-13T09:13:11.299 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:11.297+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:11.302 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:11.302 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:11.302 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:11.302 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:11.311 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:11.400 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:11.398+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:11.456 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:11.456 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:11.456 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:11.456 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:11.456 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:11.464 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:11.507 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:11.505+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:11.602 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:11.600+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:11.611 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:11.611 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:11.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:11.611 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:11.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.611 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17752 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2125824~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.611 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138424+0000", 2024-09-13T09:13:11.611 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.470278414999999, 2024-09-13T09:13:11.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045520176000000002, 2024-09-13T09:13:11.611 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.611 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.612 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17752 2024-09-13T09:13:11.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.612 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138424+0000", 2024-09-13T09:13:11.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138424+0000", 2024-09-13T09:13:11.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138423+0000", 2024-09-13T09:13:11.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138425+0000", 2024-09-13T09:13:11.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0640000000000001e-06 2024-09-13T09:13:11.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138431+0000", 2024-09-13T09:13:11.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0900000000000001e-06 2024-09-13T09:13:11.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138437+0000", 2024-09-13T09:13:11.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0329999999999997e-06 2024-09-13T09:13:11.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183854+0000", 2024-09-13T09:13:11.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045416099000000001 2024-09-13T09:13:11.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183894+0000", 2024-09-13T09:13:11.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0935e-05 2024-09-13T09:13:11.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183944+0000", 2024-09-13T09:13:11.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9329999999999997e-05 2024-09-13T09:13:11.615 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.615 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.615 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.616 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17753 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2148352~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.616 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138442+0000", 2024-09-13T09:13:11.616 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.470259693999999, 2024-09-13T09:13:11.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045633606, 2024-09-13T09:13:11.616 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.616 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.616 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.616 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.616 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.616 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17753 2024-09-13T09:13:11.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.616 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138442+0000", 2024-09-13T09:13:11.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138442+0000", 2024-09-13T09:13:11.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138441+0000", 2024-09-13T09:13:11.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:13:11.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138444+0000", 2024-09-13T09:13:11.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0120000000000001e-06 2024-09-13T09:13:11.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138450+0000", 2024-09-13T09:13:11.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6180000000000003e-06 2024-09-13T09:13:11.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138458+0000", 2024-09-13T09:13:11.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9130000000000003e-06 2024-09-13T09:13:11.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.183973+0000", 2024-09-13T09:13:11.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045514741999999997 2024-09-13T09:13:11.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184022+0000", 2024-09-13T09:13:11.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9521999999999998e-05 2024-09-13T09:13:11.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184076+0000", 2024-09-13T09:13:11.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3888000000000003e-05 2024-09-13T09:13:11.621 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.621 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.621 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.622 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17754 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2170880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.622 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:13:11.622 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.470230751999999, 2024-09-13T09:13:11.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045714312999999999, 2024-09-13T09:13:11.622 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.622 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.622 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.622 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.622 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.622 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17754 2024-09-13T09:13:11.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.622 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:13:11.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:13:11.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138471+0000", 2024-09-13T09:13:11.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138474+0000", 2024-09-13T09:13:11.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8540000000000001e-06 2024-09-13T09:13:11.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138480+0000", 2024-09-13T09:13:11.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5139999999999998e-06 2024-09-13T09:13:11.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138485+0000", 2024-09-13T09:13:11.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.989e-06 2024-09-13T09:13:11.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184095+0000", 2024-09-13T09:13:11.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045609702000000002 2024-09-13T09:13:11.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184135+0000", 2024-09-13T09:13:11.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0510999999999999e-05 2024-09-13T09:13:11.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184186+0000", 2024-09-13T09:13:11.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0262000000000001e-05 2024-09-13T09:13:11.626 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.626 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.626 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.627 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17755 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2193408~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.627 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:13:11.627 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.470211564, 2024-09-13T09:13:11.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045899349999999998, 2024-09-13T09:13:11.627 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.627 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.627 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17755 2024-09-13T09:13:11.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.627 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:13:11.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:13:11.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138490+0000", 2024-09-13T09:13:11.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138492+0000", 2024-09-13T09:13:11.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4310000000000001e-06 2024-09-13T09:13:11.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138498+0000", 2024-09-13T09:13:11.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7400000000000001e-06 2024-09-13T09:13:11.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138503+0000", 2024-09-13T09:13:11.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:13:11.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184222+0000", 2024-09-13T09:13:11.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045718996999999997 2024-09-13T09:13:11.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184306+0000", 2024-09-13T09:13:11.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4293000000000001e-05 2024-09-13T09:13:11.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184390+0000", 2024-09-13T09:13:11.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3718e-05 2024-09-13T09:13:11.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.631 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.631 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17756 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2215936~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.631 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.138507+0000", 2024-09-13T09:13:11.631 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.470195017000002, 2024-09-13T09:13:11.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.046004902, 2024-09-13T09:13:11.632 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.632 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.632 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17756 2024-09-13T09:13:11.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.632 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138507+0000", 2024-09-13T09:13:11.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138507+0000", 2024-09-13T09:13:11.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138506+0000", 2024-09-13T09:13:11.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138509+0000", 2024-09-13T09:13:11.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:13:11.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138514+0000", 2024-09-13T09:13:11.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.665e-06 2024-09-13T09:13:11.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.138519+0000", 2024-09-13T09:13:11.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7530000000000001e-06 2024-09-13T09:13:11.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184415+0000", 2024-09-13T09:13:11.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045895575000000001 2024-09-13T09:13:11.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184459+0000", 2024-09-13T09:13:11.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4506999999999999e-05 2024-09-13T09:13:11.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.184512+0000", 2024-09-13T09:13:11.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2685999999999997e-05 2024-09-13T09:13:11.638 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.638 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.638 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.638 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17766 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2441216~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.638 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140631+0000", 2024-09-13T09:13:11.638 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.468071459000001, 2024-09-13T09:13:11.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045443425000000003, 2024-09-13T09:13:11.638 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.638 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.639 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.639 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.639 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.639 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17766 2024-09-13T09:13:11.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.639 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140631+0000", 2024-09-13T09:13:11.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140631+0000", 2024-09-13T09:13:11.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140630+0000", 2024-09-13T09:13:11.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140632+0000", 2024-09-13T09:13:11.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1770000000000001e-06 2024-09-13T09:13:11.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140638+0000", 2024-09-13T09:13:11.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5550000000000002e-06 2024-09-13T09:13:11.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140643+0000", 2024-09-13T09:13:11.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5589999999999999e-06 2024-09-13T09:13:11.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.185929+0000", 2024-09-13T09:13:11.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045286356999999999 2024-09-13T09:13:11.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.185985+0000", 2024-09-13T09:13:11.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6042999999999999e-05 2024-09-13T09:13:11.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186074+0000", 2024-09-13T09:13:11.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9083000000000001e-05 2024-09-13T09:13:11.643 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.643 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.643 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.643 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17767 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2463744~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.643 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:13:11.643 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.468055666000001, 2024-09-13T09:13:11.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045589383999999997, 2024-09-13T09:13:11.644 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.644 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.644 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17767 2024-09-13T09:13:11.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.644 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:13:11.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:13:11.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140646+0000", 2024-09-13T09:13:11.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140648+0000", 2024-09-13T09:13:11.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.001e-06 2024-09-13T09:13:11.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140654+0000", 2024-09-13T09:13:11.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6690000000000003e-06 2024-09-13T09:13:11.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140658+0000", 2024-09-13T09:13:11.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4220000000000002e-06 2024-09-13T09:13:11.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186115+0000", 2024-09-13T09:13:11.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045456435000000003 2024-09-13T09:13:11.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186172+0000", 2024-09-13T09:13:11.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7088e-05 2024-09-13T09:13:11.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186236+0000", 2024-09-13T09:13:11.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4082000000000001e-05 2024-09-13T09:13:11.649 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.649 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.649 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.649 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17768 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2486272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.649 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:13:11.649 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.468039698999998, 2024-09-13T09:13:11.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045865601999999998, 2024-09-13T09:13:11.649 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.650 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.650 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.650 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.650 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.650 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17768 2024-09-13T09:13:11.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.650 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:13:11.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:13:11.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140662+0000", 2024-09-13T09:13:11.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140664+0000", 2024-09-13T09:13:11.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1449999999999998e-06 2024-09-13T09:13:11.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140670+0000", 2024-09-13T09:13:11.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9460000000000003e-06 2024-09-13T09:13:11.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140675+0000", 2024-09-13T09:13:11.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5770000000000004e-06 2024-09-13T09:13:11.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186267+0000", 2024-09-13T09:13:11.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045591957000000002 2024-09-13T09:13:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186322+0000", 2024-09-13T09:13:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5049000000000001e-05 2024-09-13T09:13:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186528+0000", 2024-09-13T09:13:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020626600000000001 2024-09-13T09:13:11.654 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.655 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.655 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.655 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17769 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2508800~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.655 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:13:11.655 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.468023637999998, 2024-09-13T09:13:11.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045981877999999997, 2024-09-13T09:13:11.655 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.655 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.655 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.655 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.656 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17769 2024-09-13T09:13:11.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.656 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:13:11.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:13:11.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140678+0000", 2024-09-13T09:13:11.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140680+0000", 2024-09-13T09:13:11.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0209999999999998e-06 2024-09-13T09:13:11.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140686+0000", 2024-09-13T09:13:11.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2840000000000003e-06 2024-09-13T09:13:11.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.140691+0000", 2024-09-13T09:13:11.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.369e-06 2024-09-13T09:13:11.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186562+0000", 2024-09-13T09:13:11.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045870834999999999 2024-09-13T09:13:11.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186606+0000", 2024-09-13T09:13:11.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4631e-05 2024-09-13T09:13:11.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.186660+0000", 2024-09-13T09:13:11.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4042000000000002e-05 2024-09-13T09:13:11.660 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.660 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.660 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.660 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17885 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2217984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.660 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:13:11.660 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.466050539000001, 2024-09-13T09:13:11.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045427829000000003, 2024-09-13T09:13:11.660 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.661 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.661 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17885 2024-09-13T09:13:11.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.661 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:13:11.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:13:11.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142651+0000", 2024-09-13T09:13:11.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142653+0000", 2024-09-13T09:13:11.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2500000000000001e-06 2024-09-13T09:13:11.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142659+0000", 2024-09-13T09:13:11.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.366e-06 2024-09-13T09:13:11.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142663+0000", 2024-09-13T09:13:11.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5190000000000003e-06 2024-09-13T09:13:11.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.187980+0000", 2024-09-13T09:13:11.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045316233999999997 2024-09-13T09:13:11.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188032+0000", 2024-09-13T09:13:11.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1981000000000003e-05 2024-09-13T09:13:11.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188079+0000", 2024-09-13T09:13:11.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7803000000000003e-05 2024-09-13T09:13:11.665 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.665 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.665 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.665 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17886 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2240512~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.665 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142686+0000", 2024-09-13T09:13:11.665 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.466016131, 2024-09-13T09:13:11.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045507535000000002, 2024-09-13T09:13:11.666 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.666 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.666 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17886 2024-09-13T09:13:11.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.666 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142686+0000", 2024-09-13T09:13:11.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142686+0000", 2024-09-13T09:13:11.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142685+0000", 2024-09-13T09:13:11.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142688+0000", 2024-09-13T09:13:11.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.83e-06 2024-09-13T09:13:11.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142694+0000", 2024-09-13T09:13:11.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2910000000000004e-06 2024-09-13T09:13:11.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142699+0000", 2024-09-13T09:13:11.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9599999999999999e-06 2024-09-13T09:13:11.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188102+0000", 2024-09-13T09:13:11.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045402459999999999 2024-09-13T09:13:11.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188143+0000", 2024-09-13T09:13:11.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1607000000000003e-05 2024-09-13T09:13:11.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188193+0000", 2024-09-13T09:13:11.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9947000000000001e-05 2024-09-13T09:13:11.670 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.670 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.670 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.670 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17887 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2263040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.670 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.142704+0000", 2024-09-13T09:13:11.671 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.46599814, 2024-09-13T09:13:11.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045652102999999999, 2024-09-13T09:13:11.671 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.671 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.671 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.671 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.671 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.671 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17887 2024-09-13T09:13:11.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.671 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142704+0000", 2024-09-13T09:13:11.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142704+0000", 2024-09-13T09:13:11.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142703+0000", 2024-09-13T09:13:11.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142706+0000", 2024-09-13T09:13:11.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.351e-06 2024-09-13T09:13:11.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142712+0000", 2024-09-13T09:13:11.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8799999999999996e-06 2024-09-13T09:13:11.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.142716+0000", 2024-09-13T09:13:11.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5839999999999996e-06 2024-09-13T09:13:11.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188230+0000", 2024-09-13T09:13:11.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045513801 2024-09-13T09:13:11.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188303+0000", 2024-09-13T09:13:11.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3032000000000001e-05 2024-09-13T09:13:11.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.188356+0000", 2024-09-13T09:13:11.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2899000000000003e-05 2024-09-13T09:13:11.675 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.675 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.675 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.676 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17893 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2398208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.676 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:13:11.676 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.464946605000002, 2024-09-13T09:13:11.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045574960999999997, 2024-09-13T09:13:11.676 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.676 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.676 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.676 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.676 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.676 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17893 2024-09-13T09:13:11.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.677 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:13:11.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:13:11.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143755+0000", 2024-09-13T09:13:11.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:11.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143757+0000", 2024-09-13T09:13:11.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2000000000000001e-06 2024-09-13T09:13:11.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143763+0000", 2024-09-13T09:13:11.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.553e-06 2024-09-13T09:13:11.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143768+0000", 2024-09-13T09:13:11.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0019999999999997e-06 2024-09-13T09:13:11.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189160+0000", 2024-09-13T09:13:11.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045392147000000001 2024-09-13T09:13:11.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189243+0000", 2024-09-13T09:13:11.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2594000000000002e-05 2024-09-13T09:13:11.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189330+0000", 2024-09-13T09:13:11.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7764999999999997e-05 2024-09-13T09:13:11.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.681 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.681 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17894 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2420736~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.681 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.143886+0000", 2024-09-13T09:13:11.681 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.464816121999998, 2024-09-13T09:13:11.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045562601000000001, 2024-09-13T09:13:11.681 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.682 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.682 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.682 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.682 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.682 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17894 2024-09-13T09:13:11.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.682 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143886+0000", 2024-09-13T09:13:11.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143886+0000", 2024-09-13T09:13:11.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143771+0000", 2024-09-13T09:13:11.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998856 2024-09-13T09:13:11.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143889+0000", 2024-09-13T09:13:11.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000117314 2024-09-13T09:13:11.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143896+0000", 2024-09-13T09:13:11.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5920000000000003e-06 2024-09-13T09:13:11.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143902+0000", 2024-09-13T09:13:11.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:13:11.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189359+0000", 2024-09-13T09:13:11.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045457414000000002 2024-09-13T09:13:11.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189399+0000", 2024-09-13T09:13:11.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9653999999999997e-05 2024-09-13T09:13:11.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189449+0000", 2024-09-13T09:13:11.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9456999999999998e-05 2024-09-13T09:13:11.686 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.686 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.686 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.686 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17895 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2443264~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.686 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.143990+0000", 2024-09-13T09:13:11.686 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.464712147, 2024-09-13T09:13:11.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045666642, 2024-09-13T09:13:11.687 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.687 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.687 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.687 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.687 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.687 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17895 2024-09-13T09:13:11.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.687 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143990+0000", 2024-09-13T09:13:11.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143990+0000", 2024-09-13T09:13:11.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143907+0000", 2024-09-13T09:13:11.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999166 2024-09-13T09:13:11.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.143993+0000", 2024-09-13T09:13:11.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6049000000000002e-05 2024-09-13T09:13:11.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144000+0000", 2024-09-13T09:13:11.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6550000000000004e-06 2024-09-13T09:13:11.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144006+0000", 2024-09-13T09:13:11.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4450000000000004e-06 2024-09-13T09:13:11.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189483+0000", 2024-09-13T09:13:11.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045477228000000001 2024-09-13T09:13:11.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189567+0000", 2024-09-13T09:13:11.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3721999999999996e-05 2024-09-13T09:13:11.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189657+0000", 2024-09-13T09:13:11.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9764000000000003e-05 2024-09-13T09:13:11.692 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.692 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.692 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.692 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17896 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2465792~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.692 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144260+0000", 2024-09-13T09:13:11.692 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.46444228, 2024-09-13T09:13:11.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045647222000000001, 2024-09-13T09:13:11.692 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.692 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.692 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.693 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.693 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.693 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17896 2024-09-13T09:13:11.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.693 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144260+0000", 2024-09-13T09:13:11.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144260+0000", 2024-09-13T09:13:11.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144010+0000", 2024-09-13T09:13:11.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9997506 2024-09-13T09:13:11.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144263+0000", 2024-09-13T09:13:11.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025252299999999999 2024-09-13T09:13:11.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144270+0000", 2024-09-13T09:13:11.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5259999999999996e-06 2024-09-13T09:13:11.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144276+0000", 2024-09-13T09:13:11.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:13:11.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189682+0000", 2024-09-13T09:13:11.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045406482999999997 2024-09-13T09:13:11.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189723+0000", 2024-09-13T09:13:11.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1059000000000001e-05 2024-09-13T09:13:11.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189907+0000", 2024-09-13T09:13:11.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018378099999999999 2024-09-13T09:13:11.697 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.697 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.697 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.697 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17897 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2488320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.697 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144430+0000", 2024-09-13T09:13:11.697 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.464272137999998, 2024-09-13T09:13:11.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045662734000000003, 2024-09-13T09:13:11.698 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.698 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.698 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.698 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.698 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.698 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17897 2024-09-13T09:13:11.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.698 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144430+0000", 2024-09-13T09:13:11.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144430+0000", 2024-09-13T09:13:11.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144281+0000", 2024-09-13T09:13:11.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998507 2024-09-13T09:13:11.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144433+0000", 2024-09-13T09:13:11.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00015261800000000001 2024-09-13T09:13:11.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144441+0000", 2024-09-13T09:13:11.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6790000000000005e-06 2024-09-13T09:13:11.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144446+0000", 2024-09-13T09:13:11.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5280000000000003e-06 2024-09-13T09:13:11.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.189943+0000", 2024-09-13T09:13:11.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045496178999999998 2024-09-13T09:13:11.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190025+0000", 2024-09-13T09:13:11.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2325999999999997e-05 2024-09-13T09:13:11.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190093+0000", 2024-09-13T09:13:11.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7780000000000005e-05 2024-09-13T09:13:11.702 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.702 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.702 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.702 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17898 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2510848~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.703 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144655+0000", 2024-09-13T09:13:11.703 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.464046963000001, 2024-09-13T09:13:11.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045552875, 2024-09-13T09:13:11.703 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.703 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.703 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.703 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.703 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.703 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17898 2024-09-13T09:13:11.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.703 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144655+0000", 2024-09-13T09:13:11.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144655+0000", 2024-09-13T09:13:11.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144451+0000", 2024-09-13T09:13:11.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9997959 2024-09-13T09:13:11.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144658+0000", 2024-09-13T09:13:11.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000207188 2024-09-13T09:13:11.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144666+0000", 2024-09-13T09:13:11.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1380000000000007e-06 2024-09-13T09:13:11.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144672+0000", 2024-09-13T09:13:11.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7980000000000002e-06 2024-09-13T09:13:11.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190116+0000", 2024-09-13T09:13:11.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045443413000000002 2024-09-13T09:13:11.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190158+0000", 2024-09-13T09:13:11.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2055000000000003e-05 2024-09-13T09:13:11.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190208+0000", 2024-09-13T09:13:11.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0387999999999999e-05 2024-09-13T09:13:11.707 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.707 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.707 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.708 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17899 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2533376~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.708 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144737+0000", 2024-09-13T09:13:11.708 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.463964820000001, 2024-09-13T09:13:11.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045677290000000002, 2024-09-13T09:13:11.708 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.708 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.709 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17899 2024-09-13T09:13:11.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.709 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144737+0000", 2024-09-13T09:13:11.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144737+0000", 2024-09-13T09:13:11.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144678+0000", 2024-09-13T09:13:11.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999409 2024-09-13T09:13:11.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144740+0000", 2024-09-13T09:13:11.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2116e-05 2024-09-13T09:13:11.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144748+0000", 2024-09-13T09:13:11.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2130000000000008e-06 2024-09-13T09:13:11.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144754+0000", 2024-09-13T09:13:11.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.66e-06 2024-09-13T09:13:11.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190238+0000", 2024-09-13T09:13:11.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045483824999999999 2024-09-13T09:13:11.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190322+0000", 2024-09-13T09:13:11.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4438999999999996e-05 2024-09-13T09:13:11.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190415+0000", 2024-09-13T09:13:11.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2227999999999999e-05 2024-09-13T09:13:11.713 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.713 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.713 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.713 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:17900 3.3 3:c0a2d13f:::rbd_data.1123d4249472.0000000000000004:10 [read 2555904~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:11.713 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.144938+0000", 2024-09-13T09:13:11.713 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.463763919000002, 2024-09-13T09:13:11.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045612509000000002, 2024-09-13T09:13:11.713 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:11.714 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:11.714 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:11.714 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:11.714 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:11.714 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 17900 2024-09-13T09:13:11.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.714 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:11.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:11.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144938+0000", 2024-09-13T09:13:11.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:11.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144938+0000", 2024-09-13T09:13:11.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:11.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:11.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144758+0000", 2024-09-13T09:13:11.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9998202 2024-09-13T09:13:11.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:11.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144941+0000", 2024-09-13T09:13:11.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000182688 2024-09-13T09:13:11.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:11.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144949+0000", 2024-09-13T09:13:11.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7300000000000005e-06 2024-09-13T09:13:11.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:11.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.144954+0000", 2024-09-13T09:13:11.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4090000000000002e-06 2024-09-13T09:13:11.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:11.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190448+0000", 2024-09-13T09:13:11.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.045493901000000003 2024-09-13T09:13:11.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:11.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190499+0000", 2024-09-13T09:13:11.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0677999999999998e-05 2024-09-13T09:13:11.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:11.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:11.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:11.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.190551+0000", 2024-09-13T09:13:11.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1820999999999997e-05 2024-09-13T09:13:11.718 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.718 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.718 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.718 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:11.718 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:11.718 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:11.720 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:11.721 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:11.719+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:11.820 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:11.819+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:11.864 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:11.864 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:11.864 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:11.864 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:11.872 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:11.921 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:11.920+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:11.944 INFO:teuthology.orchestra.run.smithi157.stdout:3443 trunc from 0x13cf600 to 0x691a00 2024-09-13T09:13:11.963 INFO:teuthology.orchestra.run.smithi157.stdout:3445 punch from 0x503c00 to 0x508600, (0x4a00 bytes) 2024-09-13T09:13:11.963 INFO:teuthology.orchestra.run.smithi157.stdout:3447 read 0x4c800 thru 0x4e64a (0x1e4b bytes) 2024-09-13T09:13:11.963 INFO:teuthology.orchestra.run.smithi157.stdout:3450 read 0x376000 thru 0x37b9d1 (0x59d2 bytes) 2024-09-13T09:13:11.964 INFO:teuthology.orchestra.run.smithi157.stdout:3451 punch from 0x3d7a00 to 0x3dd400, (0x5a00 bytes) 2024-09-13T09:13:11.977 INFO:teuthology.orchestra.run.smithi157.stdout:3452 write 0x2022c00 thru 0x2029b64 (0x6f65 bytes) 2024-09-13T09:13:11.990 INFO:teuthology.orchestra.run.smithi157.stdout:3453 trunc from 0x2029c00 to 0x4f54200 2024-09-13T09:13:12.001 INFO:teuthology.orchestra.run.smithi157.stdout:3456 write 0x1beb000 thru 0x1bef37f (0x4380 bytes) 2024-09-13T09:13:12.008 INFO:teuthology.orchestra.run.smithi157.stdout:3458 read 0x2c65e00 thru 0x2c68f35 (0x3136 bytes) 2024-09-13T09:13:12.008 INFO:teuthology.orchestra.run.smithi157.stdout:3462 write 0xe15800 thru 0xe19b64 (0x4365 bytes) 2024-09-13T09:13:12.009 INFO:teuthology.orchestra.run.smithi157.stdout:3463 read 0x329ca00 thru 0x32abacc (0xf0cd bytes) 2024-09-13T09:13:12.009 INFO:teuthology.orchestra.run.smithi157.stdout:3464 punch from 0x4ded800 to 0x4df2400, (0x4c00 bytes) 2024-09-13T09:13:12.009 INFO:teuthology.orchestra.run.smithi157.stdout:3467 write 0x9b4200 thru 0x9be807 (0xa608 bytes) 2024-09-13T09:13:12.023 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:12.022+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:12.023 INFO:teuthology.orchestra.run.smithi157.stdout:3469 trunc from 0x4f54200 to 0xd6d5a00 2024-09-13T09:13:12.032 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:12.033 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:12.041 INFO:teuthology.orchestra.run.smithi157.stdout:3471 punch from 0x6de0c00 to 0x6ded400, (0xc800 bytes) 2024-09-13T09:13:12.042 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:12.043 INFO:teuthology.orchestra.run.smithi157.stdout:3474 write 0xb093200 thru 0xb09afc5 (0x7dc6 bytes) 2024-09-13T09:13:12.043 INFO:teuthology.orchestra.run.smithi157.stdout:3475 read 0xb6dda00 thru 0xb6ea4ef (0xcaf0 bytes) 2024-09-13T09:13:12.043 INFO:teuthology.orchestra.run.smithi157.stdout:3476 read 0x490bc00 thru 0x4910328 (0x4729 bytes) 2024-09-13T09:13:12.043 INFO:teuthology.orchestra.run.smithi157.stdout:3477 read 0xb0ee800 thru 0xb0f12ab (0x2aac bytes) 2024-09-13T09:13:12.043 INFO:teuthology.orchestra.run.smithi157.stdout:3479 punch from 0x941cc00 to 0x9423800, (0x6c00 bytes) 2024-09-13T09:13:12.043 INFO:teuthology.orchestra.run.smithi157.stdout:3482 punch from 0xa45de00 to 0xa462c00, (0x4e00 bytes) 2024-09-13T09:13:12.043 INFO:teuthology.orchestra.run.smithi157.stdout:3483 read 0x216a000 thru 0x2171ee5 (0x7ee6 bytes) 2024-09-13T09:13:12.047 INFO:teuthology.orchestra.run.smithi157.stdout:3484 trunc from 0xd6d5a00 to 0xb2c2800 2024-09-13T09:13:12.059 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:13:12.074 INFO:teuthology.orchestra.run.smithi157.stdout:3485 write 0xc5e1a00 thru 0xc5ea002 (0x8603 bytes) 2024-09-13T09:13:12.080 INFO:teuthology.orchestra.run.smithi157.stdout:3487 write 0x53dc800 thru 0x53e1b83 (0x5384 bytes) 2024-09-13T09:13:12.096 INFO:teuthology.orchestra.run.smithi157.stdout:3488 write 0xe64d000 thru 0xe65bfd7 (0xefd8 bytes) 2024-09-13T09:13:12.097 INFO:teuthology.orchestra.run.smithi157.stdout:3489 write 0x83c5200 thru 0x83d29f3 (0xd7f4 bytes) 2024-09-13T09:13:12.099 INFO:teuthology.orchestra.run.smithi157.stdout:3490 trunc from 0xe65c000 to 0xcd0c000 2024-09-13T09:13:12.119 INFO:teuthology.orchestra.run.smithi157.stdout:3491 write 0x862e400 thru 0x86362f4 (0x7ef5 bytes) 2024-09-13T09:13:12.123 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:12.122+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:12.127 INFO:teuthology.orchestra.run.smithi157.stdout:3493 punch from 0x954fc00 to 0x9551400, (0x1800 bytes) 2024-09-13T09:13:12.127 INFO:teuthology.orchestra.run.smithi157.stdout:3494 read 0x2c32200 thru 0x2c33964 (0x1765 bytes) 2024-09-13T09:13:12.127 INFO:teuthology.orchestra.run.smithi157.stdout:3495 write 0x25d1e00 thru 0x25d24ff (0x700 bytes) 2024-09-13T09:13:12.127 INFO:teuthology.orchestra.run.smithi157.stdout:3499 write 0x92e400 thru 0x931927 (0x3528 bytes) 2024-09-13T09:13:12.127 INFO:teuthology.orchestra.run.smithi157.stdout:3500 read 0x12b5800 thru 0x12bc559 (0x6d5a bytes) 2024-09-13T09:13:12.129 INFO:teuthology.orchestra.run.smithi157.stdout:3501 trunc from 0xcd0c000 to 0xc1d3400 2024-09-13T09:13:12.153 INFO:teuthology.orchestra.run.smithi157.stdout:3502 trunc from 0xc1d3400 to 0x2d07a00 2024-09-13T09:13:12.162 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:12.162 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:12.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:12.162 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:12.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.162 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9035 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2342912~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.163 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:12.163 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.414002167, 2024-09-13T09:13:12.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070293606999999994, 2024-09-13T09:13:12.163 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.163 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.163 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.163 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.163 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.163 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9035 2024-09-13T09:13:12.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.164 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:12.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:12.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:12.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:12.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745602+0000", 2024-09-13T09:13:12.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-06 2024-09-13T09:13:12.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745607+0000", 2024-09-13T09:13:12.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0770000000000001e-06 2024-09-13T09:13:12.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745609+0000", 2024-09-13T09:13:12.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3499999999999999e-06 2024-09-13T09:13:12.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815626+0000", 2024-09-13T09:13:12.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070017398999999994 2024-09-13T09:13:12.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815672+0000", 2024-09-13T09:13:12.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5275999999999998e-05 2024-09-13T09:13:12.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815895+0000", 2024-09-13T09:13:12.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022328300000000001 2024-09-13T09:13:12.167 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.167 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.167 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.167 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.167 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:12.167 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.413989622999999, 2024-09-13T09:13:12.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:13:12.167 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.168 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.168 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.168 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.168 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.168 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:13:12.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.168 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:12.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:12.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:12.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:12.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:13:12.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:12.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:13:12.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:13:12.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:13:12.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:13:12.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:13:12.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:13:12.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:13:12.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:13:12.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:13:12.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:13:12.172 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.172 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.172 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.172 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.172 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:12.172 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.413980945, 2024-09-13T09:13:12.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:13:12.172 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.172 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.172 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:13:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:12.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:13:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:13:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:13:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:13:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:13:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:13:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:13:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:13:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:13:12.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:13:12.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:13:12.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:13:12.176 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.176 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.176 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.176 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.176 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:12.177 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.413971959, 2024-09-13T09:13:12.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:13:12.177 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.177 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.177 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.177 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.177 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.177 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:13:12.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.177 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:12.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:12.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:12.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:12.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:13:12.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:13:12.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:13:12.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:13:12.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:13:12.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:13:12.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:13:12.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:13:12.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:13:12.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:13:12.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:13:12.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:13:12.181 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.181 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.181 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.181 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.181 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:12.181 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.413963987000001, 2024-09-13T09:13:12.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:13:12.182 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.182 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.182 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.182 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.182 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.182 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:13:12.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.182 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:12.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:12.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:12.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:12.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:13:12.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:13:12.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:13:12.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:13:12.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:13:12.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:13:12.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:13:12.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:13:12.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:13:12.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:13:12.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:13:12.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:13:12.185 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.185 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.185 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.186 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.186 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:12.186 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.412089153, 2024-09-13T09:13:12.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:13:12.186 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.186 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.186 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.186 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.186 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.186 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:13:12.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.186 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:12.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:12.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:12.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:12.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:13:12.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:13:12.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:13:12.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:13:12.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:13:12.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:13:12.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:13:12.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:13:12.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:13:12.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:13:12.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:13:12.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:13:12.190 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.190 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.190 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.191 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.191 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:12.191 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.412078491999999, 2024-09-13T09:13:12.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:13:12.191 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.191 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.191 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.191 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.191 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.191 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:13:12.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.192 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:12.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:12.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:12.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:12.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:13:12.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:13:12.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:13:12.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:13:12.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:13:12.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:13:12.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:13:12.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:13:12.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:13:12.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:13:12.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:13:12.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:13:12.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.195 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.195 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.195 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:12.195 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.412056516, 2024-09-13T09:13:12.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:13:12.196 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.196 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.196 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:13:12.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.196 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:12.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:12.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:12.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:12.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:13:12.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:13:12.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:13:12.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:13:12.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:13:12.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:13:12.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:13:12.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:13:12.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:13:12.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:13:12.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:13:12.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:13:12.200 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.200 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.200 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.200 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.200 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:12.200 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.412043014, 2024-09-13T09:13:12.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:13:12.200 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.200 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.201 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:13:12.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.201 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:12.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:12.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:12.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:12.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:13:12.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:13:12.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:13:12.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:13:12.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:13:12.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:13:12.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:13:12.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:13:12.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:13:12.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:13:12.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:13:12.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:13:12.205 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.205 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.205 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.205 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9055 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 6930432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.205 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:12.205 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.410050234, 2024-09-13T09:13:12.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070177949000000003, 2024-09-13T09:13:12.205 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.205 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.206 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9055 2024-09-13T09:13:12.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.206 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:12.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:12.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:12.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:12.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749554+0000", 2024-09-13T09:13:12.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.11e-06 2024-09-13T09:13:12.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749557+0000", 2024-09-13T09:13:12.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7949999999999999e-06 2024-09-13T09:13:12.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749559+0000", 2024-09-13T09:13:12.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:13:12.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819633+0000", 2024-09-13T09:13:12.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073995 2024-09-13T09:13:12.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819679+0000", 2024-09-13T09:13:12.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6006e-05 2024-09-13T09:13:12.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819731+0000", 2024-09-13T09:13:12.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1934000000000001e-05 2024-09-13T09:13:12.209 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.209 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.209 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.210 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9056 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 7159808~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.210 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:12.210 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.410042185, 2024-09-13T09:13:12.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070283126000000001, 2024-09-13T09:13:12.210 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.210 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.210 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9056 2024-09-13T09:13:12.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.210 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:12.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:12.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:12.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:12.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749562+0000", 2024-09-13T09:13:12.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1340000000000001e-06 2024-09-13T09:13:12.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749565+0000", 2024-09-13T09:13:12.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8760000000000001e-06 2024-09-13T09:13:12.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749567+0000", 2024-09-13T09:13:12.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2340000000000001e-06 2024-09-13T09:13:12.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819750+0000", 2024-09-13T09:13:12.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070182537000000003 2024-09-13T09:13:12.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819792+0000", 2024-09-13T09:13:12.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2141999999999997e-05 2024-09-13T09:13:12.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819844+0000", 2024-09-13T09:13:12.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2428e-05 2024-09-13T09:13:12.214 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.214 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.214 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.214 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.214 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:12.214 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.410033916, 2024-09-13T09:13:12.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:13:12.214 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.214 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.214 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.214 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.214 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.215 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:13:12.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.215 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:12.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:12.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:12.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:12.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:13:12.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:13:12.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:13:12.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:13:12.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:13:12.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:13:12.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:13:12.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:13:12.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:13:12.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:13:12.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:13:12.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:13:12.218 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.218 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.218 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.218 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.218 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:12.219 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.410009594, 2024-09-13T09:13:12.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:13:12.219 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.219 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.219 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.219 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.219 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.219 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:13:12.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.219 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:12.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:12.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:12.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:12.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:13:12.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:13:12.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:13:12.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:13:12.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:13:12.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:13:12.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:13:12.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:13:12.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:13:12.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:13:12.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:13:12.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:13:12.222 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.222 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.223 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.223 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.223 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:12.223 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.40999862, 2024-09-13T09:13:12.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:13:12.223 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.223 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.223 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.223 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.223 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.223 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:13:12.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.223 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:12.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:12.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:13:12.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:12.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:13:12.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:13:12.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:13:12.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:13:12.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:13:12.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:13:12.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:13:12.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:13:12.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:13:12.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:13:12.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:13:12.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:13:12.227 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.227 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.227 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.228 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9063 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1589248~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.228 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:12.228 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.408094977999999, 2024-09-13T09:13:12.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070391924999999994, 2024-09-13T09:13:12.228 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.228 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.228 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.228 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.228 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.228 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9063 2024-09-13T09:13:12.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.228 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:12.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:12.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:12.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:12.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751509+0000", 2024-09-13T09:13:12.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.142e-06 2024-09-13T09:13:12.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751513+0000", 2024-09-13T09:13:12.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6669999999999998e-06 2024-09-13T09:13:12.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751520+0000", 2024-09-13T09:13:12.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5910000000000001e-06 2024-09-13T09:13:12.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821593+0000", 2024-09-13T09:13:12.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073519000000001 2024-09-13T09:13:12.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821648+0000", 2024-09-13T09:13:12.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:13:12.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821900+0000", 2024-09-13T09:13:12.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025263999999999999 2024-09-13T09:13:12.232 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.232 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.232 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.232 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.232 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:12.232 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.408080995000001, 2024-09-13T09:13:12.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:13:12.232 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.232 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.232 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.232 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.232 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.233 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:13:12.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.233 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:12.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:12.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:12.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:12.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:13:12.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:13:12.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:13:12.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:12.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:13:12.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:13:12.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:13:12.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:13:12.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:13:12.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:13:12.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:13:12.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:13:12.238 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.238 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.238 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.239 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.239 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:12.239 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.408071597999999, 2024-09-13T09:13:12.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:13:12.239 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.239 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.239 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.239 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.239 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.239 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:13:12.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.240 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:12.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:12.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:12.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:12.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:13:12.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:13:12.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:13:12.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:13:12.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:13:12.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:13:12.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:13:12.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:13:12.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:13:12.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:13:12.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:13:12.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:13:12.243 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.243 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.243 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.243 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9069 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2965504~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.243 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:12.243 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.406112219000001, 2024-09-13T09:13:12.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070184959000000005, 2024-09-13T09:13:12.244 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.244 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.244 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.244 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.244 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.244 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9069 2024-09-13T09:13:12.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.244 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:12.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753492+0000", 2024-09-13T09:13:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.403e-06 2024-09-13T09:13:12.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753495+0000", 2024-09-13T09:13:12.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9390000000000002e-06 2024-09-13T09:13:12.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753504+0000", 2024-09-13T09:13:12.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5129999999999996e-06 2024-09-13T09:13:12.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823390+0000", 2024-09-13T09:13:12.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.069886632000000004 2024-09-13T09:13:12.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823445+0000", 2024-09-13T09:13:12.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4880999999999999e-05 2024-09-13T09:13:12.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823676+0000", 2024-09-13T09:13:12.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000230812 2024-09-13T09:13:12.247 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.247 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.247 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9070 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3194880~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.406097128000001, 2024-09-13T09:13:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070280588000000005, 2024-09-13T09:13:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9070 2024-09-13T09:13:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:12.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:12.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:12.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:12.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753507+0000", 2024-09-13T09:13:12.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:12.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753510+0000", 2024-09-13T09:13:12.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0460000000000001e-06 2024-09-13T09:13:12.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753513+0000", 2024-09-13T09:13:12.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:13:12.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823689+0000", 2024-09-13T09:13:12.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070176083 2024-09-13T09:13:12.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823734+0000", 2024-09-13T09:13:12.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5056999999999999e-05 2024-09-13T09:13:12.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823787+0000", 2024-09-13T09:13:12.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3121000000000002e-05 2024-09-13T09:13:12.252 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.252 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.252 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.252 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9071 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:12.252 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:12.252 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.406088581000001, 2024-09-13T09:13:12.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070386377999999999, 2024-09-13T09:13:12.253 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:12.253 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:12.253 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:12.253 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:12.253 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:12.253 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9071 2024-09-13T09:13:12.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.253 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:12.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:12.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:12.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:12.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:12.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:12.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:12.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:12.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:12.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:12.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753516+0000", 2024-09-13T09:13:12.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1519999999999999e-06 2024-09-13T09:13:12.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:12.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753519+0000", 2024-09-13T09:13:12.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6559999999999998e-06 2024-09-13T09:13:12.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:12.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753522+0000", 2024-09-13T09:13:12.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:13:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823798+0000", 2024-09-13T09:13:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070275907999999998 2024-09-13T09:13:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823845+0000", 2024-09-13T09:13:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7593000000000003e-05 2024-09-13T09:13:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:12.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:12.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823901+0000", 2024-09-13T09:13:12.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5934999999999999e-05 2024-09-13T09:13:12.257 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.257 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.257 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.257 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:12.257 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:12.257 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:12.258 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:12.258 INFO:teuthology.orchestra.run.smithi157.stdout:3504 clone 23 order 20 su 4096 sc 10 2024-09-13T09:13:12.259 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:12.259 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:12.260 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:12.260 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:12.261 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:13:12.261 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:13:12.261 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:12.261 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:12.262 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:12.262 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:12.262 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:12.257+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:12.269 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:12.270 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:12.360 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:12.358+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:12.369 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:12.369 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:12.377 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:12.461 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:12.460+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:12.487 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:12.487 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:12.496 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:12.562 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:12.560+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:12.570 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone22 from 0x2d07a00 (overlap 0x112400) to 0x0 2024-09-13T09:13:12.644 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:12.644 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:12.653 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:12.662 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:12.660+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:12.763 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:12.763 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:12.764 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:12.762+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:12.772 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:12.863 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:12.862+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:12.882 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:12.882 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:12.892 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:12.964 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:12.963+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:13.046 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:13.046 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:13.055 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:13:13.066 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:13.064+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:13.163 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:13.166 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:13.164+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:13.172 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:13:13.232 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-13T09:13:13.262 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:13:13.284 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:13.284 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:13.284 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:13.283+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:13.293 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:13:13.339 INFO:teuthology.orchestra.run.smithi157.stdout:3505 punch from 0x6bb400 to 0x6c6800, (0xb400 bytes) 2024-09-13T09:13:13.368 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:13.367+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:13.470 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:13.468+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:13.477 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:13.477 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:13.486 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:13.493 INFO:teuthology.orchestra.run.smithi157.stdout:3506 read 0xba5a00 thru 0xbab759 (0x5d5a bytes) 2024-09-13T09:13:13.503 INFO:teuthology.orchestra.run.smithi157.stdout:3507 trunc from 0x2d07a00 to 0x9d36c00 2024-09-13T09:13:13.514 INFO:teuthology.orchestra.run.smithi157.stdout:3508 read 0x711ea00 thru 0x712c290 (0xd891 bytes) 2024-09-13T09:13:13.515 INFO:teuthology.orchestra.run.smithi157.stdout:3510 read 0x23a0a00 thru 0x23a808a (0x768b bytes) 2024-09-13T09:13:13.517 INFO:teuthology.orchestra.run.smithi157.stdout:3515 read 0x986b200 thru 0x9878d47 (0xdb48 bytes) 2024-09-13T09:13:13.517 INFO:teuthology.orchestra.run.smithi157.stdout:3516 write 0x752bc00 thru 0x7538988 (0xcd89 bytes) 2024-09-13T09:13:13.517 INFO:teuthology.orchestra.run.smithi157.stdout:3517 read 0x5a6000 thru 0x5b59b2 (0xf9b3 bytes) 2024-09-13T09:13:13.544 INFO:teuthology.orchestra.run.smithi157.stdout:3520 write 0xde95c00 thru 0xde97d43 (0x2144 bytes) 2024-09-13T09:13:13.545 INFO:teuthology.orchestra.run.smithi157.stdout:3521 read 0x409e00 thru 0x40a1ba (0x3bb bytes) 2024-09-13T09:13:13.546 INFO:teuthology.orchestra.run.smithi157.stdout:3523 read 0xdc1000 thru 0xdc5024 (0x4025 bytes) 2024-09-13T09:13:13.547 INFO:teuthology.orchestra.run.smithi157.stdout:3524 write 0x41bf000 thru 0x41ccf2d (0xdf2e bytes) 2024-09-13T09:13:13.555 INFO:teuthology.orchestra.run.smithi157.stdout:3525 trunc from 0xde97e00 to 0x7395200 2024-09-13T09:13:13.571 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:13.569+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:13.589 INFO:teuthology.orchestra.run.smithi157.stdout:3526 read 0x155fe00 thru 0x156ef52 (0xf153 bytes) 2024-09-13T09:13:13.603 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:13.603 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:13.604 INFO:teuthology.orchestra.run.smithi157.stdout:3527 trunc from 0x7395200 to 0x20b9c00 2024-09-13T09:13:13.615 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:13.673 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:13.672+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:13.688 INFO:teuthology.orchestra.run.smithi157.stdout:3528 read 0x1c65a00 thru 0x1c70c90 (0xb291 bytes) 2024-09-13T09:13:13.707 INFO:teuthology.orchestra.run.smithi157.stdout:3530 write 0x8ad3e00 thru 0x8ae1314 (0xd515 bytes) 2024-09-13T09:13:13.708 INFO:teuthology.orchestra.run.smithi157.stdout:3532 punch from 0x776a800 to 0x7777000, (0xc800 bytes) 2024-09-13T09:13:13.743 INFO:teuthology.orchestra.run.smithi157.stdout:3534 write 0x9a50a00 thru 0x9a57b95 (0x7196 bytes) 2024-09-13T09:13:13.743 INFO:teuthology.orchestra.run.smithi157.stdout:3536 punch from 0x3c57c00 to 0x3c5a800, (0x2c00 bytes) 2024-09-13T09:13:13.764 INFO:teuthology.orchestra.run.smithi157.stdout:3537 write 0x9c4c600 thru 0x9c54355 (0x7d56 bytes) 2024-09-13T09:13:13.765 INFO:teuthology.orchestra.run.smithi157.stdout:3540 read 0x7e00600 thru 0x7e05aec (0x54ed bytes) 2024-09-13T09:13:13.765 INFO:teuthology.orchestra.run.smithi157.stdout:3541 read 0x874b000 thru 0x8753960 (0x8961 bytes) 2024-09-13T09:13:13.766 INFO:teuthology.orchestra.run.smithi157.stdout:3542 read 0x6d00a00 thru 0x6d08f16 (0x8517 bytes) 2024-09-13T09:13:13.774 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:13.772+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:13.781 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:13.782 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:13.782 INFO:teuthology.orchestra.run.smithi157.stdout:3548 write 0xc553e00 thru 0xc5562e3 (0x24e4 bytes) 2024-09-13T09:13:13.783 INFO:teuthology.orchestra.run.smithi157.stdout:3549 write 0x68b6400 thru 0x68c13c3 (0xafc4 bytes) 2024-09-13T09:13:13.784 INFO:teuthology.orchestra.run.smithi157.stdout:3550 trunc from 0xc556400 to 0xb97e400 2024-09-13T09:13:13.793 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:13.814 INFO:teuthology.orchestra.run.smithi157.stdout:3552 clone 24 order 21 su 8192 sc 10 2024-09-13T09:13:13.877 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:13.876+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:13.936 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:13.936 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:13.944 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:13.978 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:13.977+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:14.046 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:14.046 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:14.055 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:14.099 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:14.097+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:14.156 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:14.156 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:14.165 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:14.181 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:14.180+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:14.281 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:13:14.282 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:14.281+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:14.306 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:14.306 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:14.315 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:14.383 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:14.382+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:14.407 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:14.407 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:14.413 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-13T09:13:14.413 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:13:14.413 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-13T09:13:14.413 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:13:14.413 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:13:14.413 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-13T09:13:14.419 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:14.419 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:14.425 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:14.425 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:14.429 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:14.430 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-09-13T09:13:14.441 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:13:14.442 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:13:14.458 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:14.458 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:14.476 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:14.476 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:14.483 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:14.482+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:14.493 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:14.493 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:14.526 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:14.526 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:13:14.526 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:14.577 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:14.577 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:14.577 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:14.577 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:14.577 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:14.584 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:14.582+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:14.586 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:14.604 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone23 from 0xb97e400 (overlap 0x20b9c00) to 0x345c00 2024-09-13T09:13:14.686 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:14.684+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:14.786 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:14.785+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:14.789 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:14.790 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:14.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:14.790 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:14.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.790 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9035 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2342912~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.790 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:14.790 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.041895857, 2024-09-13T09:13:14.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070293606999999994, 2024-09-13T09:13:14.790 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.790 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.791 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.791 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.791 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9035 2024-09-13T09:13:14.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.791 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:14.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:14.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:14.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:14.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745602+0000", 2024-09-13T09:13:14.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-06 2024-09-13T09:13:14.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745607+0000", 2024-09-13T09:13:14.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0770000000000001e-06 2024-09-13T09:13:14.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745609+0000", 2024-09-13T09:13:14.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3499999999999999e-06 2024-09-13T09:13:14.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815626+0000", 2024-09-13T09:13:14.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070017398999999994 2024-09-13T09:13:14.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815672+0000", 2024-09-13T09:13:14.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5275999999999998e-05 2024-09-13T09:13:14.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815895+0000", 2024-09-13T09:13:14.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022328300000000001 2024-09-13T09:13:14.795 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.795 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.795 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.795 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.795 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:14.795 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.041883313, 2024-09-13T09:13:14.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:13:14.795 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.795 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.795 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.795 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:13:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:14.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:14.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:13:14.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:14.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:13:14.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:13:14.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:13:14.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:13:14.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:13:14.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:13:14.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:13:14.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:13:14.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:13:14.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:13:14.799 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.799 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.799 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.800 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.800 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:14.800 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.041874634999999, 2024-09-13T09:13:14.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:13:14.800 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.800 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.800 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.800 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.800 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.800 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:13:14.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.801 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:14.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:14.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:14.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:14.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:13:14.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:13:14.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:13:14.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:13:14.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:13:14.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:13:14.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:13:14.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:13:14.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:13:14.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:13:14.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:13:14.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:13:14.804 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.805 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.805 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.805 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.805 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:14.805 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.041865648999998, 2024-09-13T09:13:14.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:13:14.805 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.805 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.806 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.806 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:13:14.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.806 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:14.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:14.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:14.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:14.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:13:14.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:13:14.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:13:14.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:13:14.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:13:14.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:13:14.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:13:14.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:13:14.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:13:14.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:13:14.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:13:14.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:13:14.809 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.809 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.809 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.809 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.809 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:14.810 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.041857676999999, 2024-09-13T09:13:14.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:13:14.810 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.810 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.810 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.810 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.810 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.810 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:13:14.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.810 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:14.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:14.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:14.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:14.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:13:14.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:13:14.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:13:14.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:13:14.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:13:14.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:13:14.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:13:14.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:13:14.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:13:14.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:13:14.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:13:14.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:13:14.814 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.814 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.814 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.815 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.815 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:14.815 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.039982843000001, 2024-09-13T09:13:14.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:13:14.815 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.815 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.815 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.815 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.815 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.815 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:13:14.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.815 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:14.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:14.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:14.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:14.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:13:14.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:13:14.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:13:14.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:13:14.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:13:14.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:13:14.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:13:14.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:13:14.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:13:14.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:13:14.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:13:14.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:13:14.819 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.819 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.819 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.819 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.819 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:14.819 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.039972182, 2024-09-13T09:13:14.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:13:14.819 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.819 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.819 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.819 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.819 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.820 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:13:14.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.820 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:14.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:14.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:14.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:14.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:13:14.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:13:14.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:13:14.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:13:14.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:13:14.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:13:14.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:13:14.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:13:14.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:13:14.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:13:14.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:13:14.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:13:14.823 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.823 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.823 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.824 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.824 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:14.824 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.039950206, 2024-09-13T09:13:14.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:13:14.824 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.824 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.824 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.824 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.824 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.824 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:13:14.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.825 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:14.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:14.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:14.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:14.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:13:14.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:13:14.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:13:14.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:13:14.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:13:14.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:13:14.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:13:14.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:13:14.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:13:14.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:13:14.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:13:14.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:13:14.828 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.828 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.828 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.829 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.829 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:14.829 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.039936703999999, 2024-09-13T09:13:14.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:13:14.829 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.829 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.829 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.829 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.829 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.829 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:13:14.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.829 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:14.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:14.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:14.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:14.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:13:14.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:13:14.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:13:14.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:13:14.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:13:14.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:13:14.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:13:14.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:13:14.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:13:14.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:13:14.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:13:14.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:13:14.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.833 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.833 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9055 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 6930432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.833 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:14.833 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.037943924, 2024-09-13T09:13:14.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070177949000000003, 2024-09-13T09:13:14.834 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.834 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.834 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9055 2024-09-13T09:13:14.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.834 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:14.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:14.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:14.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:14.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749554+0000", 2024-09-13T09:13:14.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.11e-06 2024-09-13T09:13:14.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749557+0000", 2024-09-13T09:13:14.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7949999999999999e-06 2024-09-13T09:13:14.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749559+0000", 2024-09-13T09:13:14.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:13:14.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819633+0000", 2024-09-13T09:13:14.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073995 2024-09-13T09:13:14.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819679+0000", 2024-09-13T09:13:14.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6006e-05 2024-09-13T09:13:14.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819731+0000", 2024-09-13T09:13:14.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1934000000000001e-05 2024-09-13T09:13:14.838 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.838 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.838 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.838 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9056 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 7159808~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.838 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:14.838 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.037935874999999, 2024-09-13T09:13:14.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070283126000000001, 2024-09-13T09:13:14.839 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.839 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.839 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.839 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.839 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.839 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9056 2024-09-13T09:13:14.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.839 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:14.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:14.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:14.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:14.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749562+0000", 2024-09-13T09:13:14.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1340000000000001e-06 2024-09-13T09:13:14.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749565+0000", 2024-09-13T09:13:14.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8760000000000001e-06 2024-09-13T09:13:14.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749567+0000", 2024-09-13T09:13:14.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2340000000000001e-06 2024-09-13T09:13:14.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819750+0000", 2024-09-13T09:13:14.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070182537000000003 2024-09-13T09:13:14.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819792+0000", 2024-09-13T09:13:14.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2141999999999997e-05 2024-09-13T09:13:14.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819844+0000", 2024-09-13T09:13:14.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2428e-05 2024-09-13T09:13:14.843 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.843 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.843 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.843 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.843 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:14.843 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.037927606, 2024-09-13T09:13:14.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:13:14.843 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.843 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.843 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.844 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:13:14.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.844 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:14.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:14.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:14.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:14.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:13:14.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:13:14.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:13:14.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:13:14.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:13:14.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:13:14.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:13:14.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:13:14.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:13:14.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:13:14.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:13:14.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:13:14.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.848 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.848 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.848 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:14.848 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.037903283999999, 2024-09-13T09:13:14.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:13:14.848 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.848 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.849 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:13:14.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.849 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:14.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:14.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:14.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:14.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:13:14.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:13:14.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:13:14.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:13:14.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:13:14.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:13:14.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:13:14.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:13:14.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:13:14.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:13:14.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:13:14.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:13:14.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.853 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.853 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.853 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:14.853 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.03789231, 2024-09-13T09:13:14.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:13:14.853 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.853 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.853 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.853 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.854 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:13:14.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.854 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:14.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:14.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:13:14.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:14.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:13:14.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:13:14.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:13:14.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:13:14.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:13:14.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:13:14.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:13:14.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:13:14.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:13:14.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:13:14.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:13:14.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:13:14.857 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.857 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.857 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.858 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9063 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1589248~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.858 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:14.858 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.035988668000002, 2024-09-13T09:13:14.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070391924999999994, 2024-09-13T09:13:14.858 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.858 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.858 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9063 2024-09-13T09:13:14.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.858 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:14.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:14.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:14.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:14.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751509+0000", 2024-09-13T09:13:14.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.142e-06 2024-09-13T09:13:14.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751513+0000", 2024-09-13T09:13:14.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6669999999999998e-06 2024-09-13T09:13:14.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751520+0000", 2024-09-13T09:13:14.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5910000000000001e-06 2024-09-13T09:13:14.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821593+0000", 2024-09-13T09:13:14.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073519000000001 2024-09-13T09:13:14.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821648+0000", 2024-09-13T09:13:14.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:13:14.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821900+0000", 2024-09-13T09:13:14.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025263999999999999 2024-09-13T09:13:14.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.862 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.862 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.863 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:14.863 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.035974684999999, 2024-09-13T09:13:14.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:13:14.863 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.863 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.863 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:13:14.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.864 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:14.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:14.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:14.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:14.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:13:14.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:13:14.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:13:14.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:14.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:13:14.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:13:14.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:13:14.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:13:14.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:13:14.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:13:14.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:13:14.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:13:14.867 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.867 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.867 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.868 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.868 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:14.868 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.035965288, 2024-09-13T09:13:14.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:13:14.868 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.868 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.868 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:13:14.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.868 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:14.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:14.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:14.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:14.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:13:14.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:13:14.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:13:14.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:13:14.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:13:14.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:13:14.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:13:14.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:13:14.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:13:14.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:13:14.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:13:14.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:13:14.872 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.872 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.872 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.872 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9069 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2965504~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.872 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:14.872 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.034005909000001, 2024-09-13T09:13:14.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070184959000000005, 2024-09-13T09:13:14.873 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.873 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.873 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.873 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.873 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.873 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9069 2024-09-13T09:13:14.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.873 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:14.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:14.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:14.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:14.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753492+0000", 2024-09-13T09:13:14.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.403e-06 2024-09-13T09:13:14.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753495+0000", 2024-09-13T09:13:14.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9390000000000002e-06 2024-09-13T09:13:14.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753504+0000", 2024-09-13T09:13:14.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5129999999999996e-06 2024-09-13T09:13:14.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823390+0000", 2024-09-13T09:13:14.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.069886632000000004 2024-09-13T09:13:14.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823445+0000", 2024-09-13T09:13:14.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4880999999999999e-05 2024-09-13T09:13:14.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823676+0000", 2024-09-13T09:13:14.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000230812 2024-09-13T09:13:14.877 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.877 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.877 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.877 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9070 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3194880~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.877 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:14.877 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.033990817999999, 2024-09-13T09:13:14.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070280588000000005, 2024-09-13T09:13:14.877 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.877 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.878 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9070 2024-09-13T09:13:14.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.878 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:14.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:14.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:14.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:14.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753507+0000", 2024-09-13T09:13:14.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:14.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753510+0000", 2024-09-13T09:13:14.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0460000000000001e-06 2024-09-13T09:13:14.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753513+0000", 2024-09-13T09:13:14.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:13:14.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823689+0000", 2024-09-13T09:13:14.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070176083 2024-09-13T09:13:14.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823734+0000", 2024-09-13T09:13:14.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5056999999999999e-05 2024-09-13T09:13:14.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823787+0000", 2024-09-13T09:13:14.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3121000000000002e-05 2024-09-13T09:13:14.881 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.881 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.881 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.882 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9071 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:14.882 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:14.882 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.033982270999999, 2024-09-13T09:13:14.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070386377999999999, 2024-09-13T09:13:14.882 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:14.882 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:14.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:14.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:14.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:14.882 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9071 2024-09-13T09:13:14.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.882 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:14.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:14.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:14.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:14.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:14.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:14.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:14.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:14.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:14.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:14.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753516+0000", 2024-09-13T09:13:14.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1519999999999999e-06 2024-09-13T09:13:14.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:14.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753519+0000", 2024-09-13T09:13:14.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6559999999999998e-06 2024-09-13T09:13:14.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:14.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753522+0000", 2024-09-13T09:13:14.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:13:14.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:14.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823798+0000", 2024-09-13T09:13:14.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070275907999999998 2024-09-13T09:13:14.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:14.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823845+0000", 2024-09-13T09:13:14.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7593000000000003e-05 2024-09-13T09:13:14.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:14.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:14.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:14.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823901+0000", 2024-09-13T09:13:14.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5934999999999999e-05 2024-09-13T09:13:14.886 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.886 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.886 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.886 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:14.886 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:14.886 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:14.887 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:14.886+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:14.888 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:14.988 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:14.987+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:14.995 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:14.996 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:14.996 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:15.005 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:15.089 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:15.088+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:15.152 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:15.152 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:15.152 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:15.160 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:15.190 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:15.188+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:15.291 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:15.290+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:15.317 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.317 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.783243571, 2024-09-13T09:13:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:13:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:13:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:13:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:13:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:13:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:13:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:13:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:13:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:13:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:13:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:13:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:13:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:13:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:13:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:13:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:13:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:13:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:13:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:13:15.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:13:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:13:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:13:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:13:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.783225536000003, 2024-09-13T09:13:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:13:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:13:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.324 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:15.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:13:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:13:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:13:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:13:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:13:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:13:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:13:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:13:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:13:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:13:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:13:15.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:13:15.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:13:15.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:13:15.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:13:15.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:13:15.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:13:15.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:13:15.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:13:15.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:13:15.329 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.330 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.330 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.330 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.330 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:15.330 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.783208866000003, 2024-09-13T09:13:15.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:13:15.330 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.330 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.330 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.331 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.331 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.331 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:13:15.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.331 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:15.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:15.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:13:15.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:13:15.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:15.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:13:15.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:13:15.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:13:15.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:13:15.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:13:15.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:13:15.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:13:15.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:13:15.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:13:15.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:13:15.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:13:15.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:13:15.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:13:15.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:13:15.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:13:15.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:13:15.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:13:15.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:13:15.336 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.336 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.336 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.336 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.336 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:15.336 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.783191234, 2024-09-13T09:13:15.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:13:15.336 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.336 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.337 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.337 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.337 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.337 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:13:15.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.337 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:15.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:13:15.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:13:15.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:13:15.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:13:15.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:13:15.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:13:15.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:13:15.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:13:15.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:13:15.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:13:15.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:13:15.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:13:15.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:13:15.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:13:15.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:13:15.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:13:15.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:13:15.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:13:15.342 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.342 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.342 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.342 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.342 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:15.342 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.783144929000002, 2024-09-13T09:13:15.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:13:15.342 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.342 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.342 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.342 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.342 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.343 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:13:15.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.343 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:15.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:15.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:13:15.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:13:15.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:13:15.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:13:15.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:13:15.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:13:15.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:13:15.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:13:15.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:13:15.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:13:15.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:13:15.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:13:15.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:13:15.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:13:15.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:13:15.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:13:15.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:13:15.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:13:15.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:13:15.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:13:15.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:13:15.348 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.348 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.348 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.348 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.348 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:15.348 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.783119628000001, 2024-09-13T09:13:15.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:13:15.349 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.349 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.349 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.349 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.349 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.349 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:13:15.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.349 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:15.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:15.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:13:15.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:13:15.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:13:15.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:13:15.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:13:15.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:13:15.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:13:15.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:13:15.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:13:15.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:13:15.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:13:15.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:13:15.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:13:15.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:13:15.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:13:15.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:13:15.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:13:15.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:13:15.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:13:15.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:13:15.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:13:15.354 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.354 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.354 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.354 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.355 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:15.355 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.783101399000003, 2024-09-13T09:13:15.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:13:15.355 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.355 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.355 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.355 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.355 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.355 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:13:15.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.355 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:15.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:15.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:15.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.357 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:13:15.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:13:15.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.357 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:13:15.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:13:15.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.357 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:13:15.358 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:13:15.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.358 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.358 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:13:15.358 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:13:15.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.358 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.358 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:13:15.358 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:13:15.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.358 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:13:15.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:13:15.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.359 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:13:15.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:13:15.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.359 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:13:15.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:13:15.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.360 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.360 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:13:15.360 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:13:15.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.360 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.360 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:13:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:13:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.783054079999999, 2024-09-13T09:13:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:13:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.361 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.362 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.362 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.362 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:13:15.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.362 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:15.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:15.363 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.363 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.363 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:15.363 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.363 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.363 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:13:15.363 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:13:15.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.364 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.364 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:13:15.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:13:15.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.364 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.364 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:13:15.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:13:15.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.364 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.365 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:13:15.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:13:15.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.365 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.365 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:13:15.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:13:15.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.365 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.365 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:13:15.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:13:15.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:13:15.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:13:15.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:13:15.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:13:15.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.367 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:13:15.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:13:15.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.367 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:13:15.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:13:15.367 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.367 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.367 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.368 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.368 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:15.368 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.782539841999998, 2024-09-13T09:13:15.368 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:13:15.368 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.368 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.368 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.368 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.368 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.368 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:13:15.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.369 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:15.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:15.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:13:15.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:13:15.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:13:15.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:13:15.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:13:15.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:13:15.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:13:15.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:13:15.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:13:15.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:13:15.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:13:15.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:13:15.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:13:15.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:13:15.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:13:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:13:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:13:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:13:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:13:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:13:15.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:13:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:13:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.782505311999998, 2024-09-13T09:13:15.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:13:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:13:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:15.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:13:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:13:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:13:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:13:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:13:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:13:15.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:13:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:13:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:13:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:13:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:13:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:13:15.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:13:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:13:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:13:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:13:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:13:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:13:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:13:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:13:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.782487404999998, 2024-09-13T09:13:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:13:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:13:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:13:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:13:15.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:13:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:13:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:13:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:13:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:13:15.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:13:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:13:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:13:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:13:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:13:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:13:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:13:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.773360556999997, 2024-09-13T09:13:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:13:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.386 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:13:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:13:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:13:15.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:13:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:13:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:13:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:13:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:13:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:13:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:13:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:13:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:13:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:13:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:13:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:13:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:13:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:13:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:13:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:13:15.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:13:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.773308976999999, 2024-09-13T09:13:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:13:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:13:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:13:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:13:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:13:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:13:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:13:15.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:13:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:13:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:13:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:13:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:13:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:13:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:13:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:13:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:13:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:13:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:13:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:13:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:13:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:13:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:13:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:13:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.397 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.773292701000003, 2024-09-13T09:13:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:13:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:13:15.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.400 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:13:15.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:13:15.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.400 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:13:15.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:13:15.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:13:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:13:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:13:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:13:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.401 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:13:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:13:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:13:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:13:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:13:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:13:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:13:15.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:13:15.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:13:15.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:13:15.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:13:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:13:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.773276926999998, 2024-09-13T09:13:15.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:13:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:13:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:15.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:13:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:13:15.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:13:15.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:13:15.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:13:15.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:13:15.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:13:15.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:13:15.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:13:15.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:13:15.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:13:15.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:13:15.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:13:15.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:13:15.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:13:15.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:13:15.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:13:15.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:13:15.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:13:15.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:13:15.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:13:15.410 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.410 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.411 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.411 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.411 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:15.411 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.773262052, 2024-09-13T09:13:15.411 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:13:15.411 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.411 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.411 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.411 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.411 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.411 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:13:15.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.412 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:15.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:15.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:15.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:13:15.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:13:15.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:13:15.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:13:15.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:13:15.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:13:15.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:13:15.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:13:15.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:13:15.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:13:15.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.415 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:13:15.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:13:15.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.415 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:13:15.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:13:15.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:13:15.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:13:15.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:13:15.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:13:15.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:13:15.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:13:15.416 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.417 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.417 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.417 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.417 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:15.417 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.773244589999997, 2024-09-13T09:13:15.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:13:15.417 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.417 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.417 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.417 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.418 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.418 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:13:15.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.418 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.418 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.418 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:15.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.418 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.418 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:15.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:13:15.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:13:15.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:13:15.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:13:15.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:13:15.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:13:15.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:13:15.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:13:15.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:13:15.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:13:15.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:13:15.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:13:15.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:13:15.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:13:15.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:13:15.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:13:15.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:13:15.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:13:15.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:13:15.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:13:15.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:13:15.423 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.423 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.423 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.423 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.423 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:15.423 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.771643957999999, 2024-09-13T09:13:15.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:13:15.423 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.423 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.424 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.424 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.424 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.424 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:13:15.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.424 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.424 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:15.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.424 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:15.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:13:15.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:13:15.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:13:15.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:13:15.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:13:15.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:13:15.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:13:15.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:13:15.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:13:15.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:13:15.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:13:15.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:13:15.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:13:15.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:13:15.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:13:15.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:13:15.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:13:15.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:13:15.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:13:15.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:13:15.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:13:15.429 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.429 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.429 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.429 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.429 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:15.429 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.771603906999999, 2024-09-13T09:13:15.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:13:15.430 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.430 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.430 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.430 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.430 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.430 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:13:15.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.430 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.430 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.430 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:15.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:15.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:13:15.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:13:15.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:13:15.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:13:15.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:13:15.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:13:15.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:13:15.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:13:15.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:13:15.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:13:15.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:13:15.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:13:15.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:13:15.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:13:15.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:13:15.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:13:15.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:13:15.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:13:15.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:13:15.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:13:15.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:13:15.435 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.435 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.435 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.435 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.435 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:15.435 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.771560143999999, 2024-09-13T09:13:15.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:13:15.436 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.436 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.436 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.436 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.436 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.436 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:13:15.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.436 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:15.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:15.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:15.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:13:15.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:13:15.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.438 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.438 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:13:15.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:13:15.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.438 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.438 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:13:15.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:13:15.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:13:15.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:13:15.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:13:15.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:13:15.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:13:15.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:13:15.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:15.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:13:15.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:13:15.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:13:15.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:13:15.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:13:15.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:13:15.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:13:15.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:13:15.441 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.441 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.441 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.441 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.441 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.441 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:15.443 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:15.441+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:15.443 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-13T09:13:15.443 INFO:teuthology.orchestra.run.smithi157.stdout:3553 read 0x903b800 thru 0x9046981 (0xb182 bytes) 2024-09-13T09:13:15.443 INFO:teuthology.orchestra.run.smithi157.stdout:3555 punch from 0xbec00 to 0xc0800, (0x1c00 bytes) 2024-09-13T09:13:15.444 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:15.517 INFO:teuthology.orchestra.run.smithi157.stdout:3556 write 0x4eecc00 thru 0x4eecfe0 (0x3e1 bytes) 2024-09-13T09:13:15.520 INFO:teuthology.orchestra.run.smithi157.stdout:3558 write 0xc42000 thru 0xc4f71c (0xd71d bytes) 2024-09-13T09:13:15.521 INFO:teuthology.orchestra.run.smithi157.stdout:3560 punch from 0x164be00 to 0x1652000, (0x6200 bytes) 2024-09-13T09:13:15.544 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:15.542+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:15.604 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:15.604 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:15.605 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:15.605 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:15.616 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:15.660 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:15.659+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:15.747 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:15.745+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:15.751 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:15.751 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:15.751 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:15.751 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:15.751 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:15.762 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:15.847 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:15.846+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:15.911 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:15.911 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:15.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:15.911 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:15.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.912 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.912 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:15.912 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405899241, 2024-09-13T09:13:15.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:13:15.912 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.912 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.912 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.912 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.912 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.912 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:13:15.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.913 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:15.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:15.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:13:15.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:13:15.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:13:15.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:13:15.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:13:15.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:13:15.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:13:15.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:13:15.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:13:15.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:13:15.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:13:15.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:13:15.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:13:15.916 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.916 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.916 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.916 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.916 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:15.916 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405871556999998, 2024-09-13T09:13:15.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:13:15.916 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.917 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.917 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.917 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.917 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.917 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:13:15.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.917 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:15.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:15.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:15.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:13:15.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:13:15.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:13:15.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:15.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:13:15.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:13:15.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:13:15.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:13:15.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:13:15.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:13:15.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:13:15.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:13:15.920 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.920 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.921 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.921 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.921 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:15.921 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405847215999998, 2024-09-13T09:13:15.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:13:15.921 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.921 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.921 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.921 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.921 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.921 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:13:15.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.922 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:15.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:15.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:13:15.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:13:15.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:13:15.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:13:15.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:13:15.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:13:15.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:13:15.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:13:15.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:13:15.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:13:15.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:13:15.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:13:15.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:13:15.925 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.925 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.925 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.925 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.925 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:15.925 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405830895999998, 2024-09-13T09:13:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:13:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:13:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:15.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:15.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:13:15.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:15.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:13:15.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:13:15.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:13:15.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:13:15.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.929 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:13:15.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:13:15.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.929 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.929 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:13:15.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:13:15.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.929 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.929 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:13:15.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:13:15.929 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.930 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.930 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.930 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.930 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:15.930 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405814481, 2024-09-13T09:13:15.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:13:15.930 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.930 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.930 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.930 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.930 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.930 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:13:15.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.932 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:15.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:13:15.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:13:15.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:13:15.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:13:15.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:13:15.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:13:15.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:13:15.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:13:15.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:13:15.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:13:15.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:13:15.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:13:15.936 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.936 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.936 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.936 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.936 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:15.936 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405798373000003, 2024-09-13T09:13:15.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:13:15.936 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.936 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.936 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.936 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.937 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.937 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:13:15.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.937 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:15.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:15.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:15.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:13:15.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:13:15.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:13:15.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:13:15.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:13:15.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:13:15.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:13:15.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:13:15.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:13:15.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:13:15.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:13:15.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:13:15.940 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.940 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.940 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.941 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.941 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:15.941 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405782811999998, 2024-09-13T09:13:15.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:13:15.941 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.941 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.941 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.941 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.941 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.941 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:13:15.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.941 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:15.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:15.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:15.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:13:15.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:13:15.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:13:15.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:13:15.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:13:15.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:13:15.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:13:15.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:13:15.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:13:15.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:13:15.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:13:15.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:13:15.945 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.945 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.945 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.945 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.945 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:15.945 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405765973999998, 2024-09-13T09:13:15.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:13:15.945 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.945 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.946 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.946 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.946 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.946 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:13:15.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.946 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:15.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:15.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:13:15.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:13:15.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:13:15.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:13:15.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:13:15.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:13:15.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:13:15.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:13:15.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:13:15.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:13:15.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:13:15.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:13:15.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:13:15.949 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.949 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.949 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.950 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.950 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:15.950 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405749982000003, 2024-09-13T09:13:15.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:13:15.950 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.950 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.950 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.950 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.950 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.950 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:13:15.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.951 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:15.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:15.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:13:15.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:13:15.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:13:15.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:13:15.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:13:15.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:13:15.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:13:15.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:13:15.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:13:15.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:13:15.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:13:15.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:13:15.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:13:15.954 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.954 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.954 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.954 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.954 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:15.954 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405722484000002, 2024-09-13T09:13:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:13:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:13:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:15.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:15.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:13:15.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:13:15.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:13:15.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:13:15.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:13:15.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:13:15.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:13:15.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:13:15.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:13:15.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:13:15.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:13:15.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:13:15.958 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.958 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.959 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.959 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.959 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:15.959 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405696624999997, 2024-09-13T09:13:15.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:13:15.959 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.959 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.959 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.959 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.959 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.959 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:13:15.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.960 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:15.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:15.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:15.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:13:15.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:13:15.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:13:15.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:13:15.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:13:15.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:13:15.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:13:15.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:13:15.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:13:15.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:13:15.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:13:15.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:13:15.963 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.963 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.963 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.963 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.963 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:15.963 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405655135000003, 2024-09-13T09:13:15.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:13:15.964 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.964 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.964 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.964 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.964 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.964 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:13:15.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.964 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:15.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:15.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:13:15.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:13:15.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:13:15.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:13:15.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:13:15.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:13:15.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:13:15.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:13:15.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:13:15.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:13:15.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:13:15.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:13:15.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:13:15.967 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.968 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.968 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.968 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.968 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:15.968 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405633897000001, 2024-09-13T09:13:15.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:13:15.968 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.968 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.968 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.968 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.968 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.969 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:13:15.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.969 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:15.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:15.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:13:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:13:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:13:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:13:15.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:13:15.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:13:15.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:13:15.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:13:15.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:13:15.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:13:15.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:13:15.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:13:15.972 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.972 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.972 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.972 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.972 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:15.973 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405611096999998, 2024-09-13T09:13:15.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:13:15.973 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.973 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.973 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.973 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.973 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.973 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:13:15.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.973 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:15.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:15.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:13:15.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:13:15.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:13:15.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:13:15.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:15.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:13:15.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:13:15.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:13:15.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:13:15.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:13:15.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:13:15.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:13:15.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:13:15.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.977 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.977 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.977 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:15.977 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405594641, 2024-09-13T09:13:15.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:13:15.977 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.977 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.977 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.977 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.977 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.978 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:13:15.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.978 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:15.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:15.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:15.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:13:15.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:13:15.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:13:15.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:13:15.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:13:15.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:13:15.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:13:15.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:13:15.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:13:15.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:13:15.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:13:15.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:13:15.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.981 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.981 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.982 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:15.982 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.405349504, 2024-09-13T09:13:15.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:13:15.982 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.982 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.982 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.982 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.982 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.982 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:13:15.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.982 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:15.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:15.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:13:15.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:13:15.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:13:15.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:13:15.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:13:15.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:13:15.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:13:15.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:15.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:13:15.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:13:15.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:13:15.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:13:15.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:13:15.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:13:15.986 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.986 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.986 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.986 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.986 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:15.986 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.403463944999999, 2024-09-13T09:13:15.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:13:15.986 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.986 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.986 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.987 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.987 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.987 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:13:15.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.987 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:15.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:15.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:13:15.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:13:15.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:13:15.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:13:15.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:13:15.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:13:15.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:13:15.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:13:15.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:13:15.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:13:15.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:13:15.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:13:15.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:13:15.990 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.990 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.990 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.991 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.991 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:15.991 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.403442579, 2024-09-13T09:13:15.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:13:15.991 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.991 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.991 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.991 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.991 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.991 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:13:15.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.991 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:15.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:15.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:15.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:15.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:13:15.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:13:15.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:13:15.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:13:15.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:13:15.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:13:15.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:13:15.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:13:15.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:13:15.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:13:15.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:13:15.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:13:15.995 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.995 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.995 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.995 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:15.995 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:15.995 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.403237210999997, 2024-09-13T09:13:15.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:13:15.995 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:15.995 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:15.996 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:15.996 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:15.996 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:15.996 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:13:15.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.996 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:15.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:15.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:15.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:15.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:15.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:15.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:15.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:13:15.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:13:15.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:15.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:13:15.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:13:15.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:15.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:13:15.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:13:15.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:15.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:13:15.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:13:15.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:15.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:13:15.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:13:15.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:15.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:13:15.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:13:15.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:15.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:15.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:15.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:13:15.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:13:15.999 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.999 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:15.999 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:15.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.000 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5272.0:1775 3.0 3:02a42fca:::rbd_data.11235782ee8d.0000000000000030:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2244608~40448] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:13:16.000 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:16.000 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.6616953700000003, 2024-09-13T09:13:16.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.666843732, 2024-09-13T09:13:16.000 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.000 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:16.000 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.000 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5272", 2024-09-13T09:13:16.000 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4089902097", 2024-09-13T09:13:16.000 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1775 2024-09-13T09:13:16.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.001 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:16.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:16.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247780+0000", 2024-09-13T09:13:16.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:13:16.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247865+0000", 2024-09-13T09:13:16.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4359999999999999e-05 2024-09-13T09:13:16.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247876+0000", 2024-09-13T09:13:16.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1456e-05 2024-09-13T09:13:16.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247883+0000", 2024-09-13T09:13:16.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:13:16.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247926+0000", 2024-09-13T09:13:16.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3203e-05 2024-09-13T09:13:16.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:16.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247932+0000", 2024-09-13T09:13:16.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:13:16.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909639+0000", 2024-09-13T09:13:16.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6617069099999999 2024-09-13T09:13:16.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:13:16.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909828+0000", 2024-09-13T09:13:16.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000189075 2024-09-13T09:13:16.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911676+0000", 2024-09-13T09:13:16.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001847235 2024-09-13T09:13:16.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911765+0000", 2024-09-13T09:13:16.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9530000000000005e-05 2024-09-13T09:13:16.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:13:16.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911901+0000", 2024-09-13T09:13:16.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013603899999999999 2024-09-13T09:13:16.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:13:16.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.912889+0000", 2024-09-13T09:13:16.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098807400000000003 2024-09-13T09:13:16.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:16.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914438+0000", 2024-09-13T09:13:16.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015491070000000001 2024-09-13T09:13:16.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:13:16.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914505+0000", 2024-09-13T09:13:16.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:13:16.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914625+0000", 2024-09-13T09:13:16.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120108 2024-09-13T09:13:16.007 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.007 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.007 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.007 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.007 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.007 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:16.008 INFO:teuthology.orchestra.run.smithi157.stdout:3561 trunc from 0xb97e400 to 0x258ac00 2024-09-13T09:13:16.008 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:16.006+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:16.009 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:13:16.109 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:16.108+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:16.145 INFO:teuthology.orchestra.run.smithi157.stdout:3565 trunc from 0x258ac00 to 0x5129600 2024-09-13T09:13:16.166 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:16.166 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:16.166 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:16.166 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:16.172 INFO:teuthology.orchestra.run.smithi157.stdout:3566 write 0xe08dc00 thru 0xe09aa58 (0xce59 bytes) 2024-09-13T09:13:16.173 INFO:teuthology.orchestra.run.smithi157.stdout:3568 punch from 0xb29f200 to 0xb2a1600, (0x2400 bytes) 2024-09-13T09:13:16.175 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:13:16.182 INFO:teuthology.orchestra.run.smithi157.stdout:3570 trunc from 0xe09ac00 to 0x6bd4c00 2024-09-13T09:13:16.200 INFO:teuthology.orchestra.run.smithi157.stdout:3571 read 0x42b3e00 thru 0x42bce4e (0x904f bytes) 2024-09-13T09:13:16.200 INFO:teuthology.orchestra.run.smithi157.stdout:3572 punch from 0x117b600 to 0x1180400, (0x4e00 bytes) 2024-09-13T09:13:16.205 INFO:teuthology.orchestra.run.smithi157.stdout:3573 read 0x4dda00 thru 0x4e8c1e (0xb21f bytes) 2024-09-13T09:13:16.216 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:16.215+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:16.245 INFO:teuthology.orchestra.run.smithi157.stdout:3575 read 0x3f55400 thru 0x3f63e39 (0xea3a bytes) 2024-09-13T09:13:16.245 INFO:teuthology.orchestra.run.smithi157.stdout:3578 read 0x6b42400 thru 0x6b4e715 (0xc316 bytes) 2024-09-13T09:13:16.245 INFO:teuthology.orchestra.run.smithi157.stdout:3579 punch from 0x221a800 to 0x2227800, (0xd000 bytes) 2024-09-13T09:13:16.266 INFO:teuthology.orchestra.run.smithi157.stdout:3580 write 0xc561000 thru 0xc56e6ed (0xd6ee bytes) 2024-09-13T09:13:16.266 INFO:teuthology.orchestra.run.smithi157.stdout:3581 read 0x887f600 thru 0x888e208 (0xec09 bytes) 2024-09-13T09:13:16.266 INFO:teuthology.orchestra.run.smithi157.stdout:3583 write 0x25a200 thru 0x262328 (0x8129 bytes) 2024-09-13T09:13:16.268 INFO:teuthology.orchestra.run.smithi157.stdout:3584 trunc from 0xc56e800 to 0xbb84600 2024-09-13T09:13:16.293 INFO:teuthology.orchestra.run.smithi157.stdout:3586 flatten 2024-09-13T09:13:16.305 INFO:teuthology.orchestra.run.smithi157.stdout:3587 write 0x3f37a00 thru 0x3f3bc26 (0x4227 bytes) 2024-09-13T09:13:16.306 INFO:teuthology.orchestra.run.smithi157.stdout:3588 write 0xad06c00 thru 0xad09ab1 (0x2eb2 bytes) 2024-09-13T09:13:16.307 INFO:teuthology.orchestra.run.smithi157.stdout:3589 write 0x7777a00 thru 0x778271c (0xad1d bytes) 2024-09-13T09:13:16.311 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:16.310+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:16.335 INFO:teuthology.orchestra.run.smithi157.stdout:3590 write 0xe770600 thru 0xe77078e (0x18f bytes) 2024-09-13T09:13:16.335 INFO:teuthology.orchestra.run.smithi157.stdout:3592 read 0x508d600 thru 0x5091c1e (0x461f bytes) 2024-09-13T09:13:16.335 INFO:teuthology.orchestra.run.smithi157.stdout:3596 read 0x719600 thru 0x71dbd7 (0x45d8 bytes) 2024-09-13T09:13:16.336 INFO:teuthology.orchestra.run.smithi157.stdout:3598 punch from 0x81faa00 to 0x8207600, (0xcc00 bytes) 2024-09-13T09:13:16.336 INFO:teuthology.orchestra.run.smithi157.stdout:3599 read 0x18e800 thru 0x18f006 (0x807 bytes) 2024-09-13T09:13:16.339 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:16.339 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:16.339 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:16.339 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:16.339 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:16.343 INFO:teuthology.orchestra.run.smithi157.stdout:3600 write 0x878aa00 thru 0x8795ae4 (0xb0e5 bytes) 2024-09-13T09:13:16.343 INFO:teuthology.orchestra.run.smithi157.stdout:3602 read 0x9ea6c00 thru 0x9eac924 (0x5d25 bytes) 2024-09-13T09:13:16.343 INFO:teuthology.orchestra.run.smithi157.stdout:3605 write 0x25a9400 thru 0x25afb91 (0x6792 bytes) 2024-09-13T09:13:16.344 INFO:teuthology.orchestra.run.smithi157.stdout:3607 read 0x910c00 thru 0x91d5f1 (0xc9f2 bytes) 2024-09-13T09:13:16.347 INFO:teuthology.orchestra.run.smithi157.stdout:3608 read 0x71f2000 thru 0x71f894a (0x694b bytes) 2024-09-13T09:13:16.347 INFO:teuthology.orchestra.run.smithi157.stdout:3610 read 0x1085a00 thru 0x109499d (0xef9e bytes) 2024-09-13T09:13:16.348 INFO:teuthology.orchestra.run.smithi157.stdout:3612 read 0xf56800 thru 0xf6639a (0xfb9b bytes) 2024-09-13T09:13:16.349 INFO:teuthology.orchestra.run.smithi157.stdout:3613 read 0x246000 thru 0x24a85d (0x485e bytes) 2024-09-13T09:13:16.350 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:13:16.351 INFO:teuthology.orchestra.run.smithi157.stdout:3614 write 0x5af6200 thru 0x5afba66 (0x5867 bytes) 2024-09-13T09:13:16.351 INFO:teuthology.orchestra.run.smithi157.stdout:3615 punch from 0x1a40800 to 0x1a43800, (0x3000 bytes) 2024-09-13T09:13:16.351 INFO:teuthology.orchestra.run.smithi157.stdout:3616 punch from 0x24dc400 to 0x24e1800, (0x5400 bytes) 2024-09-13T09:13:16.351 INFO:teuthology.orchestra.run.smithi157.stdout:truncating to largest ever: 0xed61e00 2024-09-13T09:13:16.351 INFO:teuthology.orchestra.run.smithi157.stdout:3619 trunc from 0xe770800 to 0xed61e00 2024-09-13T09:13:16.365 INFO:teuthology.orchestra.run.smithi157.stdout:3620 read 0x7a45800 thru 0x7a545e7 (0xede8 bytes) 2024-09-13T09:13:16.367 INFO:teuthology.orchestra.run.smithi157.stdout:3622 punch from 0x39bac00 to 0x39c8200, (0xd600 bytes) 2024-09-13T09:13:16.367 INFO:teuthology.orchestra.run.smithi157.stdout:3623 write 0x8797a00 thru 0x879b875 (0x3e76 bytes) 2024-09-13T09:13:16.368 INFO:teuthology.orchestra.run.smithi157.stdout:3625 read 0xdf83e00 thru 0xdf89c3a (0x5e3b bytes) 2024-09-13T09:13:16.370 INFO:teuthology.orchestra.run.smithi157.stdout:3628 trunc from 0xed61e00 to 0xce20c00 2024-09-13T09:13:16.392 INFO:teuthology.orchestra.run.smithi157.stdout:3629 punch from 0xa20b200 to 0xa20d000, (0x1e00 bytes) 2024-09-13T09:13:16.392 INFO:teuthology.orchestra.run.smithi157.stdout:3630 write 0x2823e00 thru 0x282478b (0x98c bytes) 2024-09-13T09:13:16.393 INFO:teuthology.orchestra.run.smithi157.stdout:3631 punch from 0xc28a400 to 0xc28e600, (0x4200 bytes) 2024-09-13T09:13:16.393 INFO:teuthology.orchestra.run.smithi157.stdout:3633 punch from 0x813b800 to 0x8146200, (0xaa00 bytes) 2024-09-13T09:13:16.393 INFO:teuthology.orchestra.run.smithi157.stdout:3634 punch from 0x54edc00 to 0x54ee600, (0xa00 bytes) 2024-09-13T09:13:16.393 INFO:teuthology.orchestra.run.smithi157.stdout:3635 write 0xedc600 thru 0xee9eff (0xd900 bytes) 2024-09-13T09:13:16.404 INFO:teuthology.orchestra.run.smithi157.stdout:3636 trunc from 0xce20c00 to 0x5359800 2024-09-13T09:13:16.435 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:16.434+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:16.443 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:13:16.450 INFO:teuthology.orchestra.run.smithi157.stdout:3640 write 0x7cfe200 thru 0x7d0b45d (0xd25e bytes) 2024-09-13T09:13:16.450 INFO:teuthology.orchestra.run.smithi157.stdout:3643 punch from 0x3273800 to 0x327c400, (0x8c00 bytes) 2024-09-13T09:13:16.450 INFO:teuthology.orchestra.run.smithi157.stdout:3650 write 0x5285600 thru 0x52936bf (0xe0c0 bytes) 2024-09-13T09:13:16.477 INFO:teuthology.orchestra.run.smithi157.stdout:3651 write 0x9c96e00 thru 0x9c9c637 (0x5838 bytes) 2024-09-13T09:13:16.485 INFO:teuthology.orchestra.run.smithi157.stdout:3652 trunc from 0x9c9c800 to 0x321a600 2024-09-13T09:13:16.504 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:13:16.512 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:16.511+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:16.516 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:16.516 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:16.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:16.516 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:16.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.516 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:6867 3.13 3:cd293326:::rbd_header.112326fb43d3:head [watch reconnect cookie 140383087120544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T09:13:16.516 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:13:16.516 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.98534093, 2024-09-13T09:13:16.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0277181360000001, 2024-09-13T09:13:16.517 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.517 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.517 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.517 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:13:16.517 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:13:16.517 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6867 2024-09-13T09:13:16.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.517 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:13:16.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:13:16.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529067+0000", 2024-09-13T09:13:16.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-09-13T09:13:16.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529073+0000", 2024-09-13T09:13:16.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:13:16.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529087+0000", 2024-09-13T09:13:16.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3825000000000001e-05 2024-09-13T09:13:16.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529097+0000", 2024-09-13T09:13:16.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9559999999999992e-06 2024-09-13T09:13:16.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.520 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529149+0000", 2024-09-13T09:13:16.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1668000000000001e-05 2024-09-13T09:13:16.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:16.520 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529161+0000", 2024-09-13T09:13:16.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1831000000000001e-05 2024-09-13T09:13:16.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.520 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.554941+0000", 2024-09-13T09:13:16.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0257799729999999 2024-09-13T09:13:16.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.555239+0000", 2024-09-13T09:13:16.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00029829100000000001 2024-09-13T09:13:16.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.556787+0000", 2024-09-13T09:13:16.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001547617 2024-09-13T09:13:16.521 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.521 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.521 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.522 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20898 3.13 3:cd293326:::rbd_header.112326fb43d3:head [watch reconnect cookie 140642395970896 gen 16] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T09:13:16.522 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:13:16.522 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.980678112, 2024-09-13T09:13:16.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0236151950000001, 2024-09-13T09:13:16.522 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.522 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.522 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.522 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:16.522 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:16.522 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20898 2024-09-13T09:13:16.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.522 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:13:16.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:13:16.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533729+0000", 2024-09-13T09:13:16.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-09-13T09:13:16.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533736+0000", 2024-09-13T09:13:16.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9670000000000004e-06 2024-09-13T09:13:16.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533749+0000", 2024-09-13T09:13:16.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2449000000000001e-05 2024-09-13T09:13:16.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533757+0000", 2024-09-13T09:13:16.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:13:16.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533813+0000", 2024-09-13T09:13:16.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5683000000000002e-05 2024-09-13T09:13:16.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:16.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533823+0000", 2024-09-13T09:13:16.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0168e-05 2024-09-13T09:13:16.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.555940+0000", 2024-09-13T09:13:16.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0221170559999999 2024-09-13T09:13:16.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.556047+0000", 2024-09-13T09:13:16.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000106881 2024-09-13T09:13:16.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.557347+0000", 2024-09-13T09:13:16.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001299323 2024-09-13T09:13:16.527 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.527 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.527 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.527 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4447 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.527 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581680+0000", 2024-09-13T09:13:16.527 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93272980800000005, 2024-09-13T09:13:16.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054674652999999997, 2024-09-13T09:13:16.527 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.527 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.527 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.528 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.528 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.528 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4447 2024-09-13T09:13:16.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.528 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581680+0000", 2024-09-13T09:13:16.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581680+0000", 2024-09-13T09:13:16.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581679+0000", 2024-09-13T09:13:16.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581682+0000", 2024-09-13T09:13:16.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4830000000000002e-06 2024-09-13T09:13:16.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581688+0000", 2024-09-13T09:13:16.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0569999999999998e-06 2024-09-13T09:13:16.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581692+0000", 2024-09-13T09:13:16.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6820000000000002e-06 2024-09-13T09:13:16.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636201+0000", 2024-09-13T09:13:16.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054508845 2024-09-13T09:13:16.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636293+0000", 2024-09-13T09:13:16.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1477000000000005e-05 2024-09-13T09:13:16.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636354+0000", 2024-09-13T09:13:16.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1651999999999999e-05 2024-09-13T09:13:16.531 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.531 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.531 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.532 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4448 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 913408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.532 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581696+0000", 2024-09-13T09:13:16.532 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93271310699999999, 2024-09-13T09:13:16.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054880384999999997, 2024-09-13T09:13:16.532 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.532 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.532 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.532 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.532 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.532 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4448 2024-09-13T09:13:16.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.532 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581696+0000", 2024-09-13T09:13:16.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581696+0000", 2024-09-13T09:13:16.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581696+0000", 2024-09-13T09:13:16.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581702+0000", 2024-09-13T09:13:16.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4250000000000003e-06 2024-09-13T09:13:16.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581716+0000", 2024-09-13T09:13:16.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3196e-05 2024-09-13T09:13:16.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581721+0000", 2024-09-13T09:13:16.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0189999999999997e-06 2024-09-13T09:13:16.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636390+0000", 2024-09-13T09:13:16.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054669061999999997 2024-09-13T09:13:16.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636479+0000", 2024-09-13T09:13:16.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9188000000000001e-05 2024-09-13T09:13:16.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636577+0000", 2024-09-13T09:13:16.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7827999999999999e-05 2024-09-13T09:13:16.536 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.536 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.536 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.536 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4449 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.536 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581725+0000", 2024-09-13T09:13:16.536 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93268426800000004, 2024-09-13T09:13:16.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055074856999999998, 2024-09-13T09:13:16.536 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.536 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.536 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.536 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.537 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.537 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4449 2024-09-13T09:13:16.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.537 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581725+0000", 2024-09-13T09:13:16.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581725+0000", 2024-09-13T09:13:16.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581725+0000", 2024-09-13T09:13:16.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581727+0000", 2024-09-13T09:13:16.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5969999999999999e-06 2024-09-13T09:13:16.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581733+0000", 2024-09-13T09:13:16.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.088e-06 2024-09-13T09:13:16.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581738+0000", 2024-09-13T09:13:16.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:13:16.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636614+0000", 2024-09-13T09:13:16.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054875637999999997 2024-09-13T09:13:16.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636703+0000", 2024-09-13T09:13:16.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8838000000000006e-05 2024-09-13T09:13:16.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636800+0000", 2024-09-13T09:13:16.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7282999999999997e-05 2024-09-13T09:13:16.540 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.540 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.540 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.540 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4450 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.540 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581743+0000", 2024-09-13T09:13:16.540 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93266675099999996, 2024-09-13T09:13:16.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055241622999999997, 2024-09-13T09:13:16.541 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.541 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.541 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.541 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.541 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.541 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4450 2024-09-13T09:13:16.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.541 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.541 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581743+0000", 2024-09-13T09:13:16.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581743+0000", 2024-09-13T09:13:16.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581742+0000", 2024-09-13T09:13:16.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581745+0000", 2024-09-13T09:13:16.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.294e-06 2024-09-13T09:13:16.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581751+0000", 2024-09-13T09:13:16.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.516e-06 2024-09-13T09:13:16.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581758+0000", 2024-09-13T09:13:16.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7780000000000002e-06 2024-09-13T09:13:16.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636860+0000", 2024-09-13T09:13:16.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055102263999999998 2024-09-13T09:13:16.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636927+0000", 2024-09-13T09:13:16.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6427000000000006e-05 2024-09-13T09:13:16.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636984+0000", 2024-09-13T09:13:16.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7739999999999999e-05 2024-09-13T09:13:16.546 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.546 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.546 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.547 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4451 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.547 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581763+0000", 2024-09-13T09:13:16.547 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93264640799999998, 2024-09-13T09:13:16.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055339227999999997, 2024-09-13T09:13:16.547 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.547 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.547 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.547 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.547 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.547 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4451 2024-09-13T09:13:16.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.547 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581763+0000", 2024-09-13T09:13:16.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581763+0000", 2024-09-13T09:13:16.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581763+0000", 2024-09-13T09:13:16.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581765+0000", 2024-09-13T09:13:16.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4030000000000001e-06 2024-09-13T09:13:16.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581771+0000", 2024-09-13T09:13:16.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4189999999999999e-06 2024-09-13T09:13:16.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581776+0000", 2024-09-13T09:13:16.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5739999999999997e-06 2024-09-13T09:13:16.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637002+0000", 2024-09-13T09:13:16.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055225819000000002 2024-09-13T09:13:16.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637044+0000", 2024-09-13T09:13:16.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1792999999999998e-05 2024-09-13T09:13:16.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637102+0000", 2024-09-13T09:13:16.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8644000000000003e-05 2024-09-13T09:13:16.551 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.551 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.551 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.551 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4452 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.551 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581780+0000", 2024-09-13T09:13:16.551 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93262935199999997, 2024-09-13T09:13:16.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055635109000000002, 2024-09-13T09:13:16.551 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.551 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.551 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.552 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.552 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.552 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4452 2024-09-13T09:13:16.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.552 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581780+0000", 2024-09-13T09:13:16.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581780+0000", 2024-09-13T09:13:16.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581780+0000", 2024-09-13T09:13:16.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581782+0000", 2024-09-13T09:13:16.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2790000000000001e-06 2024-09-13T09:13:16.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581788+0000", 2024-09-13T09:13:16.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9460000000000003e-06 2024-09-13T09:13:16.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581793+0000", 2024-09-13T09:13:16.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:13:16.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637139+0000", 2024-09-13T09:13:16.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055346237 2024-09-13T09:13:16.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637241+0000", 2024-09-13T09:13:16.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000101924 2024-09-13T09:13:16.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637415+0000", 2024-09-13T09:13:16.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017448400000000001 2024-09-13T09:13:16.555 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.555 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.555 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.556 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4453 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.556 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581797+0000", 2024-09-13T09:13:16.556 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93261270799999996, 2024-09-13T09:13:16.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055904331000000002, 2024-09-13T09:13:16.556 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.556 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.556 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.556 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.556 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.556 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4453 2024-09-13T09:13:16.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.556 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581797+0000", 2024-09-13T09:13:16.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581797+0000", 2024-09-13T09:13:16.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581796+0000", 2024-09-13T09:13:16.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581799+0000", 2024-09-13T09:13:16.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:13:16.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581804+0000", 2024-09-13T09:13:16.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8579999999999996e-06 2024-09-13T09:13:16.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581815+0000", 2024-09-13T09:13:16.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0219e-05 2024-09-13T09:13:16.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637446+0000", 2024-09-13T09:13:16.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055631667000000003 2024-09-13T09:13:16.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637490+0000", 2024-09-13T09:13:16.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3223000000000003e-05 2024-09-13T09:13:16.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637701+0000", 2024-09-13T09:13:16.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00021151500000000001 2024-09-13T09:13:16.560 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.560 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.560 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.560 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4454 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 962560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.560 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581819+0000", 2024-09-13T09:13:16.560 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93259012299999999, 2024-09-13T09:13:16.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056027577000000002, 2024-09-13T09:13:16.560 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.560 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.560 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.561 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.561 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.561 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4454 2024-09-13T09:13:16.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.561 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581819+0000", 2024-09-13T09:13:16.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581819+0000", 2024-09-13T09:13:16.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581819+0000", 2024-09-13T09:13:16.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581822+0000", 2024-09-13T09:13:16.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7020000000000001e-06 2024-09-13T09:13:16.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581828+0000", 2024-09-13T09:13:16.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.489e-06 2024-09-13T09:13:16.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581833+0000", 2024-09-13T09:13:16.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:13:16.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637742+0000", 2024-09-13T09:13:16.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055908823000000003 2024-09-13T09:13:16.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637786+0000", 2024-09-13T09:13:16.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3791999999999997e-05 2024-09-13T09:13:16.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637847+0000", 2024-09-13T09:13:16.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1371000000000006e-05 2024-09-13T09:13:16.564 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.565 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.565 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.565 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4455 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 970752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.565 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581837+0000", 2024-09-13T09:13:16.565 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93257225300000002, 2024-09-13T09:13:16.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056132979, 2024-09-13T09:13:16.565 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.565 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.565 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.565 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.565 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.566 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4455 2024-09-13T09:13:16.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.566 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581837+0000", 2024-09-13T09:13:16.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581837+0000", 2024-09-13T09:13:16.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581837+0000", 2024-09-13T09:13:16.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581839+0000", 2024-09-13T09:13:16.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.373e-06 2024-09-13T09:13:16.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581845+0000", 2024-09-13T09:13:16.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:13:16.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581850+0000", 2024-09-13T09:13:16.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5140000000000003e-06 2024-09-13T09:13:16.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637877+0000", 2024-09-13T09:13:16.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056027146999999999 2024-09-13T09:13:16.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637918+0000", 2024-09-13T09:13:16.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0911000000000002e-05 2024-09-13T09:13:16.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637970+0000", 2024-09-13T09:13:16.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2466999999999998e-05 2024-09-13T09:13:16.569 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.569 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.569 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.569 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4456 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.569 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581860+0000", 2024-09-13T09:13:16.570 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93254903300000003, 2024-09-13T09:13:16.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056264427999999998, 2024-09-13T09:13:16.570 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.570 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.570 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.570 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.570 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.570 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4456 2024-09-13T09:13:16.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.570 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581860+0000", 2024-09-13T09:13:16.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581860+0000", 2024-09-13T09:13:16.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581860+0000", 2024-09-13T09:13:16.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581862+0000", 2024-09-13T09:13:16.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3499999999999999e-06 2024-09-13T09:13:16.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581869+0000", 2024-09-13T09:13:16.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4239999999999998e-06 2024-09-13T09:13:16.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581873+0000", 2024-09-13T09:13:16.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7110000000000003e-06 2024-09-13T09:13:16.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638000+0000", 2024-09-13T09:13:16.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056126545999999999 2024-09-13T09:13:16.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638069+0000", 2024-09-13T09:13:16.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8946e-05 2024-09-13T09:13:16.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638125+0000", 2024-09-13T09:13:16.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5826999999999999e-05 2024-09-13T09:13:16.573 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.574 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.574 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.574 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4457 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.574 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581878+0000", 2024-09-13T09:13:16.574 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93253167800000003, 2024-09-13T09:13:16.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056452293000000001, 2024-09-13T09:13:16.574 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.574 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.574 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.574 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.574 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.574 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4457 2024-09-13T09:13:16.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.575 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581878+0000", 2024-09-13T09:13:16.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581878+0000", 2024-09-13T09:13:16.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581877+0000", 2024-09-13T09:13:16.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581879+0000", 2024-09-13T09:13:16.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0899999999999999e-06 2024-09-13T09:13:16.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581887+0000", 2024-09-13T09:13:16.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2400000000000001e-06 2024-09-13T09:13:16.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581891+0000", 2024-09-13T09:13:16.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8190000000000002e-06 2024-09-13T09:13:16.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638167+0000", 2024-09-13T09:13:16.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056276029999999998 2024-09-13T09:13:16.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638246+0000", 2024-09-13T09:13:16.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9373000000000001e-05 2024-09-13T09:13:16.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638330+0000", 2024-09-13T09:13:16.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4103000000000004e-05 2024-09-13T09:13:16.578 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.578 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.578 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.578 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4458 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.578 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581894+0000", 2024-09-13T09:13:16.578 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93251494199999996, 2024-09-13T09:13:16.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056678246000000002, 2024-09-13T09:13:16.579 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.579 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.579 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.579 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.579 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.579 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4458 2024-09-13T09:13:16.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.579 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581894+0000", 2024-09-13T09:13:16.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581894+0000", 2024-09-13T09:13:16.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581894+0000", 2024-09-13T09:13:16.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581896+0000", 2024-09-13T09:13:16.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.678e-06 2024-09-13T09:13:16.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581902+0000", 2024-09-13T09:13:16.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:13:16.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581906+0000", 2024-09-13T09:13:16.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4800000000000003e-06 2024-09-13T09:13:16.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638383+0000", 2024-09-13T09:13:16.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05647655 2024-09-13T09:13:16.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638431+0000", 2024-09-13T09:13:16.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7953e-05 2024-09-13T09:13:16.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638573+0000", 2024-09-13T09:13:16.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000141784 2024-09-13T09:13:16.582 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.583 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.583 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.583 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4459 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.583 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581920+0000", 2024-09-13T09:13:16.583 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93248989500000001, 2024-09-13T09:13:16.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057037290999999997, 2024-09-13T09:13:16.583 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.583 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.583 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.583 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.583 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.583 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4459 2024-09-13T09:13:16.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.584 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581920+0000", 2024-09-13T09:13:16.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581920+0000", 2024-09-13T09:13:16.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581919+0000", 2024-09-13T09:13:16.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581922+0000", 2024-09-13T09:13:16.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6809999999999998e-06 2024-09-13T09:13:16.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581929+0000", 2024-09-13T09:13:16.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2010000000000002e-06 2024-09-13T09:13:16.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581934+0000", 2024-09-13T09:13:16.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1599999999999997e-06 2024-09-13T09:13:16.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638615+0000", 2024-09-13T09:13:16.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056681242999999999 2024-09-13T09:13:16.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638738+0000", 2024-09-13T09:13:16.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012294000000000001 2024-09-13T09:13:16.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638957+0000", 2024-09-13T09:13:16.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000218476 2024-09-13T09:13:16.587 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.587 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.587 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.587 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4460 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.587 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581939+0000", 2024-09-13T09:13:16.587 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93247053499999999, 2024-09-13T09:13:16.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057296491999999997, 2024-09-13T09:13:16.588 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.588 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.588 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.588 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.588 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.588 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4460 2024-09-13T09:13:16.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.588 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581939+0000", 2024-09-13T09:13:16.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581939+0000", 2024-09-13T09:13:16.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581938+0000", 2024-09-13T09:13:16.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581946+0000", 2024-09-13T09:13:16.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2049999999999996e-06 2024-09-13T09:13:16.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581952+0000", 2024-09-13T09:13:16.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7240000000000003e-06 2024-09-13T09:13:16.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581958+0000", 2024-09-13T09:13:16.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1889999999999998e-06 2024-09-13T09:13:16.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638987+0000", 2024-09-13T09:13:16.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057029006 2024-09-13T09:13:16.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639030+0000", 2024-09-13T09:13:16.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3574e-05 2024-09-13T09:13:16.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639235+0000", 2024-09-13T09:13:16.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00020521 2024-09-13T09:13:16.591 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.591 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.592 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.592 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4461 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.592 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581979+0000", 2024-09-13T09:13:16.592 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93243027499999998, 2024-09-13T09:13:16.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057454631999999999, 2024-09-13T09:13:16.592 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.592 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.592 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.592 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.592 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.592 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4461 2024-09-13T09:13:16.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.593 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581979+0000", 2024-09-13T09:13:16.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581979+0000", 2024-09-13T09:13:16.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581979+0000", 2024-09-13T09:13:16.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581982+0000", 2024-09-13T09:13:16.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1379999999999999e-06 2024-09-13T09:13:16.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581989+0000", 2024-09-13T09:13:16.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0299999999999996e-06 2024-09-13T09:13:16.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581994+0000", 2024-09-13T09:13:16.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1660000000000002e-06 2024-09-13T09:13:16.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639261+0000", 2024-09-13T09:13:16.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057267063999999999 2024-09-13T09:13:16.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639303+0000", 2024-09-13T09:13:16.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1655e-05 2024-09-13T09:13:16.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639434+0000", 2024-09-13T09:13:16.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000131128 2024-09-13T09:13:16.596 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.596 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.596 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.596 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4462 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.596 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581999+0000", 2024-09-13T09:13:16.596 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93241075500000004, 2024-09-13T09:13:16.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057741334999999998, 2024-09-13T09:13:16.596 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.596 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.596 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.597 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.597 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.597 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4462 2024-09-13T09:13:16.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.597 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581999+0000", 2024-09-13T09:13:16.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581999+0000", 2024-09-13T09:13:16.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581998+0000", 2024-09-13T09:13:16.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582001+0000", 2024-09-13T09:13:16.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:13:16.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582007+0000", 2024-09-13T09:13:16.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0889999999999996e-06 2024-09-13T09:13:16.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582011+0000", 2024-09-13T09:13:16.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:13:16.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639465+0000", 2024-09-13T09:13:16.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057453652000000001 2024-09-13T09:13:16.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639551+0000", 2024-09-13T09:13:16.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.632e-05 2024-09-13T09:13:16.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639740+0000", 2024-09-13T09:13:16.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018866299999999999 2024-09-13T09:13:16.600 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.600 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.600 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.601 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4463 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.601 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.582017+0000", 2024-09-13T09:13:16.601 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93239265400000004, 2024-09-13T09:13:16.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057956270999999997, 2024-09-13T09:13:16.601 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.601 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.601 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.601 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.601 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.601 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4463 2024-09-13T09:13:16.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.601 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582017+0000", 2024-09-13T09:13:16.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.602 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582017+0000", 2024-09-13T09:13:16.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.602 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582016+0000", 2024-09-13T09:13:16.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582019+0000", 2024-09-13T09:13:16.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0999999999999998e-06 2024-09-13T09:13:16.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582024+0000", 2024-09-13T09:13:16.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9510000000000003e-06 2024-09-13T09:13:16.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582029+0000", 2024-09-13T09:13:16.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5449999999999997e-06 2024-09-13T09:13:16.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639769+0000", 2024-09-13T09:13:16.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057739921 2024-09-13T09:13:16.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639822+0000", 2024-09-13T09:13:16.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2710000000000002e-05 2024-09-13T09:13:16.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639973+0000", 2024-09-13T09:13:16.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000151373 2024-09-13T09:13:16.605 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.605 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.605 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.605 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4464 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:16.605 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.582033+0000", 2024-09-13T09:13:16.605 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.93237650599999999, 2024-09-13T09:13:16.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.058326451000000001, 2024-09-13T09:13:16.605 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:16.605 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:16.605 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:16.605 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:16.605 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:16.605 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4464 2024-09-13T09:13:16.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.606 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:16.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:16.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582033+0000", 2024-09-13T09:13:16.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:16.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582033+0000", 2024-09-13T09:13:16.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:16.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:16.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582033+0000", 2024-09-13T09:13:16.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:16.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:16.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582035+0000", 2024-09-13T09:13:16.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:13:16.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:16.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582041+0000", 2024-09-13T09:13:16.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8490000000000002e-06 2024-09-13T09:13:16.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:16.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582045+0000", 2024-09-13T09:13:16.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4700000000000004e-06 2024-09-13T09:13:16.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:16.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.640014+0000", 2024-09-13T09:13:16.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057969100000000003 2024-09-13T09:13:16.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:16.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.640106+0000", 2024-09-13T09:13:16.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2074e-05 2024-09-13T09:13:16.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:16.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:16.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:16.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.640359+0000", 2024-09-13T09:13:16.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00025301799999999998 2024-09-13T09:13:16.609 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.609 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.609 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.609 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:16.609 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:16.609 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:16.610 INFO:teuthology.orchestra.run.smithi157.stdout:3655 trunc from 0x321a600 to 0x10a9a00 2024-09-13T09:13:16.610 INFO:teuthology.orchestra.run.smithi157.stdout:3656 write 0x6b87e00 thru 0x6b940d2 (0xc2d3 bytes) 2024-09-13T09:13:16.611 INFO:teuthology.orchestra.run.smithi157.stdout:3658 trunc from 0x6b94200 to 0x434f600 2024-09-13T09:13:16.611 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:16.635 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:16.636 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:16.636 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:16.634+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:16.652 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:16.652 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:16.667 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:13:16.668 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:13:16.684 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:16.684 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:16.703 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:16.703 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:16.713 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:16.712+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:16.714 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:16.714 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:16.723 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:16.798 INFO:teuthology.orchestra.run.smithi157.stdout:3661 write 0xe70800 thru 0xe7c712 (0xbf13 bytes) 2024-09-13T09:13:16.800 INFO:teuthology.orchestra.run.smithi157.stdout:3662 write 0x1e92c00 thru 0x1e974ef (0x48f0 bytes) 2024-09-13T09:13:16.800 INFO:teuthology.orchestra.run.smithi157.stdout:3663 read 0x699c00 thru 0x6a3198 (0x9599 bytes) 2024-09-13T09:13:16.801 INFO:teuthology.orchestra.run.smithi157.stdout:3664 read 0x2748400 thru 0x274b05c (0x2c5d bytes) 2024-09-13T09:13:16.814 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:16.813+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:16.831 INFO:teuthology.orchestra.run.smithi157.stdout:3665 write 0x9ead600 thru 0x9eb6495 (0x8e96 bytes) 2024-09-13T09:13:16.831 INFO:teuthology.orchestra.run.smithi157.stdout:3667 write 0x2a85000 thru 0x2a92abc (0xdabd bytes) 2024-09-13T09:13:16.831 INFO:teuthology.orchestra.run.smithi157.stdout:3670 write 0x4b5d200 thru 0x4b61140 (0x3f41 bytes) 2024-09-13T09:13:16.832 INFO:teuthology.orchestra.run.smithi157.stdout:3672 read 0xdf800 thru 0xe66de (0x6edf bytes) 2024-09-13T09:13:16.832 INFO:teuthology.orchestra.run.smithi157.stdout:3675 read 0x58dc800 thru 0x58e4b84 (0x8385 bytes) 2024-09-13T09:13:16.832 INFO:teuthology.orchestra.run.smithi157.stdout:3677 read 0x978e000 thru 0x97905fe (0x25ff bytes) 2024-09-13T09:13:16.841 INFO:teuthology.orchestra.run.smithi157.stdout:3678 trunc from 0x9eb6600 to 0x2aede00 2024-09-13T09:13:16.884 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:16.884 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:16.894 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:16.902 INFO:teuthology.orchestra.run.smithi157.stdout:3681 write 0x5141200 thru 0x5142912 (0x1713 bytes) 2024-09-13T09:13:16.902 INFO:teuthology.orchestra.run.smithi157.stdout:3682 read 0x36b7600 thru 0x36c45f7 (0xcff8 bytes) 2024-09-13T09:13:16.921 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:16.921 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:16.931 INFO:teuthology.orchestra.run.smithi157.stdout:3683 write 0xbd7da00 thru 0xbd89ff1 (0xc5f2 bytes) 2024-09-13T09:13:16.939 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:16.938+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:16.943 INFO:teuthology.orchestra.run.smithi157.stdout:3684 trunc from 0xbd8a000 to 0x1712600 2024-09-13T09:13:16.975 INFO:teuthology.orchestra.run.smithi157.stdout:3685 punch from 0xbda600 to 0xbdea00, (0x4400 bytes) 2024-09-13T09:13:16.989 INFO:teuthology.orchestra.run.smithi157.stdout:3687 trunc from 0x1712600 to 0xc9bf000 2024-09-13T09:13:16.998 INFO:teuthology.orchestra.run.smithi157.stdout:3688 read 0x1d05600 thru 0x1d105ed (0xafee bytes) 2024-09-13T09:13:17.011 INFO:teuthology.orchestra.run.smithi157.stdout:3689 write 0xe3f1200 thru 0xe3ff011 (0xde12 bytes) 2024-09-13T09:13:17.012 INFO:teuthology.orchestra.run.smithi157.stdout:3690 read 0x47c2c00 thru 0x47c8d95 (0x6196 bytes) 2024-09-13T09:13:17.012 INFO:teuthology.orchestra.run.smithi157.stdout:3691 read 0xe21200 thru 0xe2ce3d (0xbc3e bytes) 2024-09-13T09:13:17.013 INFO:teuthology.orchestra.run.smithi157.stdout:3692 write 0x99ff600 thru 0x9a00319 (0xd1a bytes) 2024-09-13T09:13:17.013 INFO:teuthology.orchestra.run.smithi157.stdout:3693 write 0xc201600 thru 0xc2109da (0xf3db bytes) 2024-09-13T09:13:17.013 INFO:teuthology.orchestra.run.smithi157.stdout:3694 punch from 0x8dff800 to 0x8e08600, (0x8e00 bytes) 2024-09-13T09:13:17.014 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:17.014 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:17.014 INFO:teuthology.orchestra.run.smithi157.stdout:3696 punch from 0xba54800 to 0xba62c00, (0xe400 bytes) 2024-09-13T09:13:17.014 INFO:teuthology.orchestra.run.smithi157.stdout:3697 read 0x6216c00 thru 0x621fbd0 (0x8fd1 bytes) 2024-09-13T09:13:17.016 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:17.015+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:17.026 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:17.028 INFO:teuthology.orchestra.run.smithi157.stdout:3698 trunc from 0xe3ff200 to 0x28cc000 2024-09-13T09:13:17.067 INFO:teuthology.orchestra.run.smithi157.stdout:3706 punch from 0xc6b200 to 0xc76c00, (0xba00 bytes) 2024-09-13T09:13:17.083 INFO:teuthology.orchestra.run.smithi157.stdout:3707 trunc from 0x28cc000 to 0xeb4be00 2024-09-13T09:13:17.092 INFO:teuthology.orchestra.run.smithi157.stdout:3708 write 0xe3d1a00 thru 0xe3da309 (0x890a bytes) 2024-09-13T09:13:17.093 INFO:teuthology.orchestra.run.smithi157.stdout:3709 write 0xb8e8e00 thru 0xb8ef039 (0x623a bytes) 2024-09-13T09:13:17.099 INFO:teuthology.orchestra.run.smithi157.stdout:3710 trunc from 0xeb4be00 to 0x9a59800 2024-09-13T09:13:17.117 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:17.116+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:17.137 INFO:teuthology.orchestra.run.smithi157.stdout:3712 write 0xb40da00 thru 0xb41949a (0xba9b bytes) 2024-09-13T09:13:17.138 INFO:teuthology.orchestra.run.smithi157.stdout:3715 read 0x6aada00 thru 0x6abacce (0xd2cf bytes) 2024-09-13T09:13:17.138 INFO:teuthology.orchestra.run.smithi157.stdout:3716 read 0xa0d400 thru 0xa149c1 (0x75c2 bytes) 2024-09-13T09:13:17.139 INFO:teuthology.orchestra.run.smithi157.stdout:3717 write 0xabbca00 thru 0xabca672 (0xdc73 bytes) 2024-09-13T09:13:17.139 INFO:teuthology.orchestra.run.smithi157.stdout:3719 read 0x4193a00 thru 0x419c995 (0x8f96 bytes) 2024-09-13T09:13:17.139 INFO:teuthology.orchestra.run.smithi157.stdout:3720 read 0x9986c00 thru 0x99902fe (0x96ff bytes) 2024-09-13T09:13:17.139 INFO:teuthology.orchestra.run.smithi157.stdout:3721 write 0x51bce00 thru 0x51c1b58 (0x4d59 bytes) 2024-09-13T09:13:17.139 INFO:teuthology.orchestra.run.smithi157.stdout:3723 trunc from 0xb419600 to 0xb424600 2024-09-13T09:13:17.165 INFO:teuthology.orchestra.run.smithi157.stdout:3724 read 0x5075200 thru 0x5082cfc (0xdafd bytes) 2024-09-13T09:13:17.182 INFO:teuthology.orchestra.run.smithi157.stdout:3725 write 0xe583e00 thru 0xe590c2a (0xce2b bytes) 2024-09-13T09:13:17.182 INFO:teuthology.orchestra.run.smithi157.stdout:3727 write 0x836c200 thru 0x8374de4 (0x8be5 bytes) 2024-09-13T09:13:17.182 INFO:teuthology.orchestra.run.smithi157.stdout:3728 read 0x418be00 thru 0x418ca32 (0xc33 bytes) 2024-09-13T09:13:17.182 INFO:teuthology.orchestra.run.smithi157.stdout:3730 read 0x8533800 thru 0x853d6ff (0x9f00 bytes) 2024-09-13T09:13:17.183 INFO:teuthology.orchestra.run.smithi157.stdout:3735 write 0x6079400 thru 0x6079754 (0x355 bytes) 2024-09-13T09:13:17.183 INFO:teuthology.orchestra.run.smithi157.stdout:3736 read 0x3c52800 thru 0x3c53279 (0xa7a bytes) 2024-09-13T09:13:17.183 INFO:teuthology.orchestra.run.smithi157.stdout:3738 write 0x89d5400 thru 0x89e4829 (0xf42a bytes) 2024-09-13T09:13:17.183 INFO:teuthology.orchestra.run.smithi157.stdout:3739 read 0x7006400 thru 0x7008520 (0x2121 bytes) 2024-09-13T09:13:17.183 INFO:teuthology.orchestra.run.smithi157.stdout:3740 punch from 0x7a71a00 to 0x7a73400, (0x1a00 bytes) 2024-09-13T09:13:17.195 INFO:teuthology.orchestra.run.smithi157.stdout:3744 trunc from 0xe590e00 to 0x3927e00 2024-09-13T09:13:17.200 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:17.200 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:17.211 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:17.238 INFO:teuthology.orchestra.run.smithi157.stdout:3745 read 0x2f71400 thru 0x2f80be1 (0xf7e2 bytes) 2024-09-13T09:13:17.255 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:17.254+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:17.257 INFO:teuthology.orchestra.run.smithi157.stdout:3747 write 0x64bb000 thru 0x64bb71d (0x71e bytes) 2024-09-13T09:13:17.257 INFO:teuthology.orchestra.run.smithi157.stdout:3748 read 0x58f8400 thru 0x59036bf (0xb2c0 bytes) 2024-09-13T09:13:17.257 INFO:teuthology.orchestra.run.smithi157.stdout:3750 write 0x3356e00 thru 0x335c88d (0x5a8e bytes) 2024-09-13T09:13:17.281 INFO:teuthology.orchestra.run.smithi157.stdout:3751 write 0xc5aba00 thru 0xc5b7d97 (0xc398 bytes) 2024-09-13T09:13:17.281 INFO:teuthology.orchestra.run.smithi157.stdout:3752 write 0x1aa2c00 thru 0x1aa81ae (0x55af bytes) 2024-09-13T09:13:17.285 INFO:teuthology.orchestra.run.smithi157.stdout:3753 trunc from 0xc5b7e00 to 0xe50ea00 2024-09-13T09:13:17.319 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:17.318+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:17.319 INFO:teuthology.orchestra.run.smithi157.stdout:3754 trunc from 0xe50ea00 to 0x235e200 2024-09-13T09:13:17.335 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:17.335 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:17.344 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:17.359 INFO:teuthology.orchestra.run.smithi157.stdout:3756 write 0xb67e800 thru 0xb68b4bb (0xccbc bytes) 2024-09-13T09:13:17.359 INFO:teuthology.orchestra.run.smithi157.stdout:3757 read 0x7fa400 thru 0x80a2a4 (0xfea5 bytes) 2024-09-13T09:13:17.364 INFO:teuthology.orchestra.run.smithi157.stdout:3759 write 0x8394600 thru 0x83954d7 (0xed8 bytes) 2024-09-13T09:13:17.364 INFO:teuthology.orchestra.run.smithi157.stdout:3761 read 0x46e9800 thru 0x46ea785 (0xf86 bytes) 2024-09-13T09:13:17.365 INFO:teuthology.orchestra.run.smithi157.stdout:3763 write 0x581dc00 thru 0x5821996 (0x3d97 bytes) 2024-09-13T09:13:17.365 INFO:teuthology.orchestra.run.smithi157.stdout:3764 read 0x1f3a800 thru 0x1f3b19f (0x9a0 bytes) 2024-09-13T09:13:17.365 INFO:teuthology.orchestra.run.smithi157.stdout:3766 read 0x7c7c400 thru 0x7c7d603 (0x1204 bytes) 2024-09-13T09:13:17.365 INFO:teuthology.orchestra.run.smithi157.stdout:3768 read 0x8731800 thru 0x873af05 (0x9706 bytes) 2024-09-13T09:13:17.365 INFO:teuthology.orchestra.run.smithi157.stdout:3770 write 0x3ce1000 thru 0x3ce6079 (0x507a bytes) 2024-09-13T09:13:17.366 INFO:teuthology.orchestra.run.smithi157.stdout:3771 trunc from 0xb68b600 to 0x9508800 2024-09-13T09:13:17.398 INFO:teuthology.orchestra.run.smithi157.stdout:3772 write 0x7fc7400 thru 0x7fc8179 (0xd7a bytes) 2024-09-13T09:13:17.398 INFO:teuthology.orchestra.run.smithi157.stdout:3773 read 0x7b32e00 thru 0x7b3b9c0 (0x8bc1 bytes) 2024-09-13T09:13:17.399 INFO:teuthology.orchestra.run.smithi157.stdout:3774 punch from 0x577b400 to 0x577c800, (0x1400 bytes) 2024-09-13T09:13:17.402 INFO:teuthology.orchestra.run.smithi157.stdout:3777 trunc from 0x9508800 to 0x764c000 2024-09-13T09:13:17.425 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-09-13T09:13:17.430 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:17.428+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:17.436 INFO:teuthology.orchestra.run.smithi157.stdout:3781 write 0xa977a00 thru 0xa97f8bb (0x7ebc bytes) 2024-09-13T09:13:17.437 INFO:teuthology.orchestra.run.smithi157.stdout:3782 read 0x30c7600 thru 0x30d0da2 (0x97a3 bytes) 2024-09-13T09:13:17.437 INFO:teuthology.orchestra.run.smithi157.stdout:3783 write 0x19cee00 thru 0x19d6ec7 (0x80c8 bytes) 2024-09-13T09:13:17.437 INFO:teuthology.orchestra.run.smithi157.stdout:3785 punch from 0x9f16400 to 0x9f19400, (0x3000 bytes) 2024-09-13T09:13:17.448 INFO:teuthology.orchestra.run.smithi157.stdout:3786 trunc from 0xa97fa00 to 0x1b86600 2024-09-13T09:13:17.495 INFO:teuthology.orchestra.run.smithi157.stdout:3787 write 0xdfdca00 thru 0xdfe04d4 (0x3ad5 bytes) 2024-09-13T09:13:17.495 INFO:teuthology.orchestra.run.smithi157.stdout:3789 punch from 0x857fc00 to 0x858f600, (0xfa00 bytes) 2024-09-13T09:13:17.495 INFO:teuthology.orchestra.run.smithi157.stdout:3793 write 0x32b5e00 thru 0x32c5be0 (0xfde1 bytes) 2024-09-13T09:13:17.495 INFO:teuthology.orchestra.run.smithi157.stdout:3794 write 0x19c1800 thru 0x19c250f (0xd10 bytes) 2024-09-13T09:13:17.496 INFO:teuthology.orchestra.run.smithi157.stdout:3798 read 0x1c99c00 thru 0x1ca4db1 (0xb1b2 bytes) 2024-09-13T09:13:17.512 INFO:teuthology.orchestra.run.smithi157.stdout:3799 trunc from 0xdfe0600 to 0x2d20800 2024-09-13T09:13:17.517 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:17.517 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:17.521 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:17.520+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:17.528 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:17.544 INFO:teuthology.orchestra.run.smithi157.stdout:3800 write 0x2120200 thru 0x2120b01 (0x902 bytes) 2024-09-13T09:13:17.545 INFO:teuthology.orchestra.run.smithi157.stdout:3802 punch from 0x2ab0e00 to 0x2ab5200, (0x4400 bytes) 2024-09-13T09:13:17.545 INFO:teuthology.orchestra.run.smithi157.stdout:3807 write 0x10b4800 thru 0x10bda86 (0x9287 bytes) 2024-09-13T09:13:17.558 INFO:teuthology.orchestra.run.smithi157.stdout:3811 write 0x31bf200 thru 0x31c8877 (0x9678 bytes) 2024-09-13T09:13:17.559 INFO:teuthology.orchestra.run.smithi157.stdout:3813 punch from 0x214e400 to 0x215c600, (0xe200 bytes) 2024-09-13T09:13:17.572 INFO:teuthology.orchestra.run.smithi157.stdout:3815 trunc from 0x31c8a00 to 0xd404c00 2024-09-13T09:13:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:3818 punch from 0xb00ea00 to 0xb019000, (0xa600 bytes) 2024-09-13T09:13:17.584 INFO:teuthology.orchestra.run.smithi157.stdout:3819 write 0x24ac200 thru 0x24b7a03 (0xb804 bytes) 2024-09-13T09:13:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:3822 read 0x8993e00 thru 0x899fe21 (0xc022 bytes) 2024-09-13T09:13:17.586 INFO:teuthology.orchestra.run.smithi157.stdout:3823 punch from 0x811ba00 to 0x812aa00, (0xf000 bytes) 2024-09-13T09:13:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:3828 write 0xac28000 thru 0xac3613a (0xe13b bytes) 2024-09-13T09:13:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:3830 punch from 0xbcc3a00 to 0xbccb400, (0x7a00 bytes) 2024-09-13T09:13:17.587 INFO:teuthology.orchestra.run.smithi157.stdout:3831 read 0x3c70000 thru 0x3c7bad6 (0xbad7 bytes) 2024-09-13T09:13:17.595 INFO:teuthology.orchestra.run.smithi157.stdout:3833 trunc from 0xd404c00 to 0x5e80a00 2024-09-13T09:13:17.622 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:17.621+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:17.645 INFO:teuthology.orchestra.run.smithi157.stdout:3834 trunc from 0x5e80a00 to 0xd81aa00 2024-09-13T09:13:17.660 INFO:teuthology.orchestra.run.smithi157.stdout:3835 punch from 0x1110600 to 0x1111a00, (0x1400 bytes) 2024-09-13T09:13:17.677 INFO:teuthology.orchestra.run.smithi157.stdout:3837 write 0xe3c3600 thru 0xe3cb984 (0x8385 bytes) 2024-09-13T09:13:17.688 INFO:teuthology.orchestra.run.smithi157.stdout:3839 trunc from 0xe3cba00 to 0x5399600 2024-09-13T09:13:17.701 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:17.701 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:17.712 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:17.713 INFO:teuthology.orchestra.run.smithi157.stdout:3842 read 0xa51000 thru 0xa5e8f1 (0xd8f2 bytes) 2024-09-13T09:13:17.720 INFO:teuthology.orchestra.run.smithi157.stdout:3844 read 0x3e75600 thru 0x3e7e26e (0x8c6f bytes) 2024-09-13T09:13:17.720 INFO:teuthology.orchestra.run.smithi157.stdout:3845 read 0x44f4800 thru 0x44fe85d (0xa05e bytes) 2024-09-13T09:13:17.720 INFO:teuthology.orchestra.run.smithi157.stdout:3848 read 0xa8ee00 thru 0xa94888 (0x5a89 bytes) 2024-09-13T09:13:17.721 INFO:teuthology.orchestra.run.smithi157.stdout:3850 read 0x15abe00 thru 0x15ba78c (0xe98d bytes) 2024-09-13T09:13:17.722 INFO:teuthology.orchestra.run.smithi157.stdout:3853 write 0x2cc7a00 thru 0x2cd0723 (0x8d24 bytes) 2024-09-13T09:13:17.722 INFO:teuthology.orchestra.run.smithi157.stdout:3855 read 0x4549200 thru 0x4550943 (0x7744 bytes) 2024-09-13T09:13:17.723 INFO:teuthology.orchestra.run.smithi157.stdout:3856 punch from 0x1c0aa00 to 0x1c0ea00, (0x4000 bytes) 2024-09-13T09:13:17.723 INFO:teuthology.orchestra.run.smithi157.stdout:3858 read 0x8d9800 thru 0x8e7e7e (0xe67f bytes) 2024-09-13T09:13:17.724 INFO:teuthology.orchestra.run.smithi157.stdout:3859 write 0x45bd200 thru 0x45bef1e (0x1d1f bytes) 2024-09-13T09:13:17.731 INFO:teuthology.orchestra.run.smithi157.stdout:3864 trunc from 0x5399600 to 0xb2f1800 2024-09-13T09:13:17.746 INFO:teuthology.orchestra.run.smithi157.stdout:3865 write 0x54b600 thru 0x54bcdc (0x6dd bytes) 2024-09-13T09:13:17.756 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:17.755+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:17.757 INFO:teuthology.orchestra.run.smithi157.stdout:3866 trunc from 0xb2f1800 to 0x788fa00 2024-09-13T09:13:17.789 INFO:teuthology.orchestra.run.smithi157.stdout:3868 write 0xaede000 thru 0xaeea4d2 (0xc4d3 bytes) 2024-09-13T09:13:17.789 INFO:teuthology.orchestra.run.smithi157.stdout:3871 write 0x891a600 thru 0x891c672 (0x2073 bytes) 2024-09-13T09:13:17.789 INFO:teuthology.orchestra.run.smithi157.stdout:3872 read 0x860b000 thru 0x860d49e (0x249f bytes) 2024-09-13T09:13:17.791 INFO:teuthology.orchestra.run.smithi157.stdout:3873 trunc from 0xaeea600 to 0xce9d200 2024-09-13T09:13:17.820 INFO:teuthology.orchestra.run.smithi157.stdout:3874 trunc from 0xce9d200 to 0xe3c1a00 2024-09-13T09:13:17.823 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:17.822+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:17.836 INFO:teuthology.orchestra.run.smithi157.stdout:3877 write 0x56c8400 thru 0x56c89f0 (0x5f1 bytes) 2024-09-13T09:13:17.838 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:17.838 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:17.847 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:17.849 INFO:teuthology.orchestra.run.smithi157.stdout:3880 trunc from 0xe3c1a00 to 0x2a55e00 2024-09-13T09:13:17.884 INFO:teuthology.orchestra.run.smithi157.stdout:3882 trunc from 0x2a55e00 to 0xb7c8e00 2024-09-13T09:13:17.892 INFO:teuthology.orchestra.run.smithi157.stdout:3888 write 0x97e5200 thru 0x97e67e0 (0x15e1 bytes) 2024-09-13T09:13:17.893 INFO:teuthology.orchestra.run.smithi157.stdout:3891 read 0x707d200 thru 0x707f5ed (0x23ee bytes) 2024-09-13T09:13:17.896 INFO:teuthology.orchestra.run.smithi157.stdout:3896 trunc from 0xb7c8e00 to 0x94c4400 2024-09-13T09:13:17.911 INFO:teuthology.orchestra.run.smithi157.stdout:3898 punch from 0x6d29c00 to 0x6d34000, (0xa400 bytes) 2024-09-13T09:13:17.912 INFO:teuthology.orchestra.run.smithi157.stdout:3899 write 0x92bbc00 thru 0x92c6fa5 (0xb3a6 bytes) 2024-09-13T09:13:17.912 INFO:teuthology.orchestra.run.smithi157.stdout:3900 read 0x3eb4800 thru 0x3ebffd9 (0xb7da bytes) 2024-09-13T09:13:17.913 INFO:teuthology.orchestra.run.smithi157.stdout:3902 write 0x6d79400 thru 0x6d88594 (0xf195 bytes) 2024-09-13T09:13:17.913 INFO:teuthology.orchestra.run.smithi157.stdout:3904 punch from 0x6bc2c00 to 0x6bcfc00, (0xd000 bytes) 2024-09-13T09:13:17.913 INFO:teuthology.orchestra.run.smithi157.stdout:3905 write 0x72bca00 thru 0x72c2177 (0x5778 bytes) 2024-09-13T09:13:17.913 INFO:teuthology.orchestra.run.smithi157.stdout:3906 trunc from 0x94c4400 to 0x9956c00 2024-09-13T09:13:17.924 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:17.923+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:17.939 INFO:teuthology.orchestra.run.smithi157.stdout:3907 punch from 0x662c400 to 0x663bc00, (0xf800 bytes) 2024-09-13T09:13:17.948 INFO:teuthology.orchestra.run.smithi157.stdout:3909 trunc from 0x9956c00 to 0x6aa8e00 2024-09-13T09:13:17.980 INFO:teuthology.orchestra.run.smithi157.stdout:3910 write 0xbda7600 thru 0xbdad128 (0x5b29 bytes) 2024-09-13T09:13:17.988 INFO:teuthology.orchestra.run.smithi157.stdout:3911 read 0x8e2b800 thru 0x8e2c5c9 (0xdca bytes) 2024-09-13T09:13:17.988 INFO:teuthology.orchestra.run.smithi157.stdout:3913 write 0x4527600 thru 0x452c0c4 (0x4ac5 bytes) 2024-09-13T09:13:17.988 INFO:teuthology.orchestra.run.smithi157.stdout:3914 write 0x2b2dc00 thru 0x2b365df (0x89e0 bytes) 2024-09-13T09:13:17.993 INFO:teuthology.orchestra.run.smithi157.stdout:3916 trunc from 0xbdad200 to 0x725b000 2024-09-13T09:13:18.019 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:18.019 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:18.025 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:18.024+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:18.029 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:18.032 INFO:teuthology.orchestra.run.smithi157.stdout:3919 write 0x91b6400 thru 0x91be1f1 (0x7df2 bytes) 2024-09-13T09:13:18.033 INFO:teuthology.orchestra.run.smithi157.stdout:3922 read 0x239a800 thru 0x23a9d9c (0xf59d bytes) 2024-09-13T09:13:18.037 INFO:teuthology.orchestra.run.smithi157.stdout:3928 punch from 0x8dd8e00 to 0x8de3600, (0xa800 bytes) 2024-09-13T09:13:18.041 INFO:teuthology.orchestra.run.smithi157.stdout:3929 trunc from 0x91be200 to 0x6378800 2024-09-13T09:13:18.077 INFO:teuthology.orchestra.run.smithi157.stdout:3931 trunc from 0x6378800 to 0x58ad800 2024-09-13T09:13:18.092 INFO:teuthology.orchestra.run.smithi157.stdout:3932 write 0x2aeca00 thru 0x2aedc74 (0x1275 bytes) 2024-09-13T09:13:18.093 INFO:teuthology.orchestra.run.smithi157.stdout:3933 punch from 0x5012c00 to 0x5012e00, (0x200 bytes) 2024-09-13T09:13:18.093 INFO:teuthology.orchestra.run.smithi157.stdout:3936 read 0x41b5a00 thru 0x41bc3ec (0x69ed bytes) 2024-09-13T09:13:18.094 INFO:teuthology.orchestra.run.smithi157.stdout:3939 read 0x355ae00 thru 0x3565a4e (0xac4f bytes) 2024-09-13T09:13:18.095 INFO:teuthology.orchestra.run.smithi157.stdout:3942 read 0x1e65800 thru 0x1e69b6c (0x436d bytes) 2024-09-13T09:13:18.095 INFO:teuthology.orchestra.run.smithi157.stdout:3944 write 0x3d50400 thru 0x3d52058 (0x1c59 bytes) 2024-09-13T09:13:18.095 INFO:teuthology.orchestra.run.smithi157.stdout:3946 read 0x345ea00 thru 0x3466cca (0x82cb bytes) 2024-09-13T09:13:18.096 INFO:teuthology.orchestra.run.smithi157.stdout:3950 read 0x1178600 thru 0x1186b8f (0xe590 bytes) 2024-09-13T09:13:18.097 INFO:teuthology.orchestra.run.smithi157.stdout:3951 punch from 0x5236000 to 0x5243800, (0xd800 bytes) 2024-09-13T09:13:18.097 INFO:teuthology.orchestra.run.smithi157.stdout:3954 punch from 0x1816200 to 0x181a000, (0x3e00 bytes) 2024-09-13T09:13:18.115 INFO:teuthology.orchestra.run.smithi157.stdout:3955 write 0x797a600 thru 0x79899d9 (0xf3da bytes) 2024-09-13T09:13:18.116 INFO:teuthology.orchestra.run.smithi157.stdout:3956 read 0x5b22a00 thru 0x5b2f582 (0xcb83 bytes) 2024-09-13T09:13:18.121 INFO:teuthology.orchestra.run.smithi157.stdout:3958 read 0x19b3400 thru 0x19b80f8 (0x4cf9 bytes) 2024-09-13T09:13:18.121 INFO:teuthology.orchestra.run.smithi157.stdout:3959 read 0x549d200 thru 0x54ac2dd (0xf0de bytes) 2024-09-13T09:13:18.122 INFO:teuthology.orchestra.run.smithi157.stdout:3960 write 0x4285800 thru 0x428e04f (0x8850 bytes) 2024-09-13T09:13:18.122 INFO:teuthology.orchestra.run.smithi157.stdout:3961 read 0x29a9400 thru 0x29abb9c (0x279d bytes) 2024-09-13T09:13:18.127 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:18.125+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:18.132 INFO:teuthology.orchestra.run.smithi157.stdout:3965 trunc from 0x7989a00 to 0xeafc600 2024-09-13T09:13:18.145 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:18.145 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:18.154 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:18.171 INFO:teuthology.orchestra.run.smithi157.stdout:3967 trunc from 0xeafc600 to 0x815a00 2024-09-13T09:13:18.205 INFO:teuthology.orchestra.run.smithi157.stdout:3968 punch from 0x3ee800 to 0x3f8e00, (0xa600 bytes) 2024-09-13T09:13:18.206 INFO:teuthology.orchestra.run.smithi157.stdout:3970 read 0x44b800 thru 0x451dc3 (0x65c4 bytes) 2024-09-13T09:13:18.227 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:18.226+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:18.229 INFO:teuthology.orchestra.run.smithi157.stdout:3972 write 0xbc5e800 thru 0xbc62dd3 (0x45d4 bytes) 2024-09-13T09:13:18.230 INFO:teuthology.orchestra.run.smithi157.stdout:3973 read 0x99d4c00 thru 0x99df219 (0xa61a bytes) 2024-09-13T09:13:18.230 INFO:teuthology.orchestra.run.smithi157.stdout:3974 trunc from 0xbc62e00 to 0xb488c00 2024-09-13T09:13:18.246 INFO:teuthology.orchestra.run.smithi157.stdout:3975 punch from 0x888ba00 to 0x8895600, (0x9c00 bytes) 2024-09-13T09:13:18.247 INFO:teuthology.orchestra.run.smithi157.stdout:3979 write 0x8e49400 thru 0x8e580a8 (0xeca9 bytes) 2024-09-13T09:13:18.247 INFO:teuthology.orchestra.run.smithi157.stdout:3980 read 0x5a7000 thru 0x5aee4a (0x7e4b bytes) 2024-09-13T09:13:18.258 INFO:teuthology.orchestra.run.smithi157.stdout:3981 trunc from 0xb488c00 to 0x63c4400 2024-09-13T09:13:18.272 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:18.272 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:18.282 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:18.288 INFO:teuthology.orchestra.run.smithi157.stdout:3982 write 0x3429600 thru 0x3435e2f (0xc830 bytes) 2024-09-13T09:13:18.288 INFO:teuthology.orchestra.run.smithi157.stdout:3983 write 0x4645000 thru 0x4653adf (0xeae0 bytes) 2024-09-13T09:13:18.318 INFO:teuthology.orchestra.run.smithi157.stdout:3984 write 0x923f800 thru 0x9244dd0 (0x55d1 bytes) 2024-09-13T09:13:18.318 INFO:teuthology.orchestra.run.smithi157.stdout:3985 read 0x5d3bc00 thru 0x5d4226a (0x666b bytes) 2024-09-13T09:13:18.319 INFO:teuthology.orchestra.run.smithi157.stdout:3987 write 0x3faae00 thru 0x3fad1ca (0x23cb bytes) 2024-09-13T09:13:18.319 INFO:teuthology.orchestra.run.smithi157.stdout:3989 punch from 0x51c3800 to 0x51c4600, (0xe00 bytes) 2024-09-13T09:13:18.325 INFO:teuthology.orchestra.run.smithi157.stdout:3990 trunc from 0x9244e00 to 0x4174400 2024-09-13T09:13:18.328 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:18.327+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:18.393 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:18.393 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:18.401 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:18.429 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:18.427+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:18.530 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:18.529+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:18.545 INFO:teuthology.orchestra.run.smithi157.stdout:3991 read 0x148d800 thru 0x149954c (0xbd4d bytes) 2024-09-13T09:13:18.550 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:18.550 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:18.560 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:18.565 INFO:teuthology.orchestra.run.smithi157.stdout:3992 write 0x9080000 thru 0x908012b (0x12c bytes) 2024-09-13T09:13:18.571 INFO:teuthology.orchestra.run.smithi157.stdout:3994 read 0x3ced200 thru 0x3cf284c (0x564d bytes) 2024-09-13T09:13:18.572 INFO:teuthology.orchestra.run.smithi157.stdout:3998 read 0x2ed600 thru 0x2f81cb (0xabcc bytes) 2024-09-13T09:13:18.573 INFO:teuthology.orchestra.run.smithi157.stdout:3999 read 0x4cf8a00 thru 0x4cff9ac (0x6fad bytes) 2024-09-13T09:13:18.575 INFO:teuthology.orchestra.run.smithi157.stdout:4000 write 0x869d800 thru 0x869f16a (0x196b bytes) 2024-09-13T09:13:18.575 INFO:teuthology.orchestra.run.smithi157.stdout:4001 read 0x2148e00 thru 0x21515f4 (0x87f5 bytes) 2024-09-13T09:13:18.575 INFO:teuthology.orchestra.run.smithi157.stdout:4002 write 0x690c800 thru 0x69141e6 (0x79e7 bytes) 2024-09-13T09:13:18.575 INFO:teuthology.orchestra.run.smithi157.stdout:4003 punch from 0x7fb0800 to 0x7fb3a00, (0x3200 bytes) 2024-09-13T09:13:18.575 INFO:teuthology.orchestra.run.smithi157.stdout:4004 write 0x1094400 thru 0x109c1a9 (0x7daa bytes) 2024-09-13T09:13:18.576 INFO:teuthology.orchestra.run.smithi157.stdout:4007 punch from 0x2248000 to 0x2257c00, (0xfc00 bytes) 2024-09-13T09:13:18.576 INFO:teuthology.orchestra.run.smithi157.stdout:4013 read 0x3ae6800 thru 0x3ae9844 (0x3045 bytes) 2024-09-13T09:13:18.578 INFO:teuthology.orchestra.run.smithi157.stdout:4014 write 0x825ba00 thru 0x826a0ce (0xe6cf bytes) 2024-09-13T09:13:18.578 INFO:teuthology.orchestra.run.smithi157.stdout:4015 write 0x3da0e00 thru 0x3daaae1 (0x9ce2 bytes) 2024-09-13T09:13:18.613 INFO:teuthology.orchestra.run.smithi157.stdout:4016 write 0xaa18800 thru 0xaa1881b (0x1c bytes) 2024-09-13T09:13:18.613 INFO:teuthology.orchestra.run.smithi157.stdout:4019 write 0x2363600 thru 0x236b461 (0x7e62 bytes) 2024-09-13T09:13:18.614 INFO:teuthology.orchestra.run.smithi157.stdout:4021 write 0x4bfe800 thru 0x4c0941e (0xac1f bytes) 2024-09-13T09:13:18.618 INFO:teuthology.orchestra.run.smithi157.stdout:4022 punch from 0x520a800 to 0x5214e00, (0xa600 bytes) 2024-09-13T09:13:18.618 INFO:teuthology.orchestra.run.smithi157.stdout:4023 write 0x4866000 thru 0x48757dd (0xf7de bytes) 2024-09-13T09:13:18.629 INFO:teuthology.orchestra.run.smithi157.stdout:4024 trunc from 0xaa18a00 to 0x6f86200 2024-09-13T09:13:18.631 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:18.629+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:18.648 INFO:teuthology.orchestra.run.smithi157.stdout:4025 write 0x65f4400 thru 0x65fe77a (0xa37b bytes) 2024-09-13T09:13:18.655 INFO:teuthology.orchestra.run.smithi157.stdout:4029 trunc from 0x6f86200 to 0xc2bc800 2024-09-13T09:13:18.672 INFO:teuthology.orchestra.run.smithi157.stdout:4030 punch from 0x1a6ae00 to 0x1a6d000, (0x2200 bytes) 2024-09-13T09:13:18.675 INFO:teuthology.orchestra.run.smithi157.stdout:4031 trunc from 0xc2bc800 to 0xe2dfe00 2024-09-13T09:13:18.684 INFO:teuthology.orchestra.run.smithi157.stdout:4032 write 0x3c1a400 thru 0x3c20477 (0x6078 bytes) 2024-09-13T09:13:18.686 INFO:teuthology.orchestra.run.smithi157.stdout:4034 write 0x6160600 thru 0x61611fa (0xbfb bytes) 2024-09-13T09:13:18.686 INFO:teuthology.orchestra.run.smithi157.stdout:4036 read 0x320fc00 thru 0x321b55a (0xb95b bytes) 2024-09-13T09:13:18.689 INFO:teuthology.orchestra.run.smithi157.stdout:4038 write 0xc2b7200 thru 0xc2bc70b (0x550c bytes) 2024-09-13T09:13:18.689 INFO:teuthology.orchestra.run.smithi157.stdout:4039 punch from 0x9ede600 to 0x9eeb200, (0xcc00 bytes) 2024-09-13T09:13:18.689 INFO:teuthology.orchestra.run.smithi157.stdout:4047 read 0x9445a00 thru 0x9449530 (0x3b31 bytes) 2024-09-13T09:13:18.690 INFO:teuthology.orchestra.run.smithi157.stdout:4055 trunc from 0xe2dfe00 to 0xd33be00 2024-09-13T09:13:18.711 INFO:teuthology.orchestra.run.smithi157.stdout:4056 write 0x4fe7c00 thru 0x4fe90b9 (0x14ba bytes) 2024-09-13T09:13:18.712 INFO:teuthology.orchestra.run.smithi157.stdout:4057 read 0x3b03600 thru 0x3b0b4a8 (0x7ea9 bytes) 2024-09-13T09:13:18.716 INFO:teuthology.orchestra.run.smithi157.stdout:4058 read 0x486600 thru 0x48eeda (0x88db bytes) 2024-09-13T09:13:18.717 INFO:teuthology.orchestra.run.smithi157.stdout:4059 write 0xa353200 thru 0xa358431 (0x5232 bytes) 2024-09-13T09:13:18.717 INFO:teuthology.orchestra.run.smithi157.stdout:4060 read 0x657f600 thru 0x658a557 (0xaf58 bytes) 2024-09-13T09:13:18.718 INFO:teuthology.orchestra.run.smithi157.stdout:4061 read 0x610fe00 thru 0x611d595 (0xd796 bytes) 2024-09-13T09:13:18.718 INFO:teuthology.orchestra.run.smithi157.stdout:4063 write 0x7b97200 thru 0x7ba2cd4 (0xbad5 bytes) 2024-09-13T09:13:18.720 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:18.720 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:18.730 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:18.732 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:18.731+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:18.739 INFO:teuthology.orchestra.run.smithi157.stdout:4064 write 0xda35e00 thru 0xda3f9e8 (0x9be9 bytes) 2024-09-13T09:13:18.740 INFO:teuthology.orchestra.run.smithi157.stdout:4066 read 0xd114400 thru 0xd115ec4 (0x1ac5 bytes) 2024-09-13T09:13:18.745 INFO:teuthology.orchestra.run.smithi157.stdout:4067 trunc from 0xda3fa00 to 0xa68d400 2024-09-13T09:13:18.774 INFO:teuthology.orchestra.run.smithi157.stdout:4068 punch from 0x511fa00 to 0x5126400, (0x6a00 bytes) 2024-09-13T09:13:18.774 INFO:teuthology.orchestra.run.smithi157.stdout:4070 read 0x9197c00 thru 0x91a6872 (0xec73 bytes) 2024-09-13T09:13:18.775 INFO:teuthology.orchestra.run.smithi157.stdout:4071 write 0x5825200 thru 0x5835186 (0xff87 bytes) 2024-09-13T09:13:18.799 INFO:teuthology.orchestra.run.smithi157.stdout:4072 write 0xdd22600 thru 0xdd288de (0x62df bytes) 2024-09-13T09:13:18.799 INFO:teuthology.orchestra.run.smithi157.stdout:4074 read 0x3f1de00 thru 0x3f1f58d (0x178e bytes) 2024-09-13T09:13:18.799 INFO:teuthology.orchestra.run.smithi157.stdout:4075 write 0x448f800 thru 0x4490b7a (0x137b bytes) 2024-09-13T09:13:18.800 INFO:teuthology.orchestra.run.smithi157.stdout:4076 read 0x3ae4800 thru 0x3aebcb1 (0x74b2 bytes) 2024-09-13T09:13:18.801 INFO:teuthology.orchestra.run.smithi157.stdout:4081 read 0xd189000 thru 0xd18f1b8 (0x61b9 bytes) 2024-09-13T09:13:18.801 INFO:teuthology.orchestra.run.smithi157.stdout:4083 read 0x4dafe00 thru 0x4db1a15 (0x1c16 bytes) 2024-09-13T09:13:18.804 INFO:teuthology.orchestra.run.smithi157.stdout:4086 trunc from 0xdd28a00 to 0xbe42400 2024-09-13T09:13:18.830 INFO:teuthology.orchestra.run.smithi157.stdout:4089 punch from 0x9120000 to 0x9127e00, (0x7e00 bytes) 2024-09-13T09:13:18.830 INFO:teuthology.orchestra.run.smithi157.stdout:4090 write 0x3738c00 thru 0x373b918 (0x2d19 bytes) 2024-09-13T09:13:18.831 INFO:teuthology.orchestra.run.smithi157.stdout:4092 punch from 0x4ef2800 to 0x4efbc00, (0x9400 bytes) 2024-09-13T09:13:18.832 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:18.831+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:18.836 INFO:teuthology.orchestra.run.smithi157.stdout:4093 trunc from 0xbe42400 to 0x7a4d600 2024-09-13T09:13:18.851 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:18.851 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:18.855 INFO:teuthology.orchestra.run.smithi157.stdout:4094 read 0x6f70400 thru 0x6f76877 (0x6478 bytes) 2024-09-13T09:13:18.860 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:13:18.861 INFO:teuthology.orchestra.run.smithi157.stdout:4095 read 0x1de6600 thru 0x1df00e5 (0x9ae6 bytes) 2024-09-13T09:13:18.870 INFO:teuthology.orchestra.run.smithi157.stdout:4097 trunc from 0x7a4d600 to 0xe144e00 2024-09-13T09:13:18.878 INFO:teuthology.orchestra.run.smithi157.stdout:4102 read 0x65a2c00 thru 0x65aade2 (0x81e3 bytes) 2024-09-13T09:13:18.885 INFO:teuthology.orchestra.run.smithi157.stdout:4103 punch from 0x526ea00 to 0x5276400, (0x7a00 bytes) 2024-09-13T09:13:18.886 INFO:teuthology.orchestra.run.smithi157.stdout:4104 read 0x8fe0c00 thru 0x8fe9314 (0x8715 bytes) 2024-09-13T09:13:18.886 INFO:teuthology.orchestra.run.smithi157.stdout:4105 write 0xb784000 thru 0xb791b15 (0xdb16 bytes) 2024-09-13T09:13:18.899 INFO:teuthology.orchestra.run.smithi157.stdout:4106 trunc from 0xe144e00 to 0x3397800 2024-09-13T09:13:18.932 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:13:18.933 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:18.932+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:18.948 INFO:teuthology.orchestra.run.smithi157.stdout:4108 trunc from 0x3397800 to 0xaf22200 2024-09-13T09:13:18.956 INFO:teuthology.orchestra.run.smithi157.stdout:4113 punch from 0x9493e00 to 0x949d200, (0x9400 bytes) 2024-09-13T09:13:18.958 INFO:teuthology.orchestra.run.smithi157.stdout:4116 read 0x42e7800 thru 0x42ec2d3 (0x4ad4 bytes) 2024-09-13T09:13:18.960 INFO:teuthology.orchestra.run.smithi157.stdout:4117 trunc from 0xaf22200 to 0xca47200 2024-09-13T09:13:18.979 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:18.979 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:18.989 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:13:18.989 INFO:teuthology.orchestra.run.smithi157.stdout:4118 write 0xcbc0200 thru 0xcbc6c4e (0x6a4f bytes) 2024-09-13T09:13:18.997 INFO:teuthology.orchestra.run.smithi157.stdout:4119 trunc from 0xcbc6e00 to 0x5b7ca00 2024-09-13T09:13:19.027 INFO:teuthology.orchestra.run.smithi157.stdout:4120 trunc from 0x5b7ca00 to 0x9131000 2024-09-13T09:13:19.035 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:19.033+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:19.036 INFO:teuthology.orchestra.run.smithi157.stdout:4122 punch from 0x2107600 to 0x2110600, (0x9000 bytes) 2024-09-13T09:13:19.038 INFO:teuthology.orchestra.run.smithi157.stdout:4124 write 0x5e6a600 thru 0x5e736e2 (0x90e3 bytes) 2024-09-13T09:13:19.038 INFO:teuthology.orchestra.run.smithi157.stdout:4125 read 0x84c7000 thru 0x84d38bc (0xc8bd bytes) 2024-09-13T09:13:19.038 INFO:teuthology.orchestra.run.smithi157.stdout:4128 punch from 0x29a00 to 0x2b400, (0x1a00 bytes) 2024-09-13T09:13:19.038 INFO:teuthology.orchestra.run.smithi157.stdout:4129 read 0x6de2800 thru 0x6def182 (0xc983 bytes) 2024-09-13T09:13:19.038 INFO:teuthology.orchestra.run.smithi157.stdout:4130 read 0x7dbaa00 thru 0x7dc38d9 (0x8eda bytes) 2024-09-13T09:13:19.054 INFO:teuthology.orchestra.run.smithi157.stdout:4131 write 0xcfabe00 thru 0xcfb8162 (0xc363 bytes) 2024-09-13T09:13:19.060 INFO:teuthology.orchestra.run.smithi157.stdout:4132 write 0xb432400 thru 0xb4327a1 (0x3a2 bytes) 2024-09-13T09:13:19.060 INFO:teuthology.orchestra.run.smithi157.stdout:4133 read 0xb894c00 thru 0xb89c339 (0x773a bytes) 2024-09-13T09:13:19.060 INFO:teuthology.orchestra.run.smithi157.stdout:4135 write 0x1d91400 thru 0x1d98e65 (0x7a66 bytes) 2024-09-13T09:13:19.061 INFO:teuthology.orchestra.run.smithi157.stdout:4137 read 0x7f2ca00 thru 0x7f385f4 (0xbbf5 bytes) 2024-09-13T09:13:19.061 INFO:teuthology.orchestra.run.smithi157.stdout:4138 punch from 0x72e5600 to 0x72f1800, (0xc200 bytes) 2024-09-13T09:13:19.065 INFO:teuthology.orchestra.run.smithi157.stdout:4140 trunc from 0xcfb8200 to 0x9972800 2024-09-13T09:13:19.073 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:19.073 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:19.093 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:19.093 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:19.103 INFO:teuthology.orchestra.run.smithi157.stdout:4141 trunc from 0x9972800 to 0x304b600 2024-09-13T09:13:19.120 INFO:teuthology.orchestra.run.smithi157.stdout:4144 read 0x1c2d200 thru 0x1c2d91a (0x71b bytes) 2024-09-13T09:13:19.121 INFO:teuthology.orchestra.run.smithi157.stdout:4145 read 0x2305600 thru 0x2313548 (0xdf49 bytes) 2024-09-13T09:13:19.122 INFO:teuthology.orchestra.run.smithi157.stdout:4146 read 0x2e49400 thru 0x2e4b10f (0x1d10 bytes) 2024-09-13T09:13:19.123 INFO:teuthology.orchestra.run.smithi157.stdout:4148 trunc from 0x304b600 to 0x3d97800 2024-09-13T09:13:19.125 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:13:19.125 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:13:19.126 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:19.127 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:19.127 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:19.127 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:19.127 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:19.135 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:19.133+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:19.136 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:13:19.143 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:19.143 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:19.156 INFO:teuthology.orchestra.run.smithi157.stdout:4150 write 0xcd7dc00 thru 0xcd7e95d (0xd5e bytes) 2024-09-13T09:13:19.165 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:19.165 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:19.167 INFO:teuthology.orchestra.run.smithi157.stdout:4152 trunc from 0xcd7ea00 to 0x408ce00 2024-09-13T09:13:19.182 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:19.183 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:19.194 INFO:teuthology.orchestra.run.smithi157.stdout:4153 write 0x5974800 thru 0x5975b25 (0x1326 bytes) 2024-09-13T09:13:19.195 INFO:teuthology.orchestra.run.smithi157.stdout:4155 read 0x595f000 thru 0x5963f0c (0x4f0d bytes) 2024-09-13T09:13:19.195 INFO:teuthology.orchestra.run.smithi157.stdout:4156 write 0x5845a00 thru 0x584daac (0x80ad bytes) 2024-09-13T09:13:19.195 INFO:teuthology.orchestra.run.smithi157.stdout:4159 trunc from 0x5975c00 to 0x5812e00 2024-09-13T09:13:19.222 INFO:teuthology.orchestra.run.smithi157.stdout:4161 trunc from 0x5812e00 to 0xd229c00 2024-09-13T09:13:19.230 INFO:teuthology.orchestra.run.smithi157.stdout:4162 read 0xa4f7800 thru 0xa4fcb1e (0x531f bytes) 2024-09-13T09:13:19.232 INFO:teuthology.orchestra.run.smithi157.stdout:4164 write 0x337b600 thru 0x338644d (0xae4e bytes) 2024-09-13T09:13:19.236 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:19.234+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:19.246 INFO:teuthology.orchestra.run.smithi157.stdout:4167 trunc from 0xd229c00 to 0x1dde800 2024-09-13T09:13:19.261 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:19.261 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:19.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:19.261 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:19.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.261 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:6867 3.13 3:cd293326:::rbd_header.112326fb43d3:head [watch reconnect cookie 140383087120544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T09:13:19.261 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:13:19.261 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 35.729932214999998, 2024-09-13T09:13:19.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0277181360000001, 2024-09-13T09:13:19.262 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.262 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.262 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.262 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:13:19.262 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:13:19.262 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6867 2024-09-13T09:13:19.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.262 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:13:19.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:13:19.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529067+0000", 2024-09-13T09:13:19.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-09-13T09:13:19.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529073+0000", 2024-09-13T09:13:19.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:13:19.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529087+0000", 2024-09-13T09:13:19.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3825000000000001e-05 2024-09-13T09:13:19.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529097+0000", 2024-09-13T09:13:19.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9559999999999992e-06 2024-09-13T09:13:19.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529149+0000", 2024-09-13T09:13:19.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1668000000000001e-05 2024-09-13T09:13:19.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529161+0000", 2024-09-13T09:13:19.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1831000000000001e-05 2024-09-13T09:13:19.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.554941+0000", 2024-09-13T09:13:19.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0257799729999999 2024-09-13T09:13:19.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.555239+0000", 2024-09-13T09:13:19.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00029829100000000001 2024-09-13T09:13:19.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.556787+0000", 2024-09-13T09:13:19.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001547617 2024-09-13T09:13:19.267 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.267 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.267 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.268 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20898 3.13 3:cd293326:::rbd_header.112326fb43d3:head [watch reconnect cookie 140642395970896 gen 16] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T09:13:19.268 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:13:19.268 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 35.725269396999998, 2024-09-13T09:13:19.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0236151950000001, 2024-09-13T09:13:19.268 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.268 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.268 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.268 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:19.268 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.268 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20898 2024-09-13T09:13:19.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.269 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:13:19.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:13:19.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533729+0000", 2024-09-13T09:13:19.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-09-13T09:13:19.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533736+0000", 2024-09-13T09:13:19.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9670000000000004e-06 2024-09-13T09:13:19.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533749+0000", 2024-09-13T09:13:19.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2449000000000001e-05 2024-09-13T09:13:19.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533757+0000", 2024-09-13T09:13:19.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:13:19.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533813+0000", 2024-09-13T09:13:19.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5683000000000002e-05 2024-09-13T09:13:19.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533823+0000", 2024-09-13T09:13:19.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0168e-05 2024-09-13T09:13:19.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.555940+0000", 2024-09-13T09:13:19.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0221170559999999 2024-09-13T09:13:19.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.273 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.556047+0000", 2024-09-13T09:13:19.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000106881 2024-09-13T09:13:19.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.273 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.557347+0000", 2024-09-13T09:13:19.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001299323 2024-09-13T09:13:19.274 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.274 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.274 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.274 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4447 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.274 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581680+0000", 2024-09-13T09:13:19.274 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.6773210929999998, 2024-09-13T09:13:19.274 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054674652999999997, 2024-09-13T09:13:19.274 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.274 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.274 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.275 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.275 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.275 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4447 2024-09-13T09:13:19.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.275 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581680+0000", 2024-09-13T09:13:19.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581680+0000", 2024-09-13T09:13:19.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581679+0000", 2024-09-13T09:13:19.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581682+0000", 2024-09-13T09:13:19.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4830000000000002e-06 2024-09-13T09:13:19.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581688+0000", 2024-09-13T09:13:19.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0569999999999998e-06 2024-09-13T09:13:19.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581692+0000", 2024-09-13T09:13:19.278 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6820000000000002e-06 2024-09-13T09:13:19.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.278 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.278 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636201+0000", 2024-09-13T09:13:19.278 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054508845 2024-09-13T09:13:19.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.278 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.278 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636293+0000", 2024-09-13T09:13:19.278 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1477000000000005e-05 2024-09-13T09:13:19.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.279 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.279 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636354+0000", 2024-09-13T09:13:19.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1651999999999999e-05 2024-09-13T09:13:19.279 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.279 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.279 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.279 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4448 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 913408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.280 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581696+0000", 2024-09-13T09:13:19.280 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.6773043919999999, 2024-09-13T09:13:19.280 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054880384999999997, 2024-09-13T09:13:19.280 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.280 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.280 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.280 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.280 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.280 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4448 2024-09-13T09:13:19.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.281 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581696+0000", 2024-09-13T09:13:19.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581696+0000", 2024-09-13T09:13:19.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.282 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.282 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581696+0000", 2024-09-13T09:13:19.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.282 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.282 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581702+0000", 2024-09-13T09:13:19.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4250000000000003e-06 2024-09-13T09:13:19.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581716+0000", 2024-09-13T09:13:19.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3196e-05 2024-09-13T09:13:19.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581721+0000", 2024-09-13T09:13:19.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0189999999999997e-06 2024-09-13T09:13:19.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636390+0000", 2024-09-13T09:13:19.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054669061999999997 2024-09-13T09:13:19.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636479+0000", 2024-09-13T09:13:19.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9188000000000001e-05 2024-09-13T09:13:19.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636577+0000", 2024-09-13T09:13:19.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7827999999999999e-05 2024-09-13T09:13:19.285 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.285 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.285 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.285 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4449 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.285 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581725+0000", 2024-09-13T09:13:19.285 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.6772755529999999, 2024-09-13T09:13:19.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055074856999999998, 2024-09-13T09:13:19.286 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.286 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.286 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.286 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.286 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.286 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4449 2024-09-13T09:13:19.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.286 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.286 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581725+0000", 2024-09-13T09:13:19.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.287 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581725+0000", 2024-09-13T09:13:19.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.287 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581725+0000", 2024-09-13T09:13:19.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581727+0000", 2024-09-13T09:13:19.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5969999999999999e-06 2024-09-13T09:13:19.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581733+0000", 2024-09-13T09:13:19.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.088e-06 2024-09-13T09:13:19.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581738+0000", 2024-09-13T09:13:19.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:13:19.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636614+0000", 2024-09-13T09:13:19.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054875637999999997 2024-09-13T09:13:19.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636703+0000", 2024-09-13T09:13:19.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8838000000000006e-05 2024-09-13T09:13:19.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636800+0000", 2024-09-13T09:13:19.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7282999999999997e-05 2024-09-13T09:13:19.290 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.291 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.291 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.291 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4450 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.291 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581743+0000", 2024-09-13T09:13:19.291 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.677258036, 2024-09-13T09:13:19.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055241622999999997, 2024-09-13T09:13:19.291 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.291 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.291 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.292 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4450 2024-09-13T09:13:19.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.292 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.292 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.292 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581743+0000", 2024-09-13T09:13:19.292 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581743+0000", 2024-09-13T09:13:19.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581742+0000", 2024-09-13T09:13:19.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581745+0000", 2024-09-13T09:13:19.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.294e-06 2024-09-13T09:13:19.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581751+0000", 2024-09-13T09:13:19.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.516e-06 2024-09-13T09:13:19.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581758+0000", 2024-09-13T09:13:19.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7780000000000002e-06 2024-09-13T09:13:19.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636860+0000", 2024-09-13T09:13:19.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055102263999999998 2024-09-13T09:13:19.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636927+0000", 2024-09-13T09:13:19.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6427000000000006e-05 2024-09-13T09:13:19.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636984+0000", 2024-09-13T09:13:19.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7739999999999999e-05 2024-09-13T09:13:19.296 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.296 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.296 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.296 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4451 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.296 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581763+0000", 2024-09-13T09:13:19.297 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.6772376929999999, 2024-09-13T09:13:19.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055339227999999997, 2024-09-13T09:13:19.297 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.297 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.297 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.297 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.297 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.297 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4451 2024-09-13T09:13:19.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.297 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581763+0000", 2024-09-13T09:13:19.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581763+0000", 2024-09-13T09:13:19.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581763+0000", 2024-09-13T09:13:19.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581765+0000", 2024-09-13T09:13:19.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4030000000000001e-06 2024-09-13T09:13:19.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581771+0000", 2024-09-13T09:13:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4189999999999999e-06 2024-09-13T09:13:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581776+0000", 2024-09-13T09:13:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5739999999999997e-06 2024-09-13T09:13:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637002+0000", 2024-09-13T09:13:19.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055225819000000002 2024-09-13T09:13:19.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637044+0000", 2024-09-13T09:13:19.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1792999999999998e-05 2024-09-13T09:13:19.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637102+0000", 2024-09-13T09:13:19.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8644000000000003e-05 2024-09-13T09:13:19.302 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.302 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.302 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.302 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4452 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.302 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581780+0000", 2024-09-13T09:13:19.302 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.677220637, 2024-09-13T09:13:19.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055635109000000002, 2024-09-13T09:13:19.302 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.303 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.303 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.303 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.303 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.303 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4452 2024-09-13T09:13:19.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.303 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581780+0000", 2024-09-13T09:13:19.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581780+0000", 2024-09-13T09:13:19.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581780+0000", 2024-09-13T09:13:19.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581782+0000", 2024-09-13T09:13:19.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2790000000000001e-06 2024-09-13T09:13:19.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581788+0000", 2024-09-13T09:13:19.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9460000000000003e-06 2024-09-13T09:13:19.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.306 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581793+0000", 2024-09-13T09:13:19.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:13:19.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.306 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637139+0000", 2024-09-13T09:13:19.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055346237 2024-09-13T09:13:19.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637241+0000", 2024-09-13T09:13:19.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000101924 2024-09-13T09:13:19.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637415+0000", 2024-09-13T09:13:19.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017448400000000001 2024-09-13T09:13:19.307 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.307 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.307 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.308 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4453 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.308 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581797+0000", 2024-09-13T09:13:19.308 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.677203993, 2024-09-13T09:13:19.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055904331000000002, 2024-09-13T09:13:19.308 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.308 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.308 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.308 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.308 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.308 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4453 2024-09-13T09:13:19.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.309 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581797+0000", 2024-09-13T09:13:19.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581797+0000", 2024-09-13T09:13:19.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581796+0000", 2024-09-13T09:13:19.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581799+0000", 2024-09-13T09:13:19.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:13:19.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581804+0000", 2024-09-13T09:13:19.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8579999999999996e-06 2024-09-13T09:13:19.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.311 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.311 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581815+0000", 2024-09-13T09:13:19.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0219e-05 2024-09-13T09:13:19.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.311 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.311 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637446+0000", 2024-09-13T09:13:19.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055631667000000003 2024-09-13T09:13:19.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637490+0000", 2024-09-13T09:13:19.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3223000000000003e-05 2024-09-13T09:13:19.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637701+0000", 2024-09-13T09:13:19.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00021151500000000001 2024-09-13T09:13:19.312 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.312 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.313 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.313 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4454 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 962560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.313 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581819+0000", 2024-09-13T09:13:19.313 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.677181408, 2024-09-13T09:13:19.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056027577000000002, 2024-09-13T09:13:19.313 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.313 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.313 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.313 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.314 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.314 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4454 2024-09-13T09:13:19.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.314 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581819+0000", 2024-09-13T09:13:19.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581819+0000", 2024-09-13T09:13:19.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581819+0000", 2024-09-13T09:13:19.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581822+0000", 2024-09-13T09:13:19.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7020000000000001e-06 2024-09-13T09:13:19.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581828+0000", 2024-09-13T09:13:19.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.489e-06 2024-09-13T09:13:19.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581833+0000", 2024-09-13T09:13:19.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:13:19.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637742+0000", 2024-09-13T09:13:19.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055908823000000003 2024-09-13T09:13:19.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637786+0000", 2024-09-13T09:13:19.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3791999999999997e-05 2024-09-13T09:13:19.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637847+0000", 2024-09-13T09:13:19.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1371000000000006e-05 2024-09-13T09:13:19.318 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.318 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.319 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.319 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4455 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 970752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.319 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581837+0000", 2024-09-13T09:13:19.319 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.6771635379999998, 2024-09-13T09:13:19.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056132979, 2024-09-13T09:13:19.319 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.319 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.319 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.319 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.319 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.320 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4455 2024-09-13T09:13:19.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.320 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581837+0000", 2024-09-13T09:13:19.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581837+0000", 2024-09-13T09:13:19.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581837+0000", 2024-09-13T09:13:19.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581839+0000", 2024-09-13T09:13:19.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.373e-06 2024-09-13T09:13:19.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581845+0000", 2024-09-13T09:13:19.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:13:19.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581850+0000", 2024-09-13T09:13:19.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5140000000000003e-06 2024-09-13T09:13:19.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637877+0000", 2024-09-13T09:13:19.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056027146999999999 2024-09-13T09:13:19.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637918+0000", 2024-09-13T09:13:19.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0911000000000002e-05 2024-09-13T09:13:19.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637970+0000", 2024-09-13T09:13:19.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2466999999999998e-05 2024-09-13T09:13:19.324 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.324 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.324 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.324 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4456 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.324 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581860+0000", 2024-09-13T09:13:19.324 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.6771403180000002, 2024-09-13T09:13:19.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056264427999999998, 2024-09-13T09:13:19.324 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.324 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.324 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.325 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.325 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.325 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4456 2024-09-13T09:13:19.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.325 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581860+0000", 2024-09-13T09:13:19.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581860+0000", 2024-09-13T09:13:19.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581860+0000", 2024-09-13T09:13:19.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581862+0000", 2024-09-13T09:13:19.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3499999999999999e-06 2024-09-13T09:13:19.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581869+0000", 2024-09-13T09:13:19.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4239999999999998e-06 2024-09-13T09:13:19.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581873+0000", 2024-09-13T09:13:19.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7110000000000003e-06 2024-09-13T09:13:19.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638000+0000", 2024-09-13T09:13:19.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056126545999999999 2024-09-13T09:13:19.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638069+0000", 2024-09-13T09:13:19.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8946e-05 2024-09-13T09:13:19.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638125+0000", 2024-09-13T09:13:19.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5826999999999999e-05 2024-09-13T09:13:19.329 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.329 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.329 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.329 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4457 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.329 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581878+0000", 2024-09-13T09:13:19.329 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.677122963, 2024-09-13T09:13:19.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056452293000000001, 2024-09-13T09:13:19.330 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.330 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.330 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.330 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.330 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.330 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4457 2024-09-13T09:13:19.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.330 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581878+0000", 2024-09-13T09:13:19.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581878+0000", 2024-09-13T09:13:19.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581877+0000", 2024-09-13T09:13:19.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581879+0000", 2024-09-13T09:13:19.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0899999999999999e-06 2024-09-13T09:13:19.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581887+0000", 2024-09-13T09:13:19.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2400000000000001e-06 2024-09-13T09:13:19.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581891+0000", 2024-09-13T09:13:19.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8190000000000002e-06 2024-09-13T09:13:19.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638167+0000", 2024-09-13T09:13:19.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056276029999999998 2024-09-13T09:13:19.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638246+0000", 2024-09-13T09:13:19.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9373000000000001e-05 2024-09-13T09:13:19.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638330+0000", 2024-09-13T09:13:19.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4103000000000004e-05 2024-09-13T09:13:19.334 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.334 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.334 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.335 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4458 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.335 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581894+0000", 2024-09-13T09:13:19.335 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.6771062269999999, 2024-09-13T09:13:19.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056678246000000002, 2024-09-13T09:13:19.335 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.335 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.335 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.335 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.335 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.335 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4458 2024-09-13T09:13:19.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.336 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581894+0000", 2024-09-13T09:13:19.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581894+0000", 2024-09-13T09:13:19.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581894+0000", 2024-09-13T09:13:19.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581896+0000", 2024-09-13T09:13:19.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.678e-06 2024-09-13T09:13:19.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581902+0000", 2024-09-13T09:13:19.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:13:19.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581906+0000", 2024-09-13T09:13:19.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4800000000000003e-06 2024-09-13T09:13:19.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638383+0000", 2024-09-13T09:13:19.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05647655 2024-09-13T09:13:19.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638431+0000", 2024-09-13T09:13:19.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7953e-05 2024-09-13T09:13:19.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638573+0000", 2024-09-13T09:13:19.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000141784 2024-09-13T09:13:19.340 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.340 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.340 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.340 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4459 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.340 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581920+0000", 2024-09-13T09:13:19.340 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.6770811800000001, 2024-09-13T09:13:19.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057037290999999997, 2024-09-13T09:13:19.340 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.340 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.341 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.341 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.341 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.341 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4459 2024-09-13T09:13:19.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.341 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581920+0000", 2024-09-13T09:13:19.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581920+0000", 2024-09-13T09:13:19.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581919+0000", 2024-09-13T09:13:19.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581922+0000", 2024-09-13T09:13:19.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6809999999999998e-06 2024-09-13T09:13:19.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581929+0000", 2024-09-13T09:13:19.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2010000000000002e-06 2024-09-13T09:13:19.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581934+0000", 2024-09-13T09:13:19.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1599999999999997e-06 2024-09-13T09:13:19.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638615+0000", 2024-09-13T09:13:19.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056681242999999999 2024-09-13T09:13:19.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638738+0000", 2024-09-13T09:13:19.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012294000000000001 2024-09-13T09:13:19.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638957+0000", 2024-09-13T09:13:19.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000218476 2024-09-13T09:13:19.345 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.345 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.345 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.345 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4460 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.345 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581939+0000", 2024-09-13T09:13:19.346 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.67706182, 2024-09-13T09:13:19.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057296491999999997, 2024-09-13T09:13:19.346 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.346 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.346 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.346 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.346 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.346 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4460 2024-09-13T09:13:19.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.346 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581939+0000", 2024-09-13T09:13:19.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581939+0000", 2024-09-13T09:13:19.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581938+0000", 2024-09-13T09:13:19.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581946+0000", 2024-09-13T09:13:19.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2049999999999996e-06 2024-09-13T09:13:19.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581952+0000", 2024-09-13T09:13:19.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7240000000000003e-06 2024-09-13T09:13:19.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581958+0000", 2024-09-13T09:13:19.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1889999999999998e-06 2024-09-13T09:13:19.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638987+0000", 2024-09-13T09:13:19.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057029006 2024-09-13T09:13:19.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639030+0000", 2024-09-13T09:13:19.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3574e-05 2024-09-13T09:13:19.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639235+0000", 2024-09-13T09:13:19.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00020521 2024-09-13T09:13:19.351 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.351 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.351 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.351 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4461 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.351 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581979+0000", 2024-09-13T09:13:19.351 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.67702156, 2024-09-13T09:13:19.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057454631999999999, 2024-09-13T09:13:19.351 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.352 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.352 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.352 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.352 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.352 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4461 2024-09-13T09:13:19.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.352 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581979+0000", 2024-09-13T09:13:19.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581979+0000", 2024-09-13T09:13:19.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581979+0000", 2024-09-13T09:13:19.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581982+0000", 2024-09-13T09:13:19.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1379999999999999e-06 2024-09-13T09:13:19.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581989+0000", 2024-09-13T09:13:19.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0299999999999996e-06 2024-09-13T09:13:19.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.355 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581994+0000", 2024-09-13T09:13:19.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1660000000000002e-06 2024-09-13T09:13:19.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.355 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639261+0000", 2024-09-13T09:13:19.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057267063999999999 2024-09-13T09:13:19.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639303+0000", 2024-09-13T09:13:19.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1655e-05 2024-09-13T09:13:19.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639434+0000", 2024-09-13T09:13:19.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000131128 2024-09-13T09:13:19.356 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.356 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.357 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.357 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4462 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.357 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581999+0000", 2024-09-13T09:13:19.357 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.6770020400000001, 2024-09-13T09:13:19.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057741334999999998, 2024-09-13T09:13:19.357 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.357 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.358 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.358 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.358 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.358 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4462 2024-09-13T09:13:19.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.358 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.358 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.358 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581999+0000", 2024-09-13T09:13:19.358 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.359 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581999+0000", 2024-09-13T09:13:19.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.359 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581998+0000", 2024-09-13T09:13:19.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.360 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.360 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582001+0000", 2024-09-13T09:13:19.360 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:13:19.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.360 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.360 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582007+0000", 2024-09-13T09:13:19.360 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0889999999999996e-06 2024-09-13T09:13:19.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.360 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582011+0000", 2024-09-13T09:13:19.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:13:19.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639465+0000", 2024-09-13T09:13:19.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057453652000000001 2024-09-13T09:13:19.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639551+0000", 2024-09-13T09:13:19.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.632e-05 2024-09-13T09:13:19.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639740+0000", 2024-09-13T09:13:19.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018866299999999999 2024-09-13T09:13:19.362 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.362 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.362 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.363 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4463 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.363 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.582017+0000", 2024-09-13T09:13:19.363 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.6769839389999999, 2024-09-13T09:13:19.363 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057956270999999997, 2024-09-13T09:13:19.363 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.363 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.363 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.363 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.363 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.363 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4463 2024-09-13T09:13:19.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.364 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.364 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.364 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582017+0000", 2024-09-13T09:13:19.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.364 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.364 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582017+0000", 2024-09-13T09:13:19.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.365 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.365 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582016+0000", 2024-09-13T09:13:19.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.365 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.365 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582019+0000", 2024-09-13T09:13:19.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0999999999999998e-06 2024-09-13T09:13:19.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582024+0000", 2024-09-13T09:13:19.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9510000000000003e-06 2024-09-13T09:13:19.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582029+0000", 2024-09-13T09:13:19.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5449999999999997e-06 2024-09-13T09:13:19.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639769+0000", 2024-09-13T09:13:19.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057739921 2024-09-13T09:13:19.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.367 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639822+0000", 2024-09-13T09:13:19.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2710000000000002e-05 2024-09-13T09:13:19.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.367 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639973+0000", 2024-09-13T09:13:19.368 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000151373 2024-09-13T09:13:19.368 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.368 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.368 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.368 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4464 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:19.368 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.582033+0000", 2024-09-13T09:13:19.368 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.676967791, 2024-09-13T09:13:19.368 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.058326451000000001, 2024-09-13T09:13:19.368 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:19.369 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:19.369 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:19.369 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:19.369 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.369 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4464 2024-09-13T09:13:19.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.369 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:19.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:19.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582033+0000", 2024-09-13T09:13:19.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:19.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582033+0000", 2024-09-13T09:13:19.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:19.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:19.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582033+0000", 2024-09-13T09:13:19.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:19.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582035+0000", 2024-09-13T09:13:19.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:13:19.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:19.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582041+0000", 2024-09-13T09:13:19.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8490000000000002e-06 2024-09-13T09:13:19.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582045+0000", 2024-09-13T09:13:19.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4700000000000004e-06 2024-09-13T09:13:19.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:19.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.640014+0000", 2024-09-13T09:13:19.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057969100000000003 2024-09-13T09:13:19.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:19.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.640106+0000", 2024-09-13T09:13:19.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2074e-05 2024-09-13T09:13:19.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:19.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:19.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:19.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.640359+0000", 2024-09-13T09:13:19.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00025301799999999998 2024-09-13T09:13:19.373 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.373 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.373 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.374 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:19.374 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:19.374 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:19.376 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:19.377 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:19.374+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:19.379 INFO:teuthology.orchestra.run.smithi157.stdout:4177 read 0x1b8b200 thru 0x1b8d5cf (0x23d0 bytes) 2024-09-13T09:13:19.379 INFO:teuthology.orchestra.run.smithi157.stdout:4179 read 0x18f4800 thru 0x18f741f (0x2c20 bytes) 2024-09-13T09:13:19.379 INFO:teuthology.orchestra.run.smithi157.stdout:4180 read 0x1085a00 thru 0x1092851 (0xce52 bytes) 2024-09-13T09:13:19.379 INFO:teuthology.orchestra.run.smithi157.stdout:4181 read 0x1ae4000 thru 0x1aeb90d (0x790e bytes) 2024-09-13T09:13:19.379 INFO:teuthology.orchestra.run.smithi157.stdout:4183 read 0x126aa00 thru 0x1276673 (0xbc74 bytes) 2024-09-13T09:13:19.379 INFO:teuthology.orchestra.run.smithi157.stdout:4185 write 0x5a79000 thru 0x5a85c76 (0xcc77 bytes) 2024-09-13T09:13:19.379 INFO:teuthology.orchestra.run.smithi157.stdout:4187 read 0x3d28800 thru 0x3d36ccb (0xe4cc bytes) 2024-09-13T09:13:19.379 INFO:teuthology.orchestra.run.smithi157.stdout:4188 read 0x2d5cc00 thru 0x2d62fe7 (0x63e8 bytes) 2024-09-13T09:13:19.380 INFO:teuthology.orchestra.run.smithi157.stdout:4189 write 0xa5eac00 thru 0xa5f0058 (0x5459 bytes) 2024-09-13T09:13:19.380 INFO:teuthology.orchestra.run.smithi157.stdout:4191 write 0x456c800 thru 0x4575107 (0x8908 bytes) 2024-09-13T09:13:19.380 INFO:teuthology.orchestra.run.smithi157.stdout:4194 read 0x6fe5600 thru 0x6fea1ea (0x4beb bytes) 2024-09-13T09:13:19.380 INFO:teuthology.orchestra.run.smithi157.stdout:4195 punch from 0x8156800 to 0x8159e00, (0x3600 bytes) 2024-09-13T09:13:19.380 INFO:teuthology.orchestra.run.smithi157.stdout:4197 write 0x8995000 thru 0x8999328 (0x4329 bytes) 2024-09-13T09:13:19.380 INFO:teuthology.orchestra.run.smithi157.stdout:4199 punch from 0x9c8f800 to 0x9c99a00, (0xa200 bytes) 2024-09-13T09:13:19.380 INFO:teuthology.orchestra.run.smithi157.stdout:4202 write 0x30a5800 thru 0x30ae97b (0x917c bytes) 2024-09-13T09:13:19.380 INFO:teuthology.orchestra.run.smithi157.stdout:4204 read 0x866d000 thru 0x86768c7 (0x98c8 bytes) 2024-09-13T09:13:19.380 INFO:teuthology.orchestra.run.smithi157.stdout:4209 read 0x7627800 thru 0x762fbc5 (0x83c6 bytes) 2024-09-13T09:13:19.380 INFO:teuthology.orchestra.run.smithi157.stdout:4211 read 0x252d400 thru 0x2531ac4 (0x46c5 bytes) 2024-09-13T09:13:19.380 INFO:teuthology.orchestra.run.smithi157.stdout:4213 punch from 0x4709e00 to 0x4713600, (0x9800 bytes) 2024-09-13T09:13:19.381 INFO:teuthology.orchestra.run.smithi157.stdout:4216 write 0x4adf200 thru 0x4aef0f3 (0xfef4 bytes) 2024-09-13T09:13:19.381 INFO:teuthology.orchestra.run.smithi157.stdout:4218 write 0xb541200 thru 0xb54a96d (0x976e bytes) 2024-09-13T09:13:19.381 INFO:teuthology.orchestra.run.smithi157.stdout:4219 read 0x9d0b600 thru 0x9d14f31 (0x9932 bytes) 2024-09-13T09:13:19.381 INFO:teuthology.orchestra.run.smithi157.stdout:4220 write 0x6395c00 thru 0x63a4429 (0xe82a bytes) 2024-09-13T09:13:19.381 INFO:teuthology.orchestra.run.smithi157.stdout:4221 read 0xa43aa00 thru 0xa4471bf (0xc7c0 bytes) 2024-09-13T09:13:19.381 INFO:teuthology.orchestra.run.smithi157.stdout:truncating to largest ever: 0xedb9000 2024-09-13T09:13:19.383 INFO:teuthology.orchestra.run.smithi157.stdout:4222 trunc from 0xb54aa00 to 0xedb9000 2024-09-13T09:13:19.411 INFO:teuthology.orchestra.run.smithi157.stdout:4223 punch from 0xd9f0200 to 0xd9f2200, (0x2000 bytes) 2024-09-13T09:13:19.411 INFO:teuthology.orchestra.run.smithi157.stdout:4224 punch from 0x92bc000 to 0x92c6c00, (0xac00 bytes) 2024-09-13T09:13:19.411 INFO:teuthology.orchestra.run.smithi157.stdout:4225 write 0x1810000 thru 0x18164c2 (0x64c3 bytes) 2024-09-13T09:13:19.416 INFO:teuthology.orchestra.run.smithi157.stdout:4228 trunc from 0xedb9000 to 0xb749800 2024-09-13T09:13:19.439 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:13:19.442 INFO:teuthology.orchestra.run.smithi157.stdout:4229 write 0x9a48e00 thru 0x9a5233c (0x953d bytes) 2024-09-13T09:13:19.442 INFO:teuthology.orchestra.run.smithi157.stdout:4232 punch from 0x1f4aa00 to 0x1f50400, (0x5a00 bytes) 2024-09-13T09:13:19.451 INFO:teuthology.orchestra.run.smithi157.stdout:4233 trunc from 0xb749800 to 0x4a80600 2024-09-13T09:13:19.483 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:19.482+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:19.489 INFO:teuthology.orchestra.run.smithi157.stdout:4235 clone 25 order 22 su 131072 sc 13 2024-09-13T09:13:19.507 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:19.507 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:19.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.508 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:22647 3.15 3:a9745ce4:::rbd_data.11233da12af.0000000000000001:head [read 4096~4096 out=4096b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:13:19.508 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:19.507083+0000", 2024-09-13T09:13:19.508 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.000136155, 2024-09-13T09:13:19.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00015599099999999999, 2024-09-13T09:13:19.508 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.508 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:19.508 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.508 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:19.508 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:19.508 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 22647 2024-09-13T09:13:19.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.509 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:19.507083+0000", 2024-09-13T09:13:19.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:19.507083+0000", 2024-09-13T09:13:19.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:19.507082+0000", 2024-09-13T09:13:19.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:13:19.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:19.507087+0000", 2024-09-13T09:13:19.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5689999999999998e-06 2024-09-13T09:13:19.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:19.507097+0000", 2024-09-13T09:13:19.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0190000000000001e-05 2024-09-13T09:13:19.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:19.507102+0000", 2024-09-13T09:13:19.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:13:19.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:19.507138+0000", 2024-09-13T09:13:19.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5179999999999999e-05 2024-09-13T09:13:19.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:19.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:19.507188+0000", 2024-09-13T09:13:19.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9968e-05 2024-09-13T09:13:19.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.512 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.513 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.513 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.513 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-13T09:13:19.513 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 1 2024-09-13T09:13:19.513 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:19.518 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:19.537 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-13T09:13:19.537 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:13:19.537 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-13T09:13:19.537 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:13:19.537 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:13:19.538 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-13T09:13:19.577 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:19.575+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:19.622 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:19.622 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:19.631 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:19.666 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:13:19.667 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:19.677 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:19.676+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:19.737 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:19.737 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:19.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.214205765999999, 2024-09-13T09:13:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:13:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:13:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.738 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:19.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:13:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:13:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:13:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:13:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:13:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:13:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:13:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:13:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:13:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:13:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:13:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:13:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:13:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:13:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.214156709999997, 2024-09-13T09:13:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:13:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:13:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:19.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:13:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:13:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:13:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:13:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:13:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:13:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:13:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:13:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:13:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:13:19.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:13:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:13:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:13:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:13:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.214116635000003, 2024-09-13T09:13:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:13:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:13:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:13:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:13:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:13:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:13:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:13:19.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:13:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:13:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:13:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:13:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:13:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:13:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:13:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:13:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:13:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.752 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.214086819999999, 2024-09-13T09:13:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:13:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:13:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:13:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:13:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:13:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:13:19.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:13:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:13:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:13:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:13:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:13:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:13:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:13:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:13:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:13:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.757 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.214057158999999, 2024-09-13T09:13:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:13:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:13:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:19.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:13:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:13:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:13:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:13:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:13:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:13:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:13:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:13:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:13:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:13:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:13:19.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:13:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:13:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:13:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.214027952000002, 2024-09-13T09:13:19.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:13:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:13:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:13:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:13:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:13:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:13:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:13:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:13:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:13:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:13:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:13:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:13:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:13:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:13:19.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:13:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:13:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.213971770999997, 2024-09-13T09:13:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:13:19.767 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.768 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.768 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.768 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.768 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.768 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:13:19.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.768 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:19.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:13:19.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:13:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:13:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:13:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:13:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:13:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:13:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:13:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:13:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:13:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:13:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:13:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:13:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:13:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.213918309999997, 2024-09-13T09:13:19.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:13:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:13:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:13:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:13:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:13:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:13:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:13:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:13:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:13:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:13:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:13:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:13:19.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:13:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:13:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:13:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:13:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:13:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.777 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.213861479999998, 2024-09-13T09:13:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:13:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:13:19.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:13:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:13:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:13:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:13:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:13:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:13:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:13:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:13:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:13:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:13:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:13:19.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:13:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:13:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:13:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:13:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.213810453000001, 2024-09-13T09:13:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:13:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:13:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:13:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:13:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:13:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:13:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:13:19.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:13:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:13:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:13:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:13:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.213769263000003, 2024-09-13T09:13:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:13:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:13:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:13:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:13:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:13:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:13:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:13:19.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:13:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:13:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:13:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.213727147, 2024-09-13T09:13:19.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:13:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:13:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:13:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:13:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:13:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:13:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:13:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:13:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:13:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:13:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:13:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:13:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:13:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:13:19.796 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.213683267999997, 2024-09-13T09:13:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:13:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.797 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:13:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:13:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:13:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:13:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:13:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:13:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:13:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:13:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:13:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:13:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:13:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:13:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:13:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:13:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:13:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.213637351000003, 2024-09-13T09:13:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:13:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:13:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:13:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:13:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:13:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:13:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:13:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:13:19.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:13:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:13:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:13:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:13:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:13:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:13:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:13:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:13:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:13:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.806 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.213587580000002, 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:13:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:13:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:13:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:13:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:13:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:13:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:13:19.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:13:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:13:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:13:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:13:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:13:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:13:19.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:13:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.213542175999997, 2024-09-13T09:13:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:13:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.811 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:13:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:13:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:13:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:13:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:13:19.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:13:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:13:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:13:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:13:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:13:19.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:13:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:13:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:13:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:13:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:13:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.213491538, 2024-09-13T09:13:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:13:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:13:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:13:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:13:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:13:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:13:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:13:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:13:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:13:19.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:13:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:13:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:13:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:13:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:13:19.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:13:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:13:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:13:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:19.820 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.213435695000001, 2024-09-13T09:13:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:13:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:13:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:13:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:13:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:13:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:13:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:13:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:13:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:13:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:13:19.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:13:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:13:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:13:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:13:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:13:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:13:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.213378773000002, 2024-09-13T09:13:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:13:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:13:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:13:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:13:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:13:19.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:13:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:13:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:13:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:13:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:13:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:13:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:13:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:13:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:13:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:13:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:13:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.829 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.213325210000001, 2024-09-13T09:13:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:13:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:13:19.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:13:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:13:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:13:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:13:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:13:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:13:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:13:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:13:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:13:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:13:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:13:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:13:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:13:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:13:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:19.835 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:19.836 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:19.837 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:19.835+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:19.937 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:19.936+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:19.939 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:19.939 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:19.939 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:19.939 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:19.948 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:20.038 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:20.036+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:20.050 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:20.050 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:20.051 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:20.051 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:20.051 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:20.059 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:20.143 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:20.142+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:20.204 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:20.205 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:20.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:20.205 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:20.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.205 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2185 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.205 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:20.205 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6768679979999996, 2024-09-13T09:13:20.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063348994000000006, 2024-09-13T09:13:20.205 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.205 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.206 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2185 2024-09-13T09:13:20.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.206 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:20.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:20.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525765+0000", 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2390000000000001e-06 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525772+0000", 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:13:20.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525777+0000", 2024-09-13T09:13:20.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0799999999999996e-06 2024-09-13T09:13:20.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589020+0000", 2024-09-13T09:13:20.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063242501000000007 2024-09-13T09:13:20.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589063+0000", 2024-09-13T09:13:20.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2673999999999998e-05 2024-09-13T09:13:20.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589112+0000", 2024-09-13T09:13:20.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9286000000000002e-05 2024-09-13T09:13:20.209 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.209 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.209 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.209 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2187 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.209 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:20.209 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6768484509999997, 2024-09-13T09:13:20.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063500102000000003, 2024-09-13T09:13:20.210 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.210 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.210 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2187 2024-09-13T09:13:20.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.210 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:20.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:20.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:20.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525784+0000", 2024-09-13T09:13:20.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1629999999999999e-06 2024-09-13T09:13:20.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525792+0000", 2024-09-13T09:13:20.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:20.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525796+0000", 2024-09-13T09:13:20.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7389999999999999e-06 2024-09-13T09:13:20.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589132+0000", 2024-09-13T09:13:20.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063336119999999996 2024-09-13T09:13:20.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589187+0000", 2024-09-13T09:13:20.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4126000000000003e-05 2024-09-13T09:13:20.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589283+0000", 2024-09-13T09:13:20.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6032999999999993e-05 2024-09-13T09:13:20.213 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.213 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.213 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.214 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2188 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.214 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:20.214 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6768304050000005, 2024-09-13T09:13:20.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063724617999999997, 2024-09-13T09:13:20.214 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.214 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.214 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.214 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.214 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.214 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2188 2024-09-13T09:13:20.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.214 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:20.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:20.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:20.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525802+0000", 2024-09-13T09:13:20.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0439999999999998e-06 2024-09-13T09:13:20.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525809+0000", 2024-09-13T09:13:20.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.238e-06 2024-09-13T09:13:20.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525814+0000", 2024-09-13T09:13:20.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-13T09:13:20.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589327+0000", 2024-09-13T09:13:20.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063513257000000004 2024-09-13T09:13:20.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589418+0000", 2024-09-13T09:13:20.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0796000000000003e-05 2024-09-13T09:13:20.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589525+0000", 2024-09-13T09:13:20.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010693500000000001 2024-09-13T09:13:20.218 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.218 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.218 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.218 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2190 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.218 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:20.218 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6768123189999997, 2024-09-13T09:13:20.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063863981, 2024-09-13T09:13:20.218 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.218 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.218 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.218 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.219 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.219 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2190 2024-09-13T09:13:20.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.219 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:20.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:20.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525818+0000", 2024-09-13T09:13:20.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525820+0000", 2024-09-13T09:13:20.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:13:20.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525827+0000", 2024-09-13T09:13:20.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0589999999999999e-06 2024-09-13T09:13:20.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525831+0000", 2024-09-13T09:13:20.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7870000000000001e-06 2024-09-13T09:13:20.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589562+0000", 2024-09-13T09:13:20.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063731038000000004 2024-09-13T09:13:20.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589612+0000", 2024-09-13T09:13:20.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0087999999999999e-05 2024-09-13T09:13:20.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589683+0000", 2024-09-13T09:13:20.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0121000000000002e-05 2024-09-13T09:13:20.222 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.222 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.223 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.223 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2192 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.223 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:20.223 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6767953589999998, 2024-09-13T09:13:20.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064050299000000005, 2024-09-13T09:13:20.223 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.223 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.223 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.223 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.223 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.223 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2192 2024-09-13T09:13:20.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.224 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:20.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:20.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525835+0000", 2024-09-13T09:13:20.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525837+0000", 2024-09-13T09:13:20.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:13:20.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525845+0000", 2024-09-13T09:13:20.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3660000000000003e-06 2024-09-13T09:13:20.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525850+0000", 2024-09-13T09:13:20.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0030000000000002e-06 2024-09-13T09:13:20.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589720+0000", 2024-09-13T09:13:20.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063870176000000001 2024-09-13T09:13:20.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589801+0000", 2024-09-13T09:13:20.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0889000000000004e-05 2024-09-13T09:13:20.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589886+0000", 2024-09-13T09:13:20.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5183000000000001e-05 2024-09-13T09:13:20.229 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.229 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.229 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.229 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.229 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:20.229 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6767772560000003, 2024-09-13T09:13:20.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:13:20.229 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.229 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.229 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.230 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.230 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.230 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:13:20.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.230 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:20.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:20.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:13:20.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:13:20.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:13:20.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:13:20.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:13:20.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:13:20.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:13:20.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:13:20.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:13:20.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:13:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:13:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:13:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:13:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.233 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:20.234 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6767595579999997, 2024-09-13T09:13:20.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:13:20.234 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.234 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.234 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.234 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.234 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.234 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:13:20.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.234 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:20.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:20.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:20.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:13:20.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:13:20.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:13:20.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:13:20.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:13:20.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:13:20.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:13:20.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:13:20.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:13:20.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:13:20.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:13:20.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:13:20.237 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6767389430000001, 2024-09-13T09:13:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:13:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:13:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.239 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:20.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:20.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:20.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:13:20.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:13:20.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:13:20.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:13:20.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:13:20.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:13:20.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:13:20.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:13:20.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:13:20.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:13:20.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:13:20.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:13:20.242 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.242 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.242 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.242 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.242 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:20.242 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6767199870000002, 2024-09-13T09:13:20.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:13:20.243 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.243 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.243 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.243 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.243 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.243 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:13:20.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.243 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:20.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:20.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:20.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:13:20.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:13:20.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:13:20.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:13:20.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:13:20.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:13:20.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:13:20.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:13:20.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:13:20.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:13:20.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:13:20.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:13:20.246 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.246 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.246 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.247 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.247 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:20.247 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6767027289999996, 2024-09-13T09:13:20.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:13:20.247 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.247 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.247 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.247 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.247 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.247 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:13:20.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.247 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:20.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:20.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:20.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:13:20.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:13:20.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:13:20.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:13:20.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:13:20.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:13:20.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:13:20.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:13:20.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:13:20.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:13:20.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:13:20.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:13:20.251 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.251 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.251 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.251 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.251 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:20.251 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6766827150000001, 2024-09-13T09:13:20.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:13:20.251 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.251 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.251 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.252 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.252 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.252 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:13:20.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.252 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:20.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:20.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:20.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:13:20.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:13:20.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:13:20.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:13:20.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:13:20.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:13:20.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:13:20.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:13:20.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:13:20.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:13:20.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:13:20.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:13:20.255 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.255 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.255 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.255 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.256 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:20.256 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.676657133, 2024-09-13T09:13:20.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:13:20.256 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.256 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.256 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.256 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.256 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.256 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:13:20.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.256 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:20.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:20.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:13:20.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:13:20.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:13:20.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:13:20.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:13:20.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:13:20.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:13:20.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:13:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:13:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:13:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:13:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:13:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:13:20.259 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.260 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.260 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.260 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.260 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:20.260 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6766378490000005, 2024-09-13T09:13:20.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:13:20.260 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.260 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.260 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.260 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.260 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.261 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:13:20.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.261 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:20.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:20.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:20.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:13:20.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:13:20.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:13:20.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:13:20.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:13:20.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:13:20.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:13:20.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:13:20.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:13:20.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:13:20.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:13:20.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:13:20.264 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.264 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.264 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.264 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.264 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:20.264 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.676620593, 2024-09-13T09:13:20.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:13:20.265 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.265 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.265 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.265 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.265 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.265 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:13:20.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.265 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:20.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:20.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:20.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:13:20.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:13:20.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:13:20.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:13:20.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:13:20.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:13:20.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:13:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:13:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:13:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:13:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:13:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:13:20.268 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6766011130000003, 2024-09-13T09:13:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:13:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.269 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:13:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:13:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:13:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:13:20.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:13:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:13:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:13:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:13:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:13:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:13:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:13:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:13:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:13:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:13:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.273 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6765841840000002, 2024-09-13T09:13:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:13:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:13:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.274 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:13:20.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:13:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:13:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:13:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:13:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:13:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:13:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:13:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:13:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:13:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:13:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:13:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:13:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6765650350000003, 2024-09-13T09:13:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:13:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.278 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:13:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:20.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:13:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:13:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:13:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:13:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:13:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:13:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:13:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:13:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:13:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:13:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:13:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:13:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.282 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6765468959999996, 2024-09-13T09:13:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:13:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:13:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:13:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:13:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:13:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:13:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:13:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:13:20.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:13:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:13:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:13:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:13:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:13:20.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:13:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6761364009999999, 2024-09-13T09:13:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:13:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:13:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:13:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:13:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:13:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:13:20.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:13:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:13:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:13:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:13:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:13:20.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:13:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:13:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:13:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.6761134999999996, 2024-09-13T09:13:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:13:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:13:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:13:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:13:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:13:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:13:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:13:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:13:20.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:13:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:13:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:13:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:13:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:13:20.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:13:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.296 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.296 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:20.297 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:20.298 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:20.296+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:20.398 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:20.398+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:20.444 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-09-13T09:13:20.445 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:20.445 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:20.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.445 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:22760 3.1d 3.20d8a55d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:13:20.445 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:20.443851+0000", 2024-09-13T09:13:20.445 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00049903800000000004, 2024-09-13T09:13:20.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00051511000000000005, 2024-09-13T09:13:20.445 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.445 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "op must wait for map", 2024-09-13T09:13:20.445 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.446 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:20.446 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:20.514 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 22760 2024-09-13T09:13:20.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.514 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:20.443851+0000", 2024-09-13T09:13:20.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:20.443851+0000", 2024-09-13T09:13:20.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:20.443849+0000", 2024-09-13T09:13:20.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:13:20.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:20.443859+0000", 2024-09-13T09:13:20.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0052e-05 2024-09-13T09:13:20.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:20.443877+0000", 2024-09-13T09:13:20.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7510999999999999e-05 2024-09-13T09:13:20.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:20.443888+0000", 2024-09-13T09:13:20.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1286e-05 2024-09-13T09:13:20.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:20.443937+0000", 2024-09-13T09:13:20.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8402000000000002e-05 2024-09-13T09:13:20.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:13:20.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:20.443942+0000", 2024-09-13T09:13:20.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.468e-06 2024-09-13T09:13:20.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.518 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.518 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-13T09:13:20.518 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 1 2024-09-13T09:13:20.518 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:20.520 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:20.520 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:20.518+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:20.620 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:20.619+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:20.666 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:20.666 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:20.667 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:20.667 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:20.667 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:20.676 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:20.761 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:20.759+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:20.822 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:20.820+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:20.825 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9035 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2342912~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.07863261, 2024-09-13T09:13:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070293606999999994, 2024-09-13T09:13:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.826 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9035 2024-09-13T09:13:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745602+0000", 2024-09-13T09:13:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-06 2024-09-13T09:13:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745607+0000", 2024-09-13T09:13:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0770000000000001e-06 2024-09-13T09:13:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745609+0000", 2024-09-13T09:13:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3499999999999999e-06 2024-09-13T09:13:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815626+0000", 2024-09-13T09:13:20.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070017398999999994 2024-09-13T09:13:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815672+0000", 2024-09-13T09:13:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5275999999999998e-05 2024-09-13T09:13:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815895+0000", 2024-09-13T09:13:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022328300000000001 2024-09-13T09:13:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.078620065999999, 2024-09-13T09:13:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:13:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:13:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:20.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:13:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:13:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:13:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:13:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:13:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:13:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:13:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:13:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:13:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:13:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:13:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.078611387999999, 2024-09-13T09:13:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:13:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.835 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:13:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:13:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:13:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:13:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:13:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:13:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:13:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:13:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:13:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:13:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:13:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:13:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:13:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:20.839 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.078602402000001, 2024-09-13T09:13:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:13:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:13:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:13:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:13:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:13:20.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:13:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:13:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:13:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:13:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:13:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:13:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:13:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:13:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:13:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.844 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.078594429999999, 2024-09-13T09:13:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:13:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:13:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:20.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:13:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:13:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:13:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:13:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:13:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:13:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:13:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:13:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:13:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:13:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:13:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:13:20.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.076719596, 2024-09-13T09:13:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:13:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:13:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:13:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:13:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:13:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:13:20.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:13:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:13:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:13:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:13:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:13:20.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:13:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:13:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:13:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.076708934999999, 2024-09-13T09:13:20.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:13:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:13:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:13:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:13:20.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:13:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:13:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:13:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:13:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:13:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:13:20.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:13:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:13:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:13:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:13:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.076686959, 2024-09-13T09:13:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:13:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:13:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:20.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:13:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:13:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:13:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:13:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:13:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:13:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:13:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:13:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:13:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:13:20.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:13:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:13:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.076673456999998, 2024-09-13T09:13:20.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:13:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:13:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:13:20.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:13:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:13:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:13:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:13:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:13:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:13:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:13:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:13:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:13:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:13:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:13:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.866 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9055 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 6930432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.074680677, 2024-09-13T09:13:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070177949000000003, 2024-09-13T09:13:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9055 2024-09-13T09:13:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:20.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749554+0000", 2024-09-13T09:13:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.11e-06 2024-09-13T09:13:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749557+0000", 2024-09-13T09:13:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7949999999999999e-06 2024-09-13T09:13:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749559+0000", 2024-09-13T09:13:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:13:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819633+0000", 2024-09-13T09:13:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073995 2024-09-13T09:13:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819679+0000", 2024-09-13T09:13:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6006e-05 2024-09-13T09:13:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819731+0000", 2024-09-13T09:13:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1934000000000001e-05 2024-09-13T09:13:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9056 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 7159808~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.074672627999998, 2024-09-13T09:13:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070283126000000001, 2024-09-13T09:13:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.871 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9056 2024-09-13T09:13:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:20.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749562+0000", 2024-09-13T09:13:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1340000000000001e-06 2024-09-13T09:13:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749565+0000", 2024-09-13T09:13:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8760000000000001e-06 2024-09-13T09:13:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749567+0000", 2024-09-13T09:13:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2340000000000001e-06 2024-09-13T09:13:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819750+0000", 2024-09-13T09:13:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070182537000000003 2024-09-13T09:13:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819792+0000", 2024-09-13T09:13:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2141999999999997e-05 2024-09-13T09:13:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819844+0000", 2024-09-13T09:13:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2428e-05 2024-09-13T09:13:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.875 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.074664359, 2024-09-13T09:13:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:13:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:13:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:20.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:13:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:13:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:13:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:13:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:13:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:13:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:13:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:13:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:13:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:13:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:13:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:13:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.879 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.074640037000002, 2024-09-13T09:13:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:13:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.880 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:13:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:13:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:13:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:13:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:13:20.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:13:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:13:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:13:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:13:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:13:20.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:13:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:13:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:13:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.074629063, 2024-09-13T09:13:20.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:13:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:13:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:20.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:13:20.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:13:20.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:13:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:13:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:13:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:13:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:13:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:13:20.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:13:20.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:13:20.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:13:20.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:13:20.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:13:20.888 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.889 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.889 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.889 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9063 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1589248~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.889 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:20.889 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.072725421000001, 2024-09-13T09:13:20.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070391924999999994, 2024-09-13T09:13:20.889 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.889 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.889 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.889 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.889 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.890 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9063 2024-09-13T09:13:20.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.890 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:20.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:20.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:20.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:20.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751509+0000", 2024-09-13T09:13:20.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.142e-06 2024-09-13T09:13:20.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751513+0000", 2024-09-13T09:13:20.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6669999999999998e-06 2024-09-13T09:13:20.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751520+0000", 2024-09-13T09:13:20.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5910000000000001e-06 2024-09-13T09:13:20.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821593+0000", 2024-09-13T09:13:20.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073519000000001 2024-09-13T09:13:20.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821648+0000", 2024-09-13T09:13:20.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:13:20.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821900+0000", 2024-09-13T09:13:20.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025263999999999999 2024-09-13T09:13:20.893 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.893 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.893 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.893 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.893 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:20.893 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.072711437999999, 2024-09-13T09:13:20.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:13:20.894 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.894 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.894 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.894 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.894 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.894 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:13:20.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.894 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:20.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:20.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:20.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:20.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:13:20.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:13:20.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:13:20.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:20.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:13:20.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:13:20.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:13:20.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:13:20.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:13:20.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:13:20.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:13:20.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:13:20.897 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.897 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.898 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.898 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.898 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:20.898 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.072702040999999, 2024-09-13T09:13:20.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:13:20.898 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.898 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.898 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.898 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.898 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.898 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:13:20.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.899 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:20.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:20.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:20.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:20.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:13:20.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:13:20.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:13:20.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:13:20.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:13:20.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:13:20.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:13:20.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:13:20.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:13:20.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:13:20.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:13:20.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:13:20.902 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.902 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.902 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.902 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9069 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2965504~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.902 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:20.902 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.070742662000001, 2024-09-13T09:13:20.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070184959000000005, 2024-09-13T09:13:20.902 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.902 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.903 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.903 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.903 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.903 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9069 2024-09-13T09:13:20.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.903 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:20.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:20.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:20.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:20.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753492+0000", 2024-09-13T09:13:20.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.403e-06 2024-09-13T09:13:20.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753495+0000", 2024-09-13T09:13:20.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9390000000000002e-06 2024-09-13T09:13:20.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753504+0000", 2024-09-13T09:13:20.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5129999999999996e-06 2024-09-13T09:13:20.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823390+0000", 2024-09-13T09:13:20.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.069886632000000004 2024-09-13T09:13:20.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823445+0000", 2024-09-13T09:13:20.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4880999999999999e-05 2024-09-13T09:13:20.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823676+0000", 2024-09-13T09:13:20.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000230812 2024-09-13T09:13:20.906 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.906 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.906 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.907 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9070 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3194880~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.907 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:20.907 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.070727570999999, 2024-09-13T09:13:20.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070280588000000005, 2024-09-13T09:13:20.907 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.907 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.907 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.907 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.907 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.907 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9070 2024-09-13T09:13:20.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.907 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:20.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:20.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:20.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:20.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753507+0000", 2024-09-13T09:13:20.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:20.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753510+0000", 2024-09-13T09:13:20.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0460000000000001e-06 2024-09-13T09:13:20.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753513+0000", 2024-09-13T09:13:20.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:13:20.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823689+0000", 2024-09-13T09:13:20.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070176083 2024-09-13T09:13:20.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823734+0000", 2024-09-13T09:13:20.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5056999999999999e-05 2024-09-13T09:13:20.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823787+0000", 2024-09-13T09:13:20.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3121000000000002e-05 2024-09-13T09:13:20.910 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9071 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.070719023999999, 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070386377999999999, 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9071 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.911 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:20.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:20.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:20.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:20.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:20.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:20.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:20.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:20.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:20.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:20.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753516+0000", 2024-09-13T09:13:20.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1519999999999999e-06 2024-09-13T09:13:20.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:20.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753519+0000", 2024-09-13T09:13:20.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6559999999999998e-06 2024-09-13T09:13:20.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:20.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753522+0000", 2024-09-13T09:13:20.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:13:20.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:20.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823798+0000", 2024-09-13T09:13:20.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070275907999999998 2024-09-13T09:13:20.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:20.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823845+0000", 2024-09-13T09:13:20.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7593000000000003e-05 2024-09-13T09:13:20.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:20.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:20.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:20.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823901+0000", 2024-09-13T09:13:20.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5934999999999999e-05 2024-09-13T09:13:20.915 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.915 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.915 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.915 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:20.915 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:20.915 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:20.916 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:20.960 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:20.959+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:21.024 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:21.023+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:21.031 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:21.031 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:21.038 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-13T09:13:21.040 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:21.124 INFO:teuthology.orchestra.run.smithi157.stdout:4240 read 0x3d74a00 thru 0x3d7e93e (0x9f3f bytes) 2024-09-13T09:13:21.125 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:21.123+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:21.133 INFO:teuthology.orchestra.run.smithi157.stdout:4242 trunc from 0x4a80600 to 0xab8de00 2024-09-13T09:13:21.151 INFO:teuthology.orchestra.run.smithi157.stdout:4246 read 0x654d800 thru 0x6559fd9 (0xc7da bytes) 2024-09-13T09:13:21.154 INFO:teuthology.orchestra.run.smithi157.stdout:4247 trunc from 0xab8de00 to 0xce35a00 2024-09-13T09:13:21.173 INFO:teuthology.orchestra.run.smithi157.stdout:4249 trunc from 0xce35a00 to 0x3685800 2024-09-13T09:13:21.195 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:13:21.207 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:21.208 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:21.208 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:21.208 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:21.208 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:21.223 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:21.225 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:21.224+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:21.260 INFO:teuthology.orchestra.run.smithi157.stdout:4250 write 0xe360600 thru 0xe368ab2 (0x84b3 bytes) 2024-09-13T09:13:21.261 INFO:teuthology.orchestra.run.smithi157.stdout:4253 read 0xa12d600 thru 0xa13a0fe (0xcaff bytes) 2024-09-13T09:13:21.261 INFO:teuthology.orchestra.run.smithi157.stdout:4256 write 0x7fe5c00 thru 0x7ff0451 (0xa852 bytes) 2024-09-13T09:13:21.261 INFO:teuthology.orchestra.run.smithi157.stdout:4258 punch from 0xbf1fc00 to 0xbf2e000, (0xe400 bytes) 2024-09-13T09:13:21.261 INFO:teuthology.orchestra.run.smithi157.stdout:4259 punch from 0x1387c00 to 0x138e800, (0x6c00 bytes) 2024-09-13T09:13:21.267 INFO:teuthology.orchestra.run.smithi157.stdout:4260 trunc from 0xe368c00 to 0xa9f4e00 2024-09-13T09:13:21.291 INFO:teuthology.orchestra.run.smithi157.stdout:4261 read 0xa5bfe00 thru 0xa5cc8df (0xcae0 bytes) 2024-09-13T09:13:21.292 INFO:teuthology.orchestra.run.smithi157.stdout:4263 read 0x64f7600 thru 0x64faa11 (0x3412 bytes) 2024-09-13T09:13:21.292 INFO:teuthology.orchestra.run.smithi157.stdout:4265 write 0x6378200 thru 0x6382be8 (0xa9e9 bytes) 2024-09-13T09:13:21.295 INFO:teuthology.orchestra.run.smithi157.stdout:4270 read 0x5cfae00 thru 0x5d0a51f (0xf720 bytes) 2024-09-13T09:13:21.295 INFO:teuthology.orchestra.run.smithi157.stdout:4271 write 0x8a81a00 thru 0x8a81e5c (0x45d bytes) 2024-09-13T09:13:21.296 INFO:teuthology.orchestra.run.smithi157.stdout:4272 read 0x9c5c000 thru 0x9c63a78 (0x7a79 bytes) 2024-09-13T09:13:21.296 INFO:teuthology.orchestra.run.smithi157.stdout:4273 read 0x804be00 thru 0x8053da5 (0x7fa6 bytes) 2024-09-13T09:13:21.296 INFO:teuthology.orchestra.run.smithi157.stdout:4276 read 0x4a43400 thru 0x4a52e2d (0xfa2e bytes) 2024-09-13T09:13:21.296 INFO:teuthology.orchestra.run.smithi157.stdout:4278 write 0x7ea6600 thru 0x7ea6c44 (0x645 bytes) 2024-09-13T09:13:21.302 INFO:teuthology.orchestra.run.smithi157.stdout:4279 trunc from 0xa9f4e00 to 0x3febe00 2024-09-13T09:13:21.326 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:21.326+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:21.338 INFO:teuthology.orchestra.run.smithi157.stdout:4282 trunc from 0x3febe00 to 0xa0bce00 2024-09-13T09:13:21.347 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:21.348 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:21.350 INFO:teuthology.orchestra.run.smithi157.stdout:4283 trunc from 0xa0bce00 to 0xd3ae200 2024-09-13T09:13:21.359 INFO:teuthology.orchestra.run.smithi157.stdout:4287 read 0x4309000 thru 0x4316af9 (0xdafa bytes) 2024-09-13T09:13:21.360 INFO:teuthology.orchestra.run.smithi157.stdout:4288 read 0x6b20c00 thru 0x6b25670 (0x4a71 bytes) 2024-09-13T09:13:21.360 INFO:teuthology.orchestra.run.smithi157.stdout:4294 write 0x4ac7800 thru 0x4ad1ac9 (0xa2ca bytes) 2024-09-13T09:13:21.366 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:21.366 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:21.373 INFO:teuthology.orchestra.run.smithi157.stdout:4297 trunc from 0xd3ae200 to 0x32bb400 2024-09-13T09:13:21.386 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:13:21.386 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:13:21.393 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:21.393 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:21.407 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:21.424 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:21.424 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:21.428 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:21.427+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:21.459 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:21.459 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:21.490 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:21.490 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:21.528 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:21.528+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:21.551 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:21.551 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:21.560 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:21.629 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:21.628+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:21.676 INFO:teuthology.orchestra.run.smithi157.stdout:4299 read 0x22de00 thru 0x23772e (0x992f bytes) 2024-09-13T09:13:21.677 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:21.677 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:21.684 INFO:teuthology.orchestra.run.smithi157.stdout:4300 trunc from 0x32bb400 to 0x47b1a00 2024-09-13T09:13:21.686 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:21.693 INFO:teuthology.orchestra.run.smithi157.stdout:4301 write 0x6fde00 thru 0x6fe33b (0x53c bytes) 2024-09-13T09:13:21.699 INFO:teuthology.orchestra.run.smithi157.stdout:4302 trunc from 0x47b1a00 to 0x77da200 2024-09-13T09:13:21.708 INFO:teuthology.orchestra.run.smithi157.stdout:4303 write 0x6189c00 thru 0x61969c2 (0xcdc3 bytes) 2024-09-13T09:13:21.709 INFO:teuthology.orchestra.run.smithi157.stdout:4304 write 0x5b76e00 thru 0x5b7817d (0x137e bytes) 2024-09-13T09:13:21.711 INFO:teuthology.orchestra.run.smithi157.stdout:4305 trunc from 0x77da200 to 0x5e94800 2024-09-13T09:13:21.729 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:21.729+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:21.730 INFO:teuthology.orchestra.run.smithi157.stdout:4306 read 0x1de3400 thru 0x1de4168 (0xd69 bytes) 2024-09-13T09:13:21.730 INFO:teuthology.orchestra.run.smithi157.stdout:4308 punch from 0x19c400 to 0x1a4e00, (0x8a00 bytes) 2024-09-13T09:13:21.745 INFO:teuthology.orchestra.run.smithi157.stdout:4309 write 0x96b5400 thru 0x96c3de0 (0xe9e1 bytes) 2024-09-13T09:13:21.746 INFO:teuthology.orchestra.run.smithi157.stdout:4311 trunc from 0x96c3e00 to 0x88ee400 2024-09-13T09:13:21.766 INFO:teuthology.orchestra.run.smithi157.stdout:4315 write 0x2a69000 thru 0x2a6d45b (0x445c bytes) 2024-09-13T09:13:21.766 INFO:teuthology.orchestra.run.smithi157.stdout:4316 read 0x1ee9000 thru 0x1eee0e9 (0x50ea bytes) 2024-09-13T09:13:21.767 INFO:teuthology.orchestra.run.smithi157.stdout:4320 read 0x1018c00 thru 0x1027fda (0xf3db bytes) 2024-09-13T09:13:21.774 INFO:teuthology.orchestra.run.smithi157.stdout:4321 trunc from 0x88ee400 to 0x2eaa000 2024-09-13T09:13:21.809 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:21.809 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:21.813 INFO:teuthology.orchestra.run.smithi157.stdout:4322 read 0xe88800 thru 0xe8c7bf (0x3fc0 bytes) 2024-09-13T09:13:21.813 INFO:teuthology.orchestra.run.smithi157.stdout:4323 read 0x98c600 thru 0x998faf (0xc9b0 bytes) 2024-09-13T09:13:21.814 INFO:teuthology.orchestra.run.smithi157.stdout:4324 read 0x13b800 thru 0x143c5c (0x845d bytes) 2024-09-13T09:13:21.814 INFO:teuthology.orchestra.run.smithi157.stdout:4325 punch from 0x108c000 to 0x1099a00, (0xda00 bytes) 2024-09-13T09:13:21.819 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:13:21.831 INFO:teuthology.orchestra.run.smithi157.stdout:4326 write 0x2fbc600 thru 0x2fc8347 (0xbd48 bytes) 2024-09-13T09:13:21.831 INFO:teuthology.orchestra.run.smithi157.stdout:4328 read 0x216fe00 thru 0x2172006 (0x2207 bytes) 2024-09-13T09:13:21.832 INFO:teuthology.orchestra.run.smithi157.stdout:4331 read 0x6ee600 thru 0x6f2721 (0x4122 bytes) 2024-09-13T09:13:21.832 INFO:teuthology.orchestra.run.smithi157.stdout:4333 read 0x2ec2200 thru 0x2ec5253 (0x3054 bytes) 2024-09-13T09:13:21.832 INFO:teuthology.orchestra.run.smithi157.stdout:4334 clone 26 order 19 su 4096 sc 11 2024-09-13T09:13:21.864 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:21.863+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:21.926 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:21.926 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:21.931 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:21.930+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:21.934 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:13:22.032 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:22.031+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:22.080 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:22.080 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:22.089 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:13:22.133 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:22.132+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:22.195 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:22.195 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:22.204 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:22.248 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:22.247+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:22.308 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:22.308 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:22.318 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:22.359 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:22.358+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:22.435 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:22.435+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:22.462 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:22.463 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:22.472 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:22.520 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:13:22.522 INFO:teuthology.orchestra.run.smithi157.stdout:flattening image image_client.0-clone25 2024-09-13T09:13:22.555 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:22.555+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:22.616 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:22.616 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:22.625 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:22.636 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:22.635+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:22.737 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:22.736+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:22.773 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:22.773 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:22.781 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:22.821 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-13T09:13:22.838 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:22.837+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:22.915 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:22.916 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:22.927 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:22.938 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:22.937+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:23.039 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:23.038+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:23.069 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:23.069 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:23.077 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:23.139 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:23.139+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:23.238 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:23.238 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:23.240 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:23.239+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:23.249 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:23.342 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:23.341+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:23.367 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:23.368 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:23.376 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:23.424 INFO:teuthology.orchestra.run.smithi157.stdout:4335 write 0x5b67400 thru 0x5b6e137 (0x6d38 bytes) 2024-09-13T09:13:23.454 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-09-13T09:13:23.461 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:23.460+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:23.462 INFO:teuthology.orchestra.run.smithi157.stdout:4336 write 0x7911c00 thru 0x791a60e (0x8a0f bytes) 2024-09-13T09:13:23.466 INFO:teuthology.orchestra.run.smithi157.stdout:4337 read 0x12d200 thru 0x130b9a (0x399b bytes) 2024-09-13T09:13:23.503 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:13:23.504 INFO:teuthology.orchestra.run.smithi157.stdout:4338 write 0xe381000 thru 0xe384d3f (0x3d40 bytes) 2024-09-13T09:13:23.504 INFO:teuthology.orchestra.run.smithi157.stdout:4340 read 0x6ac000 thru 0x6b83e3 (0xc3e4 bytes) 2024-09-13T09:13:23.507 INFO:teuthology.orchestra.run.smithi157.stdout:4341 write 0x2caf800 thru 0x2cb2d5d (0x355e bytes) 2024-09-13T09:13:23.508 INFO:teuthology.orchestra.run.smithi157.stdout:4344 write 0x1247000 thru 0x125594d (0xe94e bytes) 2024-09-13T09:13:23.535 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:23.536 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:23.545 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:23.551 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:23.550+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:23.629 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:23.630 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:23.645 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:23.645+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:23.647 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:23.647 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:23.660 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:23.660 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:23.660 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:23.660 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:23.663 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:13:23.663 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:13:23.669 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:23.686 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:23.686 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:23.702 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:23.703 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:23.730 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:23.746 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:23.745+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:23.793 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:23.793 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:23.794 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:23.794 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:23.803 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:23.825 INFO:teuthology.orchestra.run.smithi157.stdout:4346 write 0xe709600 thru 0xe717234 (0xdc35 bytes) 2024-09-13T09:13:23.825 INFO:teuthology.orchestra.run.smithi157.stdout:4347 write 0xc725400 thru 0xc735378 (0xff79 bytes) 2024-09-13T09:13:23.825 INFO:teuthology.orchestra.run.smithi157.stdout:4348 punch from 0x9cc5a00 to 0x9ccf800, (0x9e00 bytes) 2024-09-13T09:13:23.826 INFO:teuthology.orchestra.run.smithi157.stdout:4349 read 0x95fba00 thru 0x96037ff (0x7e00 bytes) 2024-09-13T09:13:23.826 INFO:teuthology.orchestra.run.smithi157.stdout:4350 write 0x59dbc00 thru 0x59e93dd (0xd7de bytes) 2024-09-13T09:13:23.828 INFO:teuthology.orchestra.run.smithi157.stdout:4351 read 0x5d9ba00 thru 0x5d9bf4a (0x54b bytes) 2024-09-13T09:13:23.828 INFO:teuthology.orchestra.run.smithi157.stdout:4354 read 0xa6e1200 thru 0xa6e23b0 (0x11b1 bytes) 2024-09-13T09:13:23.828 INFO:teuthology.orchestra.run.smithi157.stdout:4355 trunc from 0xe717400 to 0xebe1800 2024-09-13T09:13:23.847 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:23.846+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:23.875 INFO:teuthology.orchestra.run.smithi157.stdout:4359 write 0x7dc5400 thru 0x7dc62ec (0xeed bytes) 2024-09-13T09:13:23.879 INFO:teuthology.orchestra.run.smithi157.stdout:4361 trunc from 0xebe1800 to 0xd246800 2024-09-13T09:13:23.908 INFO:teuthology.orchestra.run.smithi157.stdout:4364 punch from 0xbf7fc00 to 0xbf82800, (0x2c00 bytes) 2024-09-13T09:13:23.908 INFO:teuthology.orchestra.run.smithi157.stdout:4365 read 0x27fa00 thru 0x28c9e3 (0xcfe4 bytes) 2024-09-13T09:13:23.912 INFO:teuthology.orchestra.run.smithi157.stdout:4368 read 0x9235600 thru 0x9239f7f (0x4980 bytes) 2024-09-13T09:13:23.913 INFO:teuthology.orchestra.run.smithi157.stdout:4369 punch from 0x9787e00 to 0x9789000, (0x1200 bytes) 2024-09-13T09:13:23.913 INFO:teuthology.orchestra.run.smithi157.stdout:4370 read 0x84a7400 thru 0x84aa536 (0x3137 bytes) 2024-09-13T09:13:23.913 INFO:teuthology.orchestra.run.smithi157.stdout:4371 punch from 0x5467800 to 0x5477200, (0xfa00 bytes) 2024-09-13T09:13:23.913 INFO:teuthology.orchestra.run.smithi157.stdout:4372 punch from 0x778ca00 to 0x7792e00, (0x6400 bytes) 2024-09-13T09:13:23.913 INFO:teuthology.orchestra.run.smithi157.stdout:4373 read 0x54f3a00 thru 0x54fe35f (0xa960 bytes) 2024-09-13T09:13:23.913 INFO:teuthology.orchestra.run.smithi157.stdout:4374 write 0x455e600 thru 0x456e153 (0xfb54 bytes) 2024-09-13T09:13:23.914 INFO:teuthology.orchestra.run.smithi157.stdout:4375 read 0x322c200 thru 0x32384c1 (0xc2c2 bytes) 2024-09-13T09:13:23.914 INFO:teuthology.orchestra.run.smithi157.stdout:4376 write 0xae6e00 thru 0xae96a3 (0x28a4 bytes) 2024-09-13T09:13:23.914 INFO:teuthology.orchestra.run.smithi157.stdout:4378 write 0x8717600 thru 0x87251ba (0xdbbb bytes) 2024-09-13T09:13:23.919 INFO:teuthology.orchestra.run.smithi157.stdout:4381 trunc from 0xd246800 to 0x9051400 2024-09-13T09:13:23.943 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.410956947000003, 2024-09-13T09:13:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:13:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:23.943 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:13:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:23.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:13:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:13:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:13:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:13:23.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:13:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:13:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:13:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:13:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:13:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:13:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:13:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:13:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:13:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:13:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:13:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:13:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:13:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:13:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:13:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:13:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:13:23.948 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.949 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:23.949 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.949 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:23.949 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:23.949 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.410938911999999, 2024-09-13T09:13:23.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:13:23.949 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:23.949 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:23.949 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:23.949 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:23.949 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:23.949 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:13:23.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.950 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:23.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:23.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:23.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:23.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:23.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:23.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:23.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:23.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:23.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:13:23.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:13:23.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:23.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:13:23.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:13:23.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:23.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:13:23.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:13:23.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:13:23.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:13:23.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:13:23.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:13:23.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:13:23.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:13:23.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:13:23.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:13:23.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:13:23.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:13:23.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:23.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:13:23.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:13:23.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:23.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:13:23.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:13:23.954 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.954 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:23.954 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.955 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:23.955 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:23.955 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.410922241999998, 2024-09-13T09:13:23.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:13:23.955 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:23.955 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:23.955 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:23.955 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:23.955 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:23.955 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:13:23.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.955 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:23.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:23.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:23.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:23.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:23.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:23.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:13:23.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:23.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:23.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:13:23.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:23.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:23.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:13:23.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:13:23.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:23.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:13:23.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:13:23.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:13:23.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:13:23.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:13:23.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:13:23.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:13:23.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:13:23.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:13:23.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:13:23.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:13:23.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:13:23.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:23.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:13:23.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:13:23.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:23.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:13:23.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:13:23.960 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.960 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:23.960 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.960 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:23.960 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:23.961 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.410904610000003, 2024-09-13T09:13:23.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:13:23.961 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:23.961 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:23.961 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:23.961 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:23.961 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:23.961 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:13:23.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.961 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:23.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:23.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:23.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:13:23.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:23.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:13:23.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:13:23.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:23.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:13:23.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:13:23.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:13:23.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:13:23.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:13:23.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:13:23.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:13:23.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:13:23.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:13:23.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:13:23.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:13:23.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:13:23.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:23.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:13:23.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:13:23.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:23.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:13:23.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:13:23.966 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.966 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:23.966 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.966 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:23.966 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:23.966 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.410858304999998, 2024-09-13T09:13:23.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:13:23.966 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:23.966 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:23.966 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:23.966 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:23.967 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:23.967 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:13:23.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.967 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:23.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:23.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:23.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:23.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:23.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:23.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:13:23.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:23.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:23.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:13:23.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:13:23.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:23.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:13:23.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:13:23.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:23.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:13:23.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:13:23.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:13:23.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:13:23.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:13:23.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:13:23.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:13:23.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:13:23.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:13:23.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:13:23.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.972 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:23.972 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:23.972 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.410833003999997, 2024-09-13T09:13:23.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:13:23.972 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:23.972 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:23.972 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:23.972 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:23.972 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:23.972 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:13:23.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.972 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:23.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:23.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:23.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:23.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:23.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:23.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:13:23.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:23.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:23.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:13:23.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:13:23.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:23.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:13:23.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:13:23.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:23.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:13:23.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:13:23.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:13:23.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:13:23.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:13:23.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:13:23.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:13:23.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:13:23.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:13:23.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:13:23.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:13:23.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:13:23.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:23.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:13:23.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:13:23.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:23.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:13:23.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:13:23.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.977 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:23.977 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.978 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:23.978 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:23.978 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.410814774999999, 2024-09-13T09:13:23.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:13:23.978 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:23.978 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:23.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:23.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:23.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:23.978 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:13:23.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.978 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:23.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:23.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:23.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:23.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:23.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:23.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:23.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:23.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:23.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:13:23.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:13:23.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:23.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:13:23.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:13:23.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:23.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:13:23.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:13:23.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:13:23.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:13:23.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:13:23.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:13:23.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:13:23.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:13:23.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:13:23.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:13:23.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:13:23.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:13:23.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:23.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:13:23.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:13:23.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:23.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:13:23.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:13:23.983 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.983 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:23.983 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.983 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:23.983 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:23.983 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.410767456000002, 2024-09-13T09:13:23.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:13:23.984 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:23.984 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:23.984 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:23.984 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:23.984 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:23.984 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:13:23.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.984 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:23.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:23.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:23.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:23.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:23.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:23.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:23.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:23.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:23.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:13:23.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:13:23.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:23.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:13:23.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:13:23.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:23.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:13:23.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:13:23.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:13:23.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:13:23.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:13:23.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:13:23.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:13:23.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:13:23.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:13:23.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:13:23.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:13:23.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:13:23.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:23.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:13:23.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:13:23.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:23.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:13:23.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:13:23.989 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.989 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:23.989 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.989 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:23.989 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:23.989 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.410253218000001, 2024-09-13T09:13:23.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:13:23.989 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:23.989 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:23.989 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:23.989 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:23.989 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:23.990 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:13:23.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.990 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:23.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:23.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:23.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:23.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:23.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:23.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:13:23.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:13:23.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:23.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:13:23.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:13:23.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:23.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:13:23.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:13:23.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:23.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:13:23.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:13:23.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:13:23.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:13:23.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:13:23.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:13:23.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:13:23.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:13:23.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:13:23.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:13:23.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:13:23.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:13:23.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:23.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:13:23.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:13:23.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:23.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:13:23.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:13:23.994 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.994 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:23.994 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:23.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.995 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:23.995 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:23.995 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.410218688, 2024-09-13T09:13:23.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:13:23.995 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:23.995 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:23.995 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:23.995 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:23.995 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:23.995 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:13:23.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.995 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:23.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:23.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:23.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:23.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:23.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:23.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:23.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:23.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:23.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:13:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:13:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:13:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:13:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:13:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:13:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:13:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:13:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:13:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:13:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:13:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:13:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:13:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:13:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:13:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:13:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:23.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:13:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:13:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:13:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:13:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.000 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.410200781, 2024-09-13T09:13:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:13:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:13:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:24.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:13:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:13:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:13:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:13:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:13:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:13:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:13:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:13:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:13:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:13:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:13:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:13:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:13:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:13:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:13:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:13:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:13:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:13:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:13:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:13:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.401073932999999, 2024-09-13T09:13:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:13:24.006 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:24.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:13:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:13:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:13:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:13:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:13:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:13:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:13:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:13:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:13:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:13:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:13:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:13:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:13:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:13:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:13:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:13:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:13:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:13:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:13:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:13:24.011 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.401022353000002, 2024-09-13T09:13:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:13:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.012 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:13:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:13:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:13:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:13:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:13:24.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:13:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:13:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:13:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:13:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:13:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.015 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:13:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:13:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:13:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:13:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:13:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:13:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:13:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:13:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:13:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:13:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:13:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:13:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.017 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.401006076999998, 2024-09-13T09:13:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:13:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:13:24.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:13:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:13:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:13:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:13:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:13:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:13:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:13:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:13:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:13:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:13:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:13:24.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:13:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:13:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:13:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:13:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:13:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:13:24.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:13:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:13:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:13:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:13:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.400990303, 2024-09-13T09:13:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:13:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:13:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:13:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:13:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:13:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:13:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:13:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:13:24.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:13:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:13:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:13:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:13:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:13:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:13:24.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:13:24.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:13:24.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:13:24.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:13:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:13:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:13:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:13:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:13:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:13:24.029 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.400975428000002, 2024-09-13T09:13:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:13:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.030 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:13:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:13:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:13:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:13:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:13:24.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:13:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:13:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:13:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:13:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:13:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:13:24.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:13:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:13:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:13:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:13:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:13:24.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:13:24.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:13:24.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:13:24.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:13:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:13:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.400957966, 2024-09-13T09:13:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:13:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.036 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:13:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:13:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:13:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:13:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:13:24.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:13:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:13:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:13:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:13:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:13:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:13:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:13:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:13:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:13:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:13:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:13:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:13:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:13:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:13:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:13:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:13:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:13:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.041 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.399357334000001, 2024-09-13T09:13:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:13:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:13:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.042 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:13:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:13:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:13:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:13:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:13:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:13:24.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:13:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:13:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:13:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:13:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:13:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:13:24.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:13:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:13:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:13:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:13:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:13:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:13:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:13:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:13:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:13:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.047 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.399317283000002, 2024-09-13T09:13:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:13:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:13:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:13:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:13:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:13:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:13:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:13:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:13:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:13:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:13:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:13:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:13:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:13:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:13:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:13:24.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:13:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:13:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:13:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:13:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:13:24.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:13:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:13:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:13:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.399273520000001, 2024-09-13T09:13:24.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:13:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:13:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:13:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:13:24.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:13:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:13:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:13:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:13:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:13:24.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:13:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:13:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:13:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:13:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:13:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:13:24.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:13:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:13:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:13:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:13:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:13:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:13:24.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:13:24.059 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.059 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.059 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.059 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.059 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.059 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:24.060 INFO:teuthology.orchestra.run.smithi157.stdout:4382 punch from 0x7ef1e00 to 0x7ef9a00, (0x7c00 bytes) 2024-09-13T09:13:24.061 INFO:teuthology.orchestra.run.smithi157.stdout:4384 write 0xd5e2000 thru 0xd5e94d9 (0x74da bytes) 2024-09-13T09:13:24.061 INFO:teuthology.orchestra.run.smithi157.stdout:4385 read 0x3f24c00 thru 0x3f24fde (0x3df bytes) 2024-09-13T09:13:24.061 INFO:teuthology.orchestra.run.smithi157.stdout:4387 trunc from 0xd5e9600 to 0xadf6800 2024-09-13T09:13:24.061 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:24.062 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:24.059+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:24.066 INFO:teuthology.orchestra.run.smithi157.stdout:4388 write 0x2d74200 thru 0x2d7d61e (0x941f bytes) 2024-09-13T09:13:24.069 INFO:teuthology.orchestra.run.smithi157.stdout:4389 read 0x1003600 thru 0x100c41d (0x8e1e bytes) 2024-09-13T09:13:24.071 INFO:teuthology.orchestra.run.smithi157.stdout:4393 punch from 0x1c6c200 to 0x1c6cc00, (0xa00 bytes) 2024-09-13T09:13:24.075 INFO:teuthology.orchestra.run.smithi157.stdout:4394 punch from 0x25a8a00 to 0x25ac000, (0x3600 bytes) 2024-09-13T09:13:24.079 INFO:teuthology.orchestra.run.smithi157.stdout:4395 write 0x50f800 thru 0x511dc8 (0x25c9 bytes) 2024-09-13T09:13:24.091 INFO:teuthology.orchestra.run.smithi157.stdout:4396 punch from 0xcb6400 to 0xcb7c00, (0x1800 bytes) 2024-09-13T09:13:24.097 INFO:teuthology.orchestra.run.smithi157.stdout:4398 write 0x1037400 thru 0x1044c0b (0xd80c bytes) 2024-09-13T09:13:24.123 INFO:teuthology.orchestra.run.smithi157.stdout:4401 read 0x4b6b200 thru 0x4b75d4c (0xab4d bytes) 2024-09-13T09:13:24.123 INFO:teuthology.orchestra.run.smithi157.stdout:4402 write 0x3743e00 thru 0x374cdc8 (0x8fc9 bytes) 2024-09-13T09:13:24.124 INFO:teuthology.orchestra.run.smithi157.stdout:4403 write 0x8808e00 thru 0x881519f (0xc3a0 bytes) 2024-09-13T09:13:24.124 INFO:teuthology.orchestra.run.smithi157.stdout:4404 read 0x2e83200 thru 0x2e8b074 (0x7e75 bytes) 2024-09-13T09:13:24.126 INFO:teuthology.orchestra.run.smithi157.stdout:4405 read 0x834ec00 thru 0x8358cec (0xa0ed bytes) 2024-09-13T09:13:24.161 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:24.160+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:24.165 INFO:teuthology.orchestra.run.smithi157.stdout:4406 read 0x991f200 thru 0x992d539 (0xe33a bytes) 2024-09-13T09:13:24.166 INFO:teuthology.orchestra.run.smithi157.stdout:4408 flatten 2024-09-13T09:13:24.229 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:24.229 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:24.240 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:24.282 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:24.281+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:24.363 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:24.362+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:24.423 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:24.423 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:24.423 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:24.423 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:24.423 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:24.434 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:24.464 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:24.463+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:24.565 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:24.564+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:24.576 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:24.576 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:24.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:24.576 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:24.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.576 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.576 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:24.576 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.070471054999999, 2024-09-13T09:13:24.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:13:24.576 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.577 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.577 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.577 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.577 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.577 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:13:24.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.577 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:24.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:24.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:13:24.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:13:24.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:13:24.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:13:24.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:13:24.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:13:24.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:13:24.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:13:24.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:13:24.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:13:24.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:13:24.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:13:24.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:13:24.580 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.580 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.580 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.581 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.581 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:24.581 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.070443371000003, 2024-09-13T09:13:24.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:13:24.581 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.581 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.581 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.581 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.581 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.581 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:13:24.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.581 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:24.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:24.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:24.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:13:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:13:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:13:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:13:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:13:24.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:13:24.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:13:24.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:13:24.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:13:24.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:13:24.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:13:24.585 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.585 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.585 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.585 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.585 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:24.585 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.070419029999996, 2024-09-13T09:13:24.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:13:24.585 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.585 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.585 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.586 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.586 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.586 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:13:24.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.586 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:24.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:24.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:13:24.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:13:24.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:13:24.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:13:24.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:13:24.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:13:24.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:13:24.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:13:24.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:13:24.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:13:24.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:13:24.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:13:24.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:13:24.589 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.589 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.590 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.590 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.590 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:24.590 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.070402710000003, 2024-09-13T09:13:24.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:13:24.590 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.590 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.590 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.590 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.590 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.590 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:13:24.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.591 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:24.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:24.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:13:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:13:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:13:24.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:13:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:13:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:13:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:13:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:13:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:13:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:13:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:13:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.594 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.070386294999999, 2024-09-13T09:13:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:13:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:13:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:13:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:13:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:13:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:13:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:13:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:13:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:13:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:13:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:13:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:13:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:13:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:13:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.070370187000002, 2024-09-13T09:13:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:13:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.599 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:13:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:13:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:13:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.602 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:13:24.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:13:24.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:13:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:13:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:13:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:13:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:13:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:13:24.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:13:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:13:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.070354625999997, 2024-09-13T09:13:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:13:24.604 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:13:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:13:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:13:24.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:13:24.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:13:24.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:13:24.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:13:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:13:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:13:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:13:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:13:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:13:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:13:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.070337788000003, 2024-09-13T09:13:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:13:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.609 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.610 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.610 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.610 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:13:24.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.610 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:24.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:13:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:13:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:13:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:13:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:13:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:13:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:13:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:13:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:13:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:13:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:13:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:13:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:13:24.613 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.070321796000002, 2024-09-13T09:13:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:13:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.614 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:13:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:13:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:13:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:13:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:13:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:13:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:13:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:13:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:13:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:13:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:13:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:13:24.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:13:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:13:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.070294298, 2024-09-13T09:13:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:13:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.618 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:13:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:13:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:13:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:13:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:13:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:13:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:13:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:13:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:13:24.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:13:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:13:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:13:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:13:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.070268439000003, 2024-09-13T09:13:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:13:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:13:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:13:24.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:13:24.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:13:24.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:13:24.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:13:24.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:13:24.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:13:24.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:13:24.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:13:24.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:13:24.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:13:24.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:13:24.627 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.627 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.627 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.627 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.627 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:24.627 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.070226949000002, 2024-09-13T09:13:24.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:13:24.627 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.627 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.627 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.628 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.628 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.628 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:13:24.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.628 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:24.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:24.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:13:24.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:13:24.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:13:24.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:13:24.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:13:24.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:13:24.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:13:24.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:13:24.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:13:24.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:13:24.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:13:24.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:13:24.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:13:24.631 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.631 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.632 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.632 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.632 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:24.632 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.070205711, 2024-09-13T09:13:24.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:13:24.632 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.632 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.632 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.632 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.632 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.633 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:13:24.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.633 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:24.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:24.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:24.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:13:24.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:13:24.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:13:24.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:13:24.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:13:24.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:13:24.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:13:24.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:13:24.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:13:24.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:13:24.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:13:24.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:13:24.636 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.636 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.636 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.637 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.637 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:24.637 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.070182911000003, 2024-09-13T09:13:24.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:13:24.638 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.638 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.638 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.638 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.638 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.638 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:13:24.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.638 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:24.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:24.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:13:24.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:13:24.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:13:24.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:13:24.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:24.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:13:24.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:13:24.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:13:24.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:13:24.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:13:24.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:13:24.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:13:24.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:13:24.641 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.641 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.641 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.642 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.642 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:24.642 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.070166454999999, 2024-09-13T09:13:24.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:13:24.642 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.642 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.642 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.642 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.642 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.642 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:13:24.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.643 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:24.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:24.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:24.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:13:24.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:13:24.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:13:24.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:13:24.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:13:24.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:13:24.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:13:24.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:13:24.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:13:24.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:13:24.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:13:24.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:13:24.646 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.646 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.646 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.646 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.646 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:24.646 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.069921317999999, 2024-09-13T09:13:24.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:13:24.646 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:24.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:13:24.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:13:24.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:13:24.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:13:24.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:13:24.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:13:24.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:13:24.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:24.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:13:24.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:13:24.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:13:24.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:13:24.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:13:24.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:13:24.650 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.650 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.650 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.650 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.650 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:24.651 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.068035758999997, 2024-09-13T09:13:24.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:13:24.651 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.651 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.651 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.651 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.651 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.651 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:13:24.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.651 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:24.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:24.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:13:24.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:13:24.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:13:24.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:13:24.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:13:24.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:13:24.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:13:24.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:13:24.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:13:24.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:13:24.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:13:24.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:13:24.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:13:24.655 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.655 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.655 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.655 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.655 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:24.655 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.068014392999999, 2024-09-13T09:13:24.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:13:24.655 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.655 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.656 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.656 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.656 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.656 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:13:24.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.656 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:24.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:24.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:24.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:24.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:13:24.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:13:24.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:13:24.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:13:24.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:13:24.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:13:24.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:13:24.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:13:24.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:13:24.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:13:24.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:13:24.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:13:24.659 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.659 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.660 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.660 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:24.660 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:24.660 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.067809025000003, 2024-09-13T09:13:24.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:13:24.660 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.660 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:24.660 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.660 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:24.660 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:24.661 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:13:24.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.661 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:24.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:24.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:13:24.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:13:24.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:13:24.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:13:24.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:13:24.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:13:24.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:13:24.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:13:24.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:13:24.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:13:24.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:13:24.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:13:24.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:13:24.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:13:24.664 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.664 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.664 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.664 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5272.0:1775 3.0 3:02a42fca:::rbd_data.11235782ee8d.0000000000000030:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2244608~40448] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:13:24.664 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:24.664 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.326267184000001, 2024-09-13T09:13:24.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.666843732, 2024-09-13T09:13:24.665 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:24.665 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:24.665 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:24.665 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5272", 2024-09-13T09:13:24.665 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4089902097", 2024-09-13T09:13:24.665 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1775 2024-09-13T09:13:24.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.665 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:24.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:24.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:24.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247780+0000", 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247865+0000", 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4359999999999999e-05 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:24.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247876+0000", 2024-09-13T09:13:24.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1456e-05 2024-09-13T09:13:24.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:24.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247883+0000", 2024-09-13T09:13:24.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:13:24.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247926+0000", 2024-09-13T09:13:24.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3203e-05 2024-09-13T09:13:24.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:24.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247932+0000", 2024-09-13T09:13:24.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:13:24.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909639+0000", 2024-09-13T09:13:24.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6617069099999999 2024-09-13T09:13:24.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:13:24.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909828+0000", 2024-09-13T09:13:24.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000189075 2024-09-13T09:13:24.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:24.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911676+0000", 2024-09-13T09:13:24.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001847235 2024-09-13T09:13:24.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:24.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911765+0000", 2024-09-13T09:13:24.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9530000000000005e-05 2024-09-13T09:13:24.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:13:24.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911901+0000", 2024-09-13T09:13:24.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013603899999999999 2024-09-13T09:13:24.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:13:24.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.912889+0000", 2024-09-13T09:13:24.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098807400000000003 2024-09-13T09:13:24.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:24.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914438+0000", 2024-09-13T09:13:24.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015491070000000001 2024-09-13T09:13:24.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:13:24.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914505+0000", 2024-09-13T09:13:24.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:13:24.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:24.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:24.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:24.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914625+0000", 2024-09-13T09:13:24.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120108 2024-09-13T09:13:24.671 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.671 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.671 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.671 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:24.671 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:24.672 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:24.673 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:24.672+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:24.674 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:13:24.675 INFO:teuthology.orchestra.run.smithi157.stdout:4409 punch from 0x9eccc00 to 0x9ecf600, (0x2a00 bytes) 2024-09-13T09:13:24.675 INFO:teuthology.orchestra.run.smithi157.stdout:4410 read 0xa95d800 thru 0xa966d1c (0x951d bytes) 2024-09-13T09:13:24.675 INFO:teuthology.orchestra.run.smithi157.stdout:4412 punch from 0x8741e00 to 0x874be00, (0xa000 bytes) 2024-09-13T09:13:24.675 INFO:teuthology.orchestra.run.smithi157.stdout:4415 trunc from 0xadf6800 to 0xa060600 2024-09-13T09:13:24.675 INFO:teuthology.orchestra.run.smithi157.stdout:4416 punch from 0x76df200 to 0x76e5600, (0x6400 bytes) 2024-09-13T09:13:24.675 INFO:teuthology.orchestra.run.smithi157.stdout:4419 write 0x8a2da00 thru 0x8a3c292 (0xe893 bytes) 2024-09-13T09:13:24.675 INFO:teuthology.orchestra.run.smithi157.stdout:4421 write 0x195c000 thru 0x19635da (0x75db bytes) 2024-09-13T09:13:24.676 INFO:teuthology.orchestra.run.smithi157.stdout:4423 read 0x5f94600 thru 0x5f95dab (0x17ac bytes) 2024-09-13T09:13:24.676 INFO:teuthology.orchestra.run.smithi157.stdout:4424 punch from 0x9f34000 to 0x9f39800, (0x5800 bytes) 2024-09-13T09:13:24.676 INFO:teuthology.orchestra.run.smithi157.stdout:4427 read 0x308e400 thru 0x308f745 (0x1346 bytes) 2024-09-13T09:13:24.676 INFO:teuthology.orchestra.run.smithi157.stdout:4428 write 0x5b6a400 thru 0x5b6bc13 (0x1814 bytes) 2024-09-13T09:13:24.676 INFO:teuthology.orchestra.run.smithi157.stdout:4429 trunc from 0xa060600 to 0x1f12600 2024-09-13T09:13:24.676 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-13T09:13:24.676 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:13:24.676 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-13T09:13:24.676 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:13:24.676 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:13:24.677 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-13T09:13:24.749 INFO:teuthology.orchestra.run.smithi157.stdout:4432 trunc from 0x1f12600 to 0x67d1c00 2024-09-13T09:13:24.759 INFO:teuthology.orchestra.run.smithi157.stdout:4433 punch from 0x2132200 to 0x213f000, (0xce00 bytes) 2024-09-13T09:13:24.761 INFO:teuthology.orchestra.run.smithi157.stdout:4438 write 0x5f9b000 thru 0x5f9f3f9 (0x43fa bytes) 2024-09-13T09:13:24.761 INFO:teuthology.orchestra.run.smithi157.stdout:4442 write 0x4505e00 thru 0x4514716 (0xe917 bytes) 2024-09-13T09:13:24.761 INFO:teuthology.orchestra.run.smithi157.stdout:4443 read 0x5d7dc00 thru 0x5d7f3b2 (0x17b3 bytes) 2024-09-13T09:13:24.774 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:24.773+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:24.799 INFO:teuthology.orchestra.run.smithi157.stdout:4445 write 0xca5fc00 thru 0xca612d5 (0x16d6 bytes) 2024-09-13T09:13:24.800 INFO:teuthology.orchestra.run.smithi157.stdout:4448 read 0x919aa00 thru 0x919fc96 (0x5297 bytes) 2024-09-13T09:13:24.800 INFO:teuthology.orchestra.run.smithi157.stdout:4451 punch from 0x94c9600 to 0x94d6000, (0xca00 bytes) 2024-09-13T09:13:24.801 INFO:teuthology.orchestra.run.smithi157.stdout:4456 read 0x5d86e00 thru 0x5d8e7d0 (0x79d1 bytes) 2024-09-13T09:13:24.801 INFO:teuthology.orchestra.run.smithi157.stdout:4459 read 0x20e8e00 thru 0x20ebc1e (0x2e1f bytes) 2024-09-13T09:13:24.801 INFO:teuthology.orchestra.run.smithi157.stdout:4467 punch from 0x13b0200 to 0x13b0400, (0x200 bytes) 2024-09-13T09:13:24.806 INFO:teuthology.orchestra.run.smithi157.stdout:4470 trunc from 0xca61400 to 0x843d200 2024-09-13T09:13:24.842 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:24.842 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:13:24.842 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:24.847 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:24.847 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:24.847 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:24.847 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:24.853 INFO:teuthology.orchestra.run.smithi157.stdout:4474 write 0xcb43a00 thru 0xcb52302 (0xe903 bytes) 2024-09-13T09:13:24.858 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:13:24.860 INFO:teuthology.orchestra.run.smithi157.stdout:4476 trunc from 0xcb52400 to 0x73f8c00 2024-09-13T09:13:24.888 INFO:teuthology.orchestra.run.smithi157.stdout:4480 read 0x8f6000 thru 0x8fe80c (0x880d bytes) 2024-09-13T09:13:24.889 INFO:teuthology.orchestra.run.smithi157.stdout:4482 read 0x6f07200 thru 0x6f12425 (0xb226 bytes) 2024-09-13T09:13:24.891 INFO:teuthology.orchestra.run.smithi157.stdout:4483 trunc from 0x73f8c00 to 0x95da200 2024-09-13T09:13:24.899 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:24.898+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:24.901 INFO:teuthology.orchestra.run.smithi157.stdout:4484 punch from 0x678800 to 0x67e600, (0x5e00 bytes) 2024-09-13T09:13:24.902 INFO:teuthology.orchestra.run.smithi157.stdout:4485 read 0x60b6000 thru 0x60b6868 (0x869 bytes) 2024-09-13T09:13:24.916 INFO:teuthology.orchestra.run.smithi157.stdout:4487 write 0xaf2e400 thru 0xaf31e2f (0x3a30 bytes) 2024-09-13T09:13:24.942 INFO:teuthology.orchestra.run.smithi157.stdout:4492 write 0xd7a6e00 thru 0xd7b4b2e (0xdd2f bytes) 2024-09-13T09:13:24.943 INFO:teuthology.orchestra.run.smithi157.stdout:4493 write 0xbdcbe00 thru 0xbdcc172 (0x373 bytes) 2024-09-13T09:13:24.944 INFO:teuthology.orchestra.run.smithi157.stdout:4494 clone 27 order 18 su 2048 sc 14 2024-09-13T09:13:24.976 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:24.975+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:25.013 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:25.014 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:25.014 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:25.014 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:25.014 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:25.022 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:13:25.077 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:25.076+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:25.170 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:25.170 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:25.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:25.170 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:25.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.171 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:6867 3.13 3:cd293326:::rbd_header.112326fb43d3:head [watch reconnect cookie 140383087120544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T09:13:25.171 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:13:25.171 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.639438298999998, 2024-09-13T09:13:25.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0277181360000001, 2024-09-13T09:13:25.171 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.171 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:13:25.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:13:25.171 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 6867 2024-09-13T09:13:25.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.171 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:13:25.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529068+0000", 2024-09-13T09:13:25.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529067+0000", 2024-09-13T09:13:25.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-09-13T09:13:25.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529073+0000", 2024-09-13T09:13:25.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:13:25.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529087+0000", 2024-09-13T09:13:25.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3825000000000001e-05 2024-09-13T09:13:25.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529097+0000", 2024-09-13T09:13:25.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9559999999999992e-06 2024-09-13T09:13:25.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529149+0000", 2024-09-13T09:13:25.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1668000000000001e-05 2024-09-13T09:13:25.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.529161+0000", 2024-09-13T09:13:25.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1831000000000001e-05 2024-09-13T09:13:25.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.554941+0000", 2024-09-13T09:13:25.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0257799729999999 2024-09-13T09:13:25.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.555239+0000", 2024-09-13T09:13:25.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00029829100000000001 2024-09-13T09:13:25.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.556787+0000", 2024-09-13T09:13:25.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001547617 2024-09-13T09:13:25.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.176 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.176 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20898 3.13 3:cd293326:::rbd_header.112326fb43d3:head [watch reconnect cookie 140642395970896 gen 16] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T09:13:25.176 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:13:25.176 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.634775480999998, 2024-09-13T09:13:25.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0236151950000001, 2024-09-13T09:13:25.176 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.176 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.176 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.176 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:25.177 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.177 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20898 2024-09-13T09:13:25.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.177 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:13:25.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533731+0000", 2024-09-13T09:13:25.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533729+0000", 2024-09-13T09:13:25.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-09-13T09:13:25.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533736+0000", 2024-09-13T09:13:25.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9670000000000004e-06 2024-09-13T09:13:25.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533749+0000", 2024-09-13T09:13:25.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2449000000000001e-05 2024-09-13T09:13:25.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533757+0000", 2024-09-13T09:13:25.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:13:25.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533813+0000", 2024-09-13T09:13:25.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5683000000000002e-05 2024-09-13T09:13:25.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.533823+0000", 2024-09-13T09:13:25.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0168e-05 2024-09-13T09:13:25.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.555940+0000", 2024-09-13T09:13:25.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0221170559999999 2024-09-13T09:13:25.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.181 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.181 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.556047+0000", 2024-09-13T09:13:25.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000106881 2024-09-13T09:13:25.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.181 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.181 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.557347+0000", 2024-09-13T09:13:25.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001299323 2024-09-13T09:13:25.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.181 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.182 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4447 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.182 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581680+0000", 2024-09-13T09:13:25.182 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.586827177, 2024-09-13T09:13:25.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054674652999999997, 2024-09-13T09:13:25.182 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.182 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.182 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.182 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.182 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.182 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4447 2024-09-13T09:13:25.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.182 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581680+0000", 2024-09-13T09:13:25.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581680+0000", 2024-09-13T09:13:25.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581679+0000", 2024-09-13T09:13:25.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581682+0000", 2024-09-13T09:13:25.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4830000000000002e-06 2024-09-13T09:13:25.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581688+0000", 2024-09-13T09:13:25.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0569999999999998e-06 2024-09-13T09:13:25.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581692+0000", 2024-09-13T09:13:25.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6820000000000002e-06 2024-09-13T09:13:25.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636201+0000", 2024-09-13T09:13:25.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054508845 2024-09-13T09:13:25.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636293+0000", 2024-09-13T09:13:25.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1477000000000005e-05 2024-09-13T09:13:25.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636354+0000", 2024-09-13T09:13:25.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1651999999999999e-05 2024-09-13T09:13:25.186 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.186 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.186 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.186 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4448 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 913408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.186 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581696+0000", 2024-09-13T09:13:25.186 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5868104760000001, 2024-09-13T09:13:25.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054880384999999997, 2024-09-13T09:13:25.186 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.186 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.187 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.187 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.187 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.187 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4448 2024-09-13T09:13:25.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.187 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.187 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581696+0000", 2024-09-13T09:13:25.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.187 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581696+0000", 2024-09-13T09:13:25.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581696+0000", 2024-09-13T09:13:25.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581702+0000", 2024-09-13T09:13:25.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4250000000000003e-06 2024-09-13T09:13:25.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581716+0000", 2024-09-13T09:13:25.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3196e-05 2024-09-13T09:13:25.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581721+0000", 2024-09-13T09:13:25.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0189999999999997e-06 2024-09-13T09:13:25.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636390+0000", 2024-09-13T09:13:25.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054669061999999997 2024-09-13T09:13:25.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636479+0000", 2024-09-13T09:13:25.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9188000000000001e-05 2024-09-13T09:13:25.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636577+0000", 2024-09-13T09:13:25.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7827999999999999e-05 2024-09-13T09:13:25.190 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.190 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.190 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.191 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4449 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.191 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581725+0000", 2024-09-13T09:13:25.191 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5867816369999996, 2024-09-13T09:13:25.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055074856999999998, 2024-09-13T09:13:25.191 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.191 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.191 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.191 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.191 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.191 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4449 2024-09-13T09:13:25.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.191 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581725+0000", 2024-09-13T09:13:25.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581725+0000", 2024-09-13T09:13:25.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581725+0000", 2024-09-13T09:13:25.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581727+0000", 2024-09-13T09:13:25.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5969999999999999e-06 2024-09-13T09:13:25.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581733+0000", 2024-09-13T09:13:25.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.088e-06 2024-09-13T09:13:25.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581738+0000", 2024-09-13T09:13:25.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:13:25.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636614+0000", 2024-09-13T09:13:25.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054875637999999997 2024-09-13T09:13:25.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636703+0000", 2024-09-13T09:13:25.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8838000000000006e-05 2024-09-13T09:13:25.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636800+0000", 2024-09-13T09:13:25.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7282999999999997e-05 2024-09-13T09:13:25.195 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.195 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.195 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.195 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4450 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 929792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.195 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581743+0000", 2024-09-13T09:13:25.195 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5867641199999998, 2024-09-13T09:13:25.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055241622999999997, 2024-09-13T09:13:25.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.195 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.196 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.196 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.196 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.196 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4450 2024-09-13T09:13:25.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.196 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581743+0000", 2024-09-13T09:13:25.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581743+0000", 2024-09-13T09:13:25.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581742+0000", 2024-09-13T09:13:25.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581745+0000", 2024-09-13T09:13:25.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.294e-06 2024-09-13T09:13:25.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581751+0000", 2024-09-13T09:13:25.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.516e-06 2024-09-13T09:13:25.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581758+0000", 2024-09-13T09:13:25.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7780000000000002e-06 2024-09-13T09:13:25.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636860+0000", 2024-09-13T09:13:25.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055102263999999998 2024-09-13T09:13:25.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636927+0000", 2024-09-13T09:13:25.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6427000000000006e-05 2024-09-13T09:13:25.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.636984+0000", 2024-09-13T09:13:25.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7739999999999999e-05 2024-09-13T09:13:25.199 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.199 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.199 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.200 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4451 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.200 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581763+0000", 2024-09-13T09:13:25.200 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5867437770000006, 2024-09-13T09:13:25.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055339227999999997, 2024-09-13T09:13:25.200 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.200 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.200 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.200 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.200 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.200 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4451 2024-09-13T09:13:25.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.200 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581763+0000", 2024-09-13T09:13:25.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581763+0000", 2024-09-13T09:13:25.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581763+0000", 2024-09-13T09:13:25.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581765+0000", 2024-09-13T09:13:25.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4030000000000001e-06 2024-09-13T09:13:25.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581771+0000", 2024-09-13T09:13:25.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4189999999999999e-06 2024-09-13T09:13:25.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581776+0000", 2024-09-13T09:13:25.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5739999999999997e-06 2024-09-13T09:13:25.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637002+0000", 2024-09-13T09:13:25.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055225819000000002 2024-09-13T09:13:25.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637044+0000", 2024-09-13T09:13:25.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1792999999999998e-05 2024-09-13T09:13:25.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637102+0000", 2024-09-13T09:13:25.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8644000000000003e-05 2024-09-13T09:13:25.203 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.204 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.204 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.204 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4452 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.204 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581780+0000", 2024-09-13T09:13:25.204 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5867267209999998, 2024-09-13T09:13:25.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055635109000000002, 2024-09-13T09:13:25.204 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.204 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.204 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.204 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.204 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.205 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4452 2024-09-13T09:13:25.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.205 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581780+0000", 2024-09-13T09:13:25.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581780+0000", 2024-09-13T09:13:25.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581780+0000", 2024-09-13T09:13:25.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581782+0000", 2024-09-13T09:13:25.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2790000000000001e-06 2024-09-13T09:13:25.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581788+0000", 2024-09-13T09:13:25.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9460000000000003e-06 2024-09-13T09:13:25.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.207 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581793+0000", 2024-09-13T09:13:25.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:13:25.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.207 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637139+0000", 2024-09-13T09:13:25.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055346237 2024-09-13T09:13:25.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.207 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637241+0000", 2024-09-13T09:13:25.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000101924 2024-09-13T09:13:25.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637415+0000", 2024-09-13T09:13:25.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017448400000000001 2024-09-13T09:13:25.208 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.208 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.208 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.208 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4453 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.208 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581797+0000", 2024-09-13T09:13:25.208 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5867100769999993, 2024-09-13T09:13:25.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055904331000000002, 2024-09-13T09:13:25.208 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.208 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.209 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.209 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.209 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.209 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4453 2024-09-13T09:13:25.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.209 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581797+0000", 2024-09-13T09:13:25.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581797+0000", 2024-09-13T09:13:25.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581796+0000", 2024-09-13T09:13:25.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581799+0000", 2024-09-13T09:13:25.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:13:25.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.211 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581804+0000", 2024-09-13T09:13:25.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8579999999999996e-06 2024-09-13T09:13:25.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.211 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.211 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581815+0000", 2024-09-13T09:13:25.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0219e-05 2024-09-13T09:13:25.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.211 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.211 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637446+0000", 2024-09-13T09:13:25.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055631667000000003 2024-09-13T09:13:25.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.212 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637490+0000", 2024-09-13T09:13:25.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3223000000000003e-05 2024-09-13T09:13:25.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.212 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637701+0000", 2024-09-13T09:13:25.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00021151500000000001 2024-09-13T09:13:25.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.212 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.212 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4454 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 962560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.213 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581819+0000", 2024-09-13T09:13:25.213 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5866874919999994, 2024-09-13T09:13:25.213 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056027577000000002, 2024-09-13T09:13:25.213 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.213 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.213 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.213 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.213 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.213 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4454 2024-09-13T09:13:25.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.213 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581819+0000", 2024-09-13T09:13:25.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581819+0000", 2024-09-13T09:13:25.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581819+0000", 2024-09-13T09:13:25.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581822+0000", 2024-09-13T09:13:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7020000000000001e-06 2024-09-13T09:13:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581828+0000", 2024-09-13T09:13:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.489e-06 2024-09-13T09:13:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581833+0000", 2024-09-13T09:13:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:13:25.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637742+0000", 2024-09-13T09:13:25.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055908823000000003 2024-09-13T09:13:25.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637786+0000", 2024-09-13T09:13:25.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3791999999999997e-05 2024-09-13T09:13:25.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637847+0000", 2024-09-13T09:13:25.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1371000000000006e-05 2024-09-13T09:13:25.217 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.217 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.217 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.217 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4455 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 970752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.217 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581837+0000", 2024-09-13T09:13:25.217 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5866696220000005, 2024-09-13T09:13:25.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056132979, 2024-09-13T09:13:25.217 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.217 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.217 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.217 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.218 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.218 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4455 2024-09-13T09:13:25.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.218 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581837+0000", 2024-09-13T09:13:25.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581837+0000", 2024-09-13T09:13:25.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581837+0000", 2024-09-13T09:13:25.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581839+0000", 2024-09-13T09:13:25.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.373e-06 2024-09-13T09:13:25.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581845+0000", 2024-09-13T09:13:25.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:13:25.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581850+0000", 2024-09-13T09:13:25.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5140000000000003e-06 2024-09-13T09:13:25.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637877+0000", 2024-09-13T09:13:25.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056027146999999999 2024-09-13T09:13:25.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637918+0000", 2024-09-13T09:13:25.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0911000000000002e-05 2024-09-13T09:13:25.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.637970+0000", 2024-09-13T09:13:25.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2466999999999998e-05 2024-09-13T09:13:25.221 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.221 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.221 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.221 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4456 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.221 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581860+0000", 2024-09-13T09:13:25.221 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5866464019999995, 2024-09-13T09:13:25.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056264427999999998, 2024-09-13T09:13:25.221 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.222 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.222 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.222 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.222 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.222 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4456 2024-09-13T09:13:25.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.222 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.222 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581860+0000", 2024-09-13T09:13:25.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581860+0000", 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581860+0000", 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581862+0000", 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3499999999999999e-06 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581869+0000", 2024-09-13T09:13:25.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4239999999999998e-06 2024-09-13T09:13:25.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581873+0000", 2024-09-13T09:13:25.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7110000000000003e-06 2024-09-13T09:13:25.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638000+0000", 2024-09-13T09:13:25.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056126545999999999 2024-09-13T09:13:25.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638069+0000", 2024-09-13T09:13:25.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8946e-05 2024-09-13T09:13:25.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638125+0000", 2024-09-13T09:13:25.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5826999999999999e-05 2024-09-13T09:13:25.225 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.225 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.225 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.225 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4457 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.225 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581878+0000", 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5866290470000006, 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056452293000000001, 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4457 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581878+0000", 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581878+0000", 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581877+0000", 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581879+0000", 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0899999999999999e-06 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.227 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.228 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581887+0000", 2024-09-13T09:13:25.228 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2400000000000001e-06 2024-09-13T09:13:25.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.228 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.228 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581891+0000", 2024-09-13T09:13:25.228 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8190000000000002e-06 2024-09-13T09:13:25.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.228 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.228 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638167+0000", 2024-09-13T09:13:25.228 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056276029999999998 2024-09-13T09:13:25.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.229 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.229 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638246+0000", 2024-09-13T09:13:25.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9373000000000001e-05 2024-09-13T09:13:25.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.229 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.229 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638330+0000", 2024-09-13T09:13:25.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4103000000000004e-05 2024-09-13T09:13:25.229 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.229 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.229 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.230 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4458 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.230 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581894+0000", 2024-09-13T09:13:25.230 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5866123109999997, 2024-09-13T09:13:25.230 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056678246000000002, 2024-09-13T09:13:25.230 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.230 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.230 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.230 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.230 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.230 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4458 2024-09-13T09:13:25.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.230 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.230 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.230 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581894+0000", 2024-09-13T09:13:25.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581894+0000", 2024-09-13T09:13:25.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581894+0000", 2024-09-13T09:13:25.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.232 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.232 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581896+0000", 2024-09-13T09:13:25.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.678e-06 2024-09-13T09:13:25.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.232 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.232 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581902+0000", 2024-09-13T09:13:25.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:13:25.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.232 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.232 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581906+0000", 2024-09-13T09:13:25.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4800000000000003e-06 2024-09-13T09:13:25.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.233 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.233 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638383+0000", 2024-09-13T09:13:25.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05647655 2024-09-13T09:13:25.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.233 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.233 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638431+0000", 2024-09-13T09:13:25.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7953e-05 2024-09-13T09:13:25.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.233 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.233 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638573+0000", 2024-09-13T09:13:25.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000141784 2024-09-13T09:13:25.234 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.234 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.234 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.234 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4459 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1003520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.234 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581920+0000", 2024-09-13T09:13:25.234 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5865872640000003, 2024-09-13T09:13:25.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057037290999999997, 2024-09-13T09:13:25.234 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.234 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.234 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.234 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.234 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4459 2024-09-13T09:13:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581920+0000", 2024-09-13T09:13:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581920+0000", 2024-09-13T09:13:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.236 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581919+0000", 2024-09-13T09:13:25.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.236 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581922+0000", 2024-09-13T09:13:25.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6809999999999998e-06 2024-09-13T09:13:25.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.236 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581929+0000", 2024-09-13T09:13:25.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2010000000000002e-06 2024-09-13T09:13:25.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581934+0000", 2024-09-13T09:13:25.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1599999999999997e-06 2024-09-13T09:13:25.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638615+0000", 2024-09-13T09:13:25.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056681242999999999 2024-09-13T09:13:25.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638738+0000", 2024-09-13T09:13:25.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012294000000000001 2024-09-13T09:13:25.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638957+0000", 2024-09-13T09:13:25.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000218476 2024-09-13T09:13:25.238 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.238 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.238 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.238 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4460 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.238 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581939+0000", 2024-09-13T09:13:25.239 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5865679040000007, 2024-09-13T09:13:25.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057296491999999997, 2024-09-13T09:13:25.239 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.239 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.239 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.239 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.239 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.239 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4460 2024-09-13T09:13:25.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.239 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.240 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581939+0000", 2024-09-13T09:13:25.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.240 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581939+0000", 2024-09-13T09:13:25.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.240 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581938+0000", 2024-09-13T09:13:25.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.241 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581946+0000", 2024-09-13T09:13:25.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2049999999999996e-06 2024-09-13T09:13:25.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.241 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581952+0000", 2024-09-13T09:13:25.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7240000000000003e-06 2024-09-13T09:13:25.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581958+0000", 2024-09-13T09:13:25.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1889999999999998e-06 2024-09-13T09:13:25.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.638987+0000", 2024-09-13T09:13:25.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057029006 2024-09-13T09:13:25.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639030+0000", 2024-09-13T09:13:25.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3574e-05 2024-09-13T09:13:25.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.243 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639235+0000", 2024-09-13T09:13:25.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00020521 2024-09-13T09:13:25.243 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.243 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.243 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.243 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4461 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.243 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581979+0000", 2024-09-13T09:13:25.243 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5865276440000002, 2024-09-13T09:13:25.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057454631999999999, 2024-09-13T09:13:25.244 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.244 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.244 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.244 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.244 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.244 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4461 2024-09-13T09:13:25.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.244 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581979+0000", 2024-09-13T09:13:25.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.245 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581979+0000", 2024-09-13T09:13:25.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.245 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581979+0000", 2024-09-13T09:13:25.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.245 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581982+0000", 2024-09-13T09:13:25.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1379999999999999e-06 2024-09-13T09:13:25.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581989+0000", 2024-09-13T09:13:25.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0299999999999996e-06 2024-09-13T09:13:25.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581994+0000", 2024-09-13T09:13:25.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1660000000000002e-06 2024-09-13T09:13:25.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639261+0000", 2024-09-13T09:13:25.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057267063999999999 2024-09-13T09:13:25.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639303+0000", 2024-09-13T09:13:25.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1655e-05 2024-09-13T09:13:25.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639434+0000", 2024-09-13T09:13:25.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000131128 2024-09-13T09:13:25.247 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.247 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.248 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.248 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4462 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.248 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.581999+0000", 2024-09-13T09:13:25.248 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5865081239999999, 2024-09-13T09:13:25.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057741334999999998, 2024-09-13T09:13:25.248 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.248 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.248 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.248 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.248 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.248 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4462 2024-09-13T09:13:25.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.249 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581999+0000", 2024-09-13T09:13:25.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581999+0000", 2024-09-13T09:13:25.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.250 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.581998+0000", 2024-09-13T09:13:25.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.250 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.250 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582001+0000", 2024-09-13T09:13:25.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:13:25.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.250 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.250 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582007+0000", 2024-09-13T09:13:25.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0889999999999996e-06 2024-09-13T09:13:25.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582011+0000", 2024-09-13T09:13:25.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:13:25.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639465+0000", 2024-09-13T09:13:25.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057453652000000001 2024-09-13T09:13:25.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639551+0000", 2024-09-13T09:13:25.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.632e-05 2024-09-13T09:13:25.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639740+0000", 2024-09-13T09:13:25.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018866299999999999 2024-09-13T09:13:25.252 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.252 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.252 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.252 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4463 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.252 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.582017+0000", 2024-09-13T09:13:25.252 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5864900229999996, 2024-09-13T09:13:25.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057956270999999997, 2024-09-13T09:13:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4463 2024-09-13T09:13:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582017+0000", 2024-09-13T09:13:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582017+0000", 2024-09-13T09:13:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582016+0000", 2024-09-13T09:13:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582019+0000", 2024-09-13T09:13:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0999999999999998e-06 2024-09-13T09:13:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582024+0000", 2024-09-13T09:13:25.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9510000000000003e-06 2024-09-13T09:13:25.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582029+0000", 2024-09-13T09:13:25.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5449999999999997e-06 2024-09-13T09:13:25.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639769+0000", 2024-09-13T09:13:25.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057739921 2024-09-13T09:13:25.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639822+0000", 2024-09-13T09:13:25.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2710000000000002e-05 2024-09-13T09:13:25.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.639973+0000", 2024-09-13T09:13:25.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000151373 2024-09-13T09:13:25.256 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.256 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.257 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.257 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5347.0:4464 3.8 3:1577f378:::rbd_data.11239759ab94.0000000000000009:1b [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:25.257 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:15.582033+0000", 2024-09-13T09:13:25.257 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.5864738749999994, 2024-09-13T09:13:25.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.058326451000000001, 2024-09-13T09:13:25.257 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:25.257 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:25.257 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:25.257 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5347", 2024-09-13T09:13:25.257 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:25.257 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 4464 2024-09-13T09:13:25.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.258 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:25.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:25.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582033+0000", 2024-09-13T09:13:25.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:25.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582033+0000", 2024-09-13T09:13:25.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:25.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:25.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582033+0000", 2024-09-13T09:13:25.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:25.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582035+0000", 2024-09-13T09:13:25.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:13:25.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:25.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582041+0000", 2024-09-13T09:13:25.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8490000000000002e-06 2024-09-13T09:13:25.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.582045+0000", 2024-09-13T09:13:25.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4700000000000004e-06 2024-09-13T09:13:25.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:25.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.640014+0000", 2024-09-13T09:13:25.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.057969100000000003 2024-09-13T09:13:25.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:25.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.640106+0000", 2024-09-13T09:13:25.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2074e-05 2024-09-13T09:13:25.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:25.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:25.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:25.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:15.640359+0000", 2024-09-13T09:13:25.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00025301799999999998 2024-09-13T09:13:25.261 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.261 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.261 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.261 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:25.261 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:25.261 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:25.263 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:25.263 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:25.262+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:25.364 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:25.363+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:25.366 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:25.366 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:25.366 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:25.366 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:25.375 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:25.465 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:25.464+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:25.467 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:13:25.480 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:25.480 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:25.480 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:25.480 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:25.480 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:25.489 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:25.566 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:25.565+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:25.595 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:25.595 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:25.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:25.595 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:25.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.595 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.595 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:25.595 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.071643180999999, 2024-09-13T09:13:25.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:13:25.595 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.596 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.596 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:13:25.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.596 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:25.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:25.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:25.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:13:25.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:13:25.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:13:25.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:13:25.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:13:25.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:13:25.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:13:25.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:13:25.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:13:25.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:13:25.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:13:25.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:13:25.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:13:25.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:13:25.600 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.600 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.601 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.601 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.601 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:25.601 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.071594124999997, 2024-09-13T09:13:25.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:13:25.601 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.601 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.601 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.601 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.601 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.602 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:13:25.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.602 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:25.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:25.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:25.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:13:25.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:13:25.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:13:25.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:13:25.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:13:25.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:13:25.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:13:25.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:13:25.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:13:25.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:13:25.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:13:25.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:13:25.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:13:25.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:13:25.606 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.606 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.606 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.606 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.606 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:25.606 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.071554050000003, 2024-09-13T09:13:25.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:13:25.606 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.606 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.606 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.606 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.607 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.607 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:13:25.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.607 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:25.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:25.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:25.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:13:25.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:13:25.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:13:25.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:13:25.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:13:25.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:13:25.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:13:25.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:13:25.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:13:25.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:13:25.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:13:25.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:13:25.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:13:25.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:13:25.611 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.611 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.611 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.612 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.612 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:25.612 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.071524234999998, 2024-09-13T09:13:25.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:13:25.612 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.612 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.612 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:13:25.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.612 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:25.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:25.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:25.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:13:25.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:13:25.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:13:25.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:25.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:13:25.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:13:25.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:13:25.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:13:25.616 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.616 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.616 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.616 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.616 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:25.616 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.071494573999999, 2024-09-13T09:13:25.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:13:25.617 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.617 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.617 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.617 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.617 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.617 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:13:25.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.617 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:25.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:25.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:25.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:13:25.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:13:25.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:13:25.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:13:25.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:13:25.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:13:25.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:13:25.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:13:25.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:13:25.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:13:25.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:13:25.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:13:25.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:13:25.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:13:25.621 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.621 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.621 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.622 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.622 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:25.622 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.071465367000002, 2024-09-13T09:13:25.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:13:25.622 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.622 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.622 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.622 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.622 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.622 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:13:25.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.622 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:25.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:25.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:25.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:13:25.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:13:25.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:13:25.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:13:25.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:13:25.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:13:25.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:13:25.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:13:25.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:13:25.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:13:25.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:13:25.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:13:25.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:13:25.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:13:25.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.627 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.627 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.627 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:25.627 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.071409185999997, 2024-09-13T09:13:25.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:13:25.627 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.627 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.628 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:13:25.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.628 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:25.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:25.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:25.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:13:25.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:13:25.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:13:25.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:13:25.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:13:25.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:13:25.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:13:25.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:13:25.632 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.632 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.632 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.632 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.632 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:25.632 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.071355724999997, 2024-09-13T09:13:25.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:13:25.632 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.632 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.633 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.633 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.633 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.633 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:13:25.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.633 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:25.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:25.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:13:25.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:13:25.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:13:25.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:13:25.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:13:25.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:13:25.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:13:25.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:13:25.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:13:25.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:13:25.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:13:25.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:13:25.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:13:25.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:13:25.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:13:25.637 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.637 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.637 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.637 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.637 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:25.637 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.071298894999998, 2024-09-13T09:13:25.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:13:25.637 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.637 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.637 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:13:25.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.638 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:25.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:25.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:13:25.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:13:25.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:13:25.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:13:25.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:13:25.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:13:25.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:13:25.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:13:25.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:13:25.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:13:25.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:13:25.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:13:25.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:13:25.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:13:25.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:13:25.642 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.642 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.642 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.642 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.642 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:25.642 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.071247868, 2024-09-13T09:13:25.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:13:25.642 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.642 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.642 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.642 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.642 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:13:25.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:13:25.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:13:25.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:13:25.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:13:25.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:13:25.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:13:25.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:13:25.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:13:25.646 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.646 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.646 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.646 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.071206678000003, 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:25.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:25.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:13:25.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:13:25.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:13:25.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:13:25.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:13:25.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:13:25.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:13:25.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:13:25.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:13:25.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:13:25.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:13:25.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:13:25.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:13:25.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:13:25.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.652 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.652 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.652 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.652 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:25.652 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.071164562, 2024-09-13T09:13:25.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:13:25.652 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.652 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.652 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.652 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.653 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:13:25.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.653 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:25.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:25.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:25.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:25.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:13:25.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:13:25.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:13:25.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:13:25.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:13:25.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:13:25.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:13:25.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:13:25.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:13:25.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:13:25.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:13:25.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:13:25.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:13:25.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:13:25.656 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.656 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.657 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.657 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.657 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:25.657 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.071120682999997, 2024-09-13T09:13:25.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:13:25.657 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.657 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.657 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.657 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.657 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.658 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:13:25.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.658 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:25.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:25.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:25.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:25.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:13:25.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:13:25.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:13:25.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:13:25.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:13:25.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:13:25.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:13:25.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:13:25.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:13:25.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:13:25.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:13:25.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:13:25.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:13:25.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:13:25.662 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.662 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.662 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.662 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.662 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:25.662 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.071074766000002, 2024-09-13T09:13:25.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:13:25.663 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.663 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.663 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:13:25.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.663 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:25.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:25.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:13:25.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:13:25.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:13:25.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:13:25.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:13:25.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:13:25.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:13:25.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:13:25.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:13:25.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:13:25.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:13:25.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:13:25.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:13:25.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:13:25.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:13:25.668 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.668 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.669 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.669 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.669 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:25.669 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.071024995000002, 2024-09-13T09:13:25.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:13:25.669 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.669 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.669 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.670 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:13:25.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.670 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:25.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:25.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:25.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:25.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:13:25.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:13:25.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:13:25.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:13:25.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:13:25.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:13:25.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:13:25.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:13:25.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:13:25.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:13:25.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:13:25.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:13:25.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:13:25.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:13:25.673 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.070979590999997, 2024-09-13T09:13:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:13:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:13:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:13:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:13:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:13:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:13:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:13:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:13:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:13:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:13:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:13:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:13:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:13:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:13:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:13:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:13:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.070928952999999, 2024-09-13T09:13:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:13:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:13:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:13:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:13:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:13:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:13:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:13:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:13:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:13:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:13:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:13:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:13:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:13:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:13:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:13:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:13:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:13:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.070873110000001, 2024-09-13T09:13:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:13:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:13:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:13:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:13:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:13:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:13:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:13:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:13:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:13:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:13:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:13:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:13:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:13:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:13:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:13:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:13:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.070816188000002, 2024-09-13T09:13:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:13:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:13:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:13:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:13:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:13:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:13:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:13:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:13:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:13:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:13:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:13:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:13:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:13:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.070762625, 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:13:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:13:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:13:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:13:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:13:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:13:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:13:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:13:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:13:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:25.699 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:25.701 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:25.701 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:25.699+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:25.749 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:13:25.800 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:25.800+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:25.845 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:25.845 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:25.845 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:25.845 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:25.853 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:25.870 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:25.871 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:25.887 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:25.887 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:25.903 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:13:25.903 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:13:25.919 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:25.920 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:25.936 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:25.936 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:25.938 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:25.938+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:25.952 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:25.953 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:25.997 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:25.997 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:26.001 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:26.001+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:26.006 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:26.103 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:26.102+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:26.104 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-13T09:13:26.160 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:26.161 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:26.169 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:26.204 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:26.204+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:26.305 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:26.304+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:26.312 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:26.312 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:26.321 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:26.406 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:26.405+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:26.473 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:26.473 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:26.477 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-09-13T09:13:26.482 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:26.522 INFO:teuthology.orchestra.run.smithi157.stdout:4496 write 0xd3e200 thru 0xd48379 (0xa17a bytes) 2024-09-13T09:13:26.526 INFO:teuthology.orchestra.run.smithi157.stdout:4497 write 0xc595200 thru 0xc599303 (0x4104 bytes) 2024-09-13T09:13:26.529 INFO:teuthology.orchestra.run.smithi157.stdout:4498 read 0x67caa00 thru 0x67d6ef0 (0xc4f1 bytes) 2024-09-13T09:13:26.531 INFO:teuthology.orchestra.run.smithi157.stdout:4499 read 0x55400 thru 0x5a32c (0x4f2d bytes) 2024-09-13T09:13:26.532 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:26.531+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:26.544 INFO:teuthology.orchestra.run.smithi157.stdout:4500 trunc from 0xd7b4c00 to 0x2283600 2024-09-13T09:13:26.608 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:26.607+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:26.610 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:26.610 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:26.622 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:26.709 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:26.708+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:26.719 INFO:teuthology.orchestra.run.smithi157.stdout:4501 punch from 0x303600 to 0x30ca00, (0x9400 bytes) 2024-09-13T09:13:26.775 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:26.775 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:26.784 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:26.828 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:26.827+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:26.839 INFO:teuthology.orchestra.run.smithi157.stdout:4504 write 0x7a41c00 thru 0x7a42422 (0x823 bytes) 2024-09-13T09:13:26.839 INFO:teuthology.orchestra.run.smithi157.stdout:4505 write 0x5ac9a00 thru 0x5ad2bf6 (0x91f7 bytes) 2024-09-13T09:13:26.839 INFO:teuthology.orchestra.run.smithi157.stdout:4510 punch from 0x82a400 to 0x838200, (0xde00 bytes) 2024-09-13T09:13:26.853 INFO:teuthology.orchestra.run.smithi157.stdout:4513 punch from 0x433ec00 to 0x4341a00, (0x2e00 bytes) 2024-09-13T09:13:26.853 INFO:teuthology.orchestra.run.smithi157.stdout:4514 write 0x41e9600 thru 0x41eb442 (0x1e43 bytes) 2024-09-13T09:13:26.853 INFO:teuthology.orchestra.run.smithi157.stdout:4515 punch from 0x335fc00 to 0x3360800, (0xc00 bytes) 2024-09-13T09:13:26.853 INFO:teuthology.orchestra.run.smithi157.stdout:4517 punch from 0x2026a00 to 0x202ee00, (0x8400 bytes) 2024-09-13T09:13:26.911 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:26.910+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:26.926 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:26.926 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:26.935 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:26.937 INFO:teuthology.orchestra.run.smithi157.stdout:4518 write 0xe076a00 thru 0xe07f233 (0x8834 bytes) 2024-09-13T09:13:26.937 INFO:teuthology.orchestra.run.smithi157.stdout:4519 write 0xa83c600 thru 0xa84577c (0x917d bytes) 2024-09-13T09:13:26.950 INFO:teuthology.orchestra.run.smithi157.stdout:4523 trunc from 0xe07f400 to 0x314f000 2024-09-13T09:13:27.012 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:27.011+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:27.046 INFO:teuthology.orchestra.run.smithi157.stdout:4525 write 0xda19800 thru 0xda261a1 (0xc9a2 bytes) 2024-09-13T09:13:27.047 INFO:teuthology.orchestra.run.smithi157.stdout:4527 read 0x942fa00 thru 0x943a2d5 (0xa8d6 bytes) 2024-09-13T09:13:27.047 INFO:teuthology.orchestra.run.smithi157.stdout:4529 write 0x4bf8c00 thru 0x4c01098 (0x8499 bytes) 2024-09-13T09:13:27.047 INFO:teuthology.orchestra.run.smithi157.stdout:4531 read 0xa249400 thru 0xa25058c (0x718d bytes) 2024-09-13T09:13:27.047 INFO:teuthology.orchestra.run.smithi157.stdout:4532 write 0x78cfa00 thru 0x78de9f3 (0xeff4 bytes) 2024-09-13T09:13:27.047 INFO:teuthology.orchestra.run.smithi157.stdout:4533 punch from 0x219aa00 to 0x21a9a00, (0xf000 bytes) 2024-09-13T09:13:27.048 INFO:teuthology.orchestra.run.smithi157.stdout:4534 write 0x55ac000 thru 0x55b057f (0x4580 bytes) 2024-09-13T09:13:27.052 INFO:teuthology.orchestra.run.smithi157.stdout:4536 trunc from 0xda26200 to 0x98b1c00 2024-09-13T09:13:27.109 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:27.109 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:27.113 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:27.112+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:27.119 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:27.121 INFO:teuthology.orchestra.run.smithi157.stdout:4537 write 0x8d09c00 thru 0x8d0ea8f (0x4e90 bytes) 2024-09-13T09:13:27.121 INFO:teuthology.orchestra.run.smithi157.stdout:4538 punch from 0x6066800 to 0x6067200, (0xa00 bytes) 2024-09-13T09:13:27.122 INFO:teuthology.orchestra.run.smithi157.stdout:4539 write 0x1a39400 thru 0x1a3b176 (0x1d77 bytes) 2024-09-13T09:13:27.123 INFO:teuthology.orchestra.run.smithi157.stdout:4540 read 0xe48a00 thru 0xe4df70 (0x5571 bytes) 2024-09-13T09:13:27.182 INFO:teuthology.orchestra.run.smithi157.stdout:4541 write 0xe434e00 thru 0xe43c6ab (0x78ac bytes) 2024-09-13T09:13:27.189 INFO:teuthology.orchestra.run.smithi157.stdout:4542 punch from 0x146da00 to 0x147a400, (0xca00 bytes) 2024-09-13T09:13:27.214 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:27.213+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:27.264 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:27.264 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:27.274 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:27.306 INFO:teuthology.orchestra.run.smithi157.stdout:4543 write 0xeca6200 thru 0xecb00b4 (0x9eb5 bytes) 2024-09-13T09:13:27.306 INFO:teuthology.orchestra.run.smithi157.stdout:4546 punch from 0xb344e00 to 0xb346400, (0x1600 bytes) 2024-09-13T09:13:27.307 INFO:teuthology.orchestra.run.smithi157.stdout:4548 read 0xc470800 thru 0xc478413 (0x7c14 bytes) 2024-09-13T09:13:27.307 INFO:teuthology.orchestra.run.smithi157.stdout:4551 punch from 0x2b45a00 to 0x2b4f400, (0x9a00 bytes) 2024-09-13T09:13:27.307 INFO:teuthology.orchestra.run.smithi157.stdout:4553 write 0x2f4fe00 thru 0x2f59cea (0x9eeb bytes) 2024-09-13T09:13:27.307 INFO:teuthology.orchestra.run.smithi157.stdout:4556 punch from 0x8614e00 to 0x861b800, (0x6a00 bytes) 2024-09-13T09:13:27.308 INFO:teuthology.orchestra.run.smithi157.stdout:4557 read 0x37fc000 thru 0x37fea39 (0x2a3a bytes) 2024-09-13T09:13:27.308 INFO:teuthology.orchestra.run.smithi157.stdout:4559 clone 28 order 24 su 1048576 sc 11 2024-09-13T09:13:27.317 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:27.316+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:27.415 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:27.415+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:27.441 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:27.441 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:27.449 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:27.516 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:27.516+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:27.553 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone27 from 0xecb0200 (overlap 0x2283600) to 0xa5a800 2024-09-13T09:13:27.563 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:27.563 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:27.578 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:13:27.618 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:27.617+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:27.719 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:27.718+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:27.758 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:27.758 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:27.768 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:13:27.820 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:27.819+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:27.908 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:27.908 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:27.917 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:13:27.921 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:27.919+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:27.966 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:13:28.022 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:28.021+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:28.025 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:28.026 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:28.035 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:28.088 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:28.089 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:28.105 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:28.105 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:28.121 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:13:28.122 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:13:28.123 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:28.121+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:28.139 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:28.139 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:28.144 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:28.144 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:28.144 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:28.144 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:28.153 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:28.158 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:28.158 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:28.178 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:28.178 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:28.230 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:28.228+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:28.240 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-13T09:13:28.267 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:28.267 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:28.268 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:28.268 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:28.276 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:28.326 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:28.324+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:28.418 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:28.418 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:28.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:28.418 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:28.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.418 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.418 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:28.419 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893972398000002, 2024-09-13T09:13:28.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:13:28.419 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.419 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.419 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:13:28.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.420 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:28.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:28.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:28.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:13:28.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:13:28.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:13:28.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:13:28.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:13:28.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:13:28.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:13:28.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:13:28.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:13:28.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:13:28.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:13:28.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:13:28.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:13:28.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:13:28.424 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.424 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.424 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.424 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.424 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:28.424 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893923342000001, 2024-09-13T09:13:28.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:13:28.425 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.425 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.425 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.428 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:13:28.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.429 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:28.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:28.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:28.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:13:28.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:13:28.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:13:28.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:13:28.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:13:28.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:13:28.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:13:28.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:13:28.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:13:28.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:13:28.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:13:28.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:13:28.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:13:28.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:13:28.432 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.433 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.433 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.433 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.433 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:28.433 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893883267, 2024-09-13T09:13:28.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:13:28.433 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.434 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.434 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:13:28.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.434 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:28.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:13:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:13:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:13:28.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:13:28.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:13:28.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:13:28.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:13:28.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:13:28.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:13:28.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:13:28.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:13:28.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:13:28.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:13:28.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:13:28.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.438 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.438 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.439 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:28.439 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893853452000002, 2024-09-13T09:13:28.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:13:28.439 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.439 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.439 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:13:28.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.439 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:28.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:28.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:28.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:13:28.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:13:28.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:13:28.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:28.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:13:28.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:13:28.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:13:28.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:13:28.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:13:28.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:13:28.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:13:28.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:13:28.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:13:28.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:13:28.443 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.443 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.443 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.444 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.444 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:28.444 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893823791000003, 2024-09-13T09:13:28.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:13:28.444 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.444 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.444 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.445 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.445 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.445 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:13:28.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.445 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:28.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:28.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:28.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:13:28.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:13:28.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:13:28.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:13:28.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:13:28.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:13:28.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:13:28.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:13:28.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:13:28.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:13:28.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:13:28.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:13:28.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:13:28.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:13:28.449 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.449 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.449 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.449 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.449 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:28.450 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893794583999998, 2024-09-13T09:13:28.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:13:28.450 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.450 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.450 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.450 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.450 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.450 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:13:28.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.451 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:28.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:28.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:28.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:13:28.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:13:28.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:13:28.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:13:28.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:13:28.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:13:28.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:13:28.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:13:28.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:13:28.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:13:28.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:13:28.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:13:28.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:13:28.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:13:28.455 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.455 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.455 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.456 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.456 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:28.456 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893738403, 2024-09-13T09:13:28.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:13:28.456 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.456 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.456 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.456 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.456 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.457 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:13:28.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.457 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:28.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:28.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:28.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:13:28.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:13:28.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:13:28.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:13:28.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:13:28.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:13:28.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:13:28.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:13:28.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:13:28.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:13:28.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:13:28.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:13:28.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:13:28.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:13:28.461 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.461 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.461 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.462 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.462 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:28.462 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893684942, 2024-09-13T09:13:28.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:13:28.462 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.462 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.462 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:13:28.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.463 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:28.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:28.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:13:28.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:13:28.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:13:28.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:13:28.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:13:28.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:13:28.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:13:28.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:13:28.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:13:28.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:13:28.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:13:28.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:13:28.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:13:28.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:13:28.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:13:28.467 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.467 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.467 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.468 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.468 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:28.468 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893628112000002, 2024-09-13T09:13:28.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:13:28.468 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.468 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.468 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.468 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.468 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.468 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:13:28.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.468 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:28.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:28.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:13:28.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:13:28.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:13:28.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:13:28.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:13:28.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:13:28.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:13:28.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:13:28.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:13:28.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:13:28.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:13:28.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:13:28.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:13:28.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:13:28.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:13:28.472 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.473 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.473 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.473 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.473 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:28.473 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893577084999997, 2024-09-13T09:13:28.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:13:28.473 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.473 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.473 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.473 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.474 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:13:28.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.474 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:28.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:28.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:13:28.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:13:28.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:13:28.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:13:28.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:13:28.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:13:28.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:13:28.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:13:28.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:13:28.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:13:28.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:13:28.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:13:28.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:13:28.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:13:28.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:13:28.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.478 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.478 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.478 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:28.478 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893535894999999, 2024-09-13T09:13:28.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:13:28.478 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.479 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.479 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:13:28.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.479 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:28.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:28.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:28.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:13:28.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:13:28.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:13:28.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:13:28.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:13:28.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:13:28.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:13:28.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:13:28.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:13:28.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:13:28.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:13:28.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:13:28.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:13:28.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:13:28.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.483 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.483 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.483 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:28.484 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893493779000003, 2024-09-13T09:13:28.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:13:28.484 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.484 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.484 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:13:28.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.484 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:28.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:28.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:28.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:28.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:13:28.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:13:28.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:13:28.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:13:28.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:13:28.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:13:28.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:13:28.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:13:28.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:13:28.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:13:28.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:13:28.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:13:28.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:13:28.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:13:28.489 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.489 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.489 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.489 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.489 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:28.489 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.8934499, 2024-09-13T09:13:28.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:13:28.490 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.490 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.490 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:13:28.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.490 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:28.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:28.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:28.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:28.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:13:28.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:13:28.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:13:28.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:13:28.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:13:28.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:13:28.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:13:28.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:13:28.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:13:28.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:13:28.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:13:28.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:13:28.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:13:28.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:13:28.494 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.494 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.494 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.495 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.495 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:28.495 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893403982999999, 2024-09-13T09:13:28.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:13:28.495 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.495 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.495 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:13:28.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.496 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:28.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:28.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:13:28.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:13:28.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:13:28.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:13:28.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:13:28.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:13:28.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:13:28.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:13:28.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:13:28.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:13:28.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:13:28.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:13:28.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:13:28.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:13:28.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:13:28.502 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.502 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.502 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.503 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.503 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:28.503 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893354211999998, 2024-09-13T09:13:28.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:13:28.503 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.503 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.503 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.504 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:13:28.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.504 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:28.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:28.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:28.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:28.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:13:28.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:13:28.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:13:28.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:13:28.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:13:28.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:13:28.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:13:28.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:13:28.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:13:28.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:13:28.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:13:28.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:13:28.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:13:28.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:13:28.511 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.512 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.512 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.512 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:28.512 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893308808, 2024-09-13T09:13:28.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:13:28.512 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.512 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.513 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:13:28.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.513 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:28.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:28.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:28.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:28.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:13:28.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:13:28.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:13:28.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:13:28.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:13:28.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:13:28.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:13:28.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:13:28.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:13:28.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:13:28.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:13:28.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:13:28.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:13:28.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:13:28.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.517 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.517 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.517 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:28.518 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893258170000003, 2024-09-13T09:13:28.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:13:28.518 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.518 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.518 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:13:28.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.518 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:28.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:28.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:13:28.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:13:28.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:13:28.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:13:28.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:13:28.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:13:28.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:13:28.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:13:28.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:13:28.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:13:28.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:13:28.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:13:28.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:13:28.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893202326999997, 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.523 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.524 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.524 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:13:28.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.524 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:28.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:28.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:28.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:13:28.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:13:28.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:13:28.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:13:28.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:13:28.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:13:28.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:13:28.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:13:28.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:13:28.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:13:28.528 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.528 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.528 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.528 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.528 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:28.528 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893145404999999, 2024-09-13T09:13:28.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:13:28.529 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.529 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.529 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:13:28.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.529 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:28.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:28.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:28.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:28.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:13:28.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:13:28.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:13:28.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:13:28.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:13:28.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:13:28.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:13:28.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:13:28.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:13:28.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:13:28.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:13:28.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:13:28.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:13:28.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:13:28.533 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.533 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.533 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.534 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:28.534 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:28.534 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.893091841999997, 2024-09-13T09:13:28.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:13:28.534 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.534 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.534 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.534 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:28.534 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:28.534 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:13:28.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.534 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:28.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:28.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:28.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:13:28.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:13:28.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:13:28.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:13:28.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:13:28.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:13:28.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:13:28.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:13:28.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:28.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:13:28.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:13:28.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:13:28.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:13:28.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:13:28.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:13:28.539 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.539 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.539 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.540 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.540 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:28.540 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:13:28.542 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:28.542 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:28.540+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:28.642 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:13:28.640+0000 7f5855240640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31831) UID: 0 2024-09-13T09:13:28.720 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:28.720 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:28.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.720 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5482.0:11214 3.3 3:c3ea1dc0:::rbd_data.1123c35b2348.00000000000000f4:head [read 454656~12288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:13:28.720 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:28.719280+0000", 2024-09-13T09:13:28.720 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.7084999999999996e-05, 2024-09-13T09:13:28.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013534899999999999, 2024-09-13T09:13:28.720 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.721 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:28.721 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.721 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5482", 2024-09-13T09:13:28.721 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/860857134", 2024-09-13T09:13:28.721 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 11214 2024-09-13T09:13:28.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.721 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:28.719280+0000", 2024-09-13T09:13:28.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:28.719280+0000", 2024-09-13T09:13:28.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:28.719280+0000", 2024-09-13T09:13:28.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:13:28.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:28.719284+0000", 2024-09-13T09:13:28.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3050000000000003e-06 2024-09-13T09:13:28.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:28.719296+0000", 2024-09-13T09:13:28.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2472e-05 2024-09-13T09:13:28.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:28.719304+0000", 2024-09-13T09:13:28.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4449999999999999e-06 2024-09-13T09:13:28.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:28.719349+0000", 2024-09-13T09:13:28.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5534000000000001e-05 2024-09-13T09:13:28.723 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.724 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.724 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.724 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.724 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-13T09:13:28.724 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 1 2024-09-13T09:13:28.724 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:28.732 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:28.742 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:28.740+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:28.843 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:28.841+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:28.873 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:28.873 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:28.873 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:28.874 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:28.874 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:28.884 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:28.944 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:28.943+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:28.989 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2185 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.46082462, 2024-09-13T09:13:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063348994000000006, 2024-09-13T09:13:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2185 2024-09-13T09:13:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525765+0000", 2024-09-13T09:13:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2390000000000001e-06 2024-09-13T09:13:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525772+0000", 2024-09-13T09:13:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:13:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525777+0000", 2024-09-13T09:13:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0799999999999996e-06 2024-09-13T09:13:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589020+0000", 2024-09-13T09:13:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063242501000000007 2024-09-13T09:13:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589063+0000", 2024-09-13T09:13:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2673999999999998e-05 2024-09-13T09:13:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589112+0000", 2024-09-13T09:13:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9286000000000002e-05 2024-09-13T09:13:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2187 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460805073, 2024-09-13T09:13:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063500102000000003, 2024-09-13T09:13:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2187 2024-09-13T09:13:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525784+0000", 2024-09-13T09:13:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1629999999999999e-06 2024-09-13T09:13:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525792+0000", 2024-09-13T09:13:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525796+0000", 2024-09-13T09:13:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7389999999999999e-06 2024-09-13T09:13:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589132+0000", 2024-09-13T09:13:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063336119999999996 2024-09-13T09:13:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589187+0000", 2024-09-13T09:13:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4126000000000003e-05 2024-09-13T09:13:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589283+0000", 2024-09-13T09:13:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6032999999999993e-05 2024-09-13T09:13:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2188 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460787027, 2024-09-13T09:13:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063724617999999997, 2024-09-13T09:13:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2188 2024-09-13T09:13:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525802+0000", 2024-09-13T09:13:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0439999999999998e-06 2024-09-13T09:13:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525809+0000", 2024-09-13T09:13:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.238e-06 2024-09-13T09:13:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525814+0000", 2024-09-13T09:13:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-13T09:13:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589327+0000", 2024-09-13T09:13:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063513257000000004 2024-09-13T09:13:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589418+0000", 2024-09-13T09:13:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0796000000000003e-05 2024-09-13T09:13:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589525+0000", 2024-09-13T09:13:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010693500000000001 2024-09-13T09:13:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2190 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460768941, 2024-09-13T09:13:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063863981, 2024-09-13T09:13:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2190 2024-09-13T09:13:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525818+0000", 2024-09-13T09:13:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525820+0000", 2024-09-13T09:13:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:13:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525827+0000", 2024-09-13T09:13:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0589999999999999e-06 2024-09-13T09:13:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525831+0000", 2024-09-13T09:13:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7870000000000001e-06 2024-09-13T09:13:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589562+0000", 2024-09-13T09:13:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063731038000000004 2024-09-13T09:13:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589612+0000", 2024-09-13T09:13:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0087999999999999e-05 2024-09-13T09:13:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589683+0000", 2024-09-13T09:13:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0121000000000002e-05 2024-09-13T09:13:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2192 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460751981, 2024-09-13T09:13:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064050299000000005, 2024-09-13T09:13:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2192 2024-09-13T09:13:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525835+0000", 2024-09-13T09:13:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525837+0000", 2024-09-13T09:13:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:13:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525845+0000", 2024-09-13T09:13:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3660000000000003e-06 2024-09-13T09:13:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525850+0000", 2024-09-13T09:13:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0030000000000002e-06 2024-09-13T09:13:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589720+0000", 2024-09-13T09:13:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063870176000000001 2024-09-13T09:13:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589801+0000", 2024-09-13T09:13:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0889000000000004e-05 2024-09-13T09:13:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589886+0000", 2024-09-13T09:13:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5183000000000001e-05 2024-09-13T09:13:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460733877999999, 2024-09-13T09:13:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:13:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:13:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:13:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:13:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:13:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:13:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:13:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:13:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:13:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:13:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:13:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:13:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:13:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:13:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:13:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.46071618, 2024-09-13T09:13:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:13:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:13:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:13:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:13:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:13:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:13:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:13:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:13:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:13:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:13:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:13:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:13:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:13:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:13:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460695565, 2024-09-13T09:13:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:13:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:13:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:13:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:13:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:13:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:13:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:13:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:13:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:13:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:13:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:13:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:13:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:13:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:13:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460676609, 2024-09-13T09:13:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:13:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:13:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:13:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:13:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:13:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:13:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:13:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:13:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:13:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:13:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:13:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:13:29.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:13:29.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:13:29.029 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.029 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.029 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.029 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.029 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:29.029 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460659351, 2024-09-13T09:13:29.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:13:29.029 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.030 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.030 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.030 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.030 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.030 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:13:29.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.030 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:29.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:29.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:29.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:13:29.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:13:29.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:13:29.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:13:29.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:13:29.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:13:29.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:13:29.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:13:29.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:13:29.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:13:29.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:13:29.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:13:29.033 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.033 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.033 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.034 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.034 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:29.034 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460639337, 2024-09-13T09:13:29.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:13:29.034 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.034 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.034 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.034 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.034 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.034 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:13:29.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.034 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:29.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:29.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:29.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:13:29.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:13:29.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:13:29.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:13:29.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:13:29.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:13:29.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:13:29.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:13:29.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:13:29.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:13:29.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460613755000001, 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:13:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:13:29.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:13:29.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:13:29.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:13:29.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:13:29.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:13:29.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:13:29.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:13:29.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:13:29.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:13:29.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:13:29.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:13:29.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:13:29.045 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.045 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.045 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.045 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.046 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:29.046 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460594471, 2024-09-13T09:13:29.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:13:29.046 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.046 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.046 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:13:29.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.047 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:29.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:29.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:29.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:13:29.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:13:29.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:13:29.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:13:29.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:13:29.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:13:29.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:13:29.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:13:29.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:13:29.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:13:29.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:13:29.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:13:29.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.050 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.050 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.050 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:29.050 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460577215000001, 2024-09-13T09:13:29.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:13:29.050 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.051 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.051 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.051 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.051 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.051 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:13:29.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.051 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:29.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:29.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:29.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:13:29.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:13:29.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:13:29.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:13:29.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:13:29.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:13:29.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:13:29.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:13:29.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:13:29.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:13:29.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:13:29.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:13:29.055 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.055 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.055 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.055 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.055 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:29.055 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460557735, 2024-09-13T09:13:29.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:13:29.055 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.055 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.056 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.056 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.056 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.056 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:13:29.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.056 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:29.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:29.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:13:29.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:13:29.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:13:29.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:13:29.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:13:29.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:13:29.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:13:29.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:13:29.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:13:29.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:13:29.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:13:29.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:13:29.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:13:29.059 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.059 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.059 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.060 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.060 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:29.060 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460540806000001, 2024-09-13T09:13:29.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:13:29.060 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.060 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.060 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.060 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.060 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.060 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:13:29.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.060 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:29.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:29.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:13:29.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:13:29.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:13:29.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:13:29.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:13:29.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:13:29.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:13:29.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:13:29.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:13:29.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:13:29.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:13:29.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:13:29.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:13:29.064 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.064 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.064 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.064 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.064 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:29.064 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460521656999999, 2024-09-13T09:13:29.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:13:29.064 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.064 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.065 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.065 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.065 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.065 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:13:29.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.065 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:29.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:29.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:29.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:13:29.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:13:29.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:13:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:13:29.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:13:29.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:13:29.068 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.068 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.068 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.068 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.068 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:29.068 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460503517999999, 2024-09-13T09:13:29.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:13:29.069 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.069 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.069 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:13:29.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.069 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:29.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:29.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:29.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:13:29.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:13:29.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:13:29.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:13:29.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:13:29.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:13:29.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:13:29.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:13:29.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:13:29.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:13:29.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:13:29.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:13:29.072 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.072 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.072 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460093023000001, 2024-09-13T09:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:13:29.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:13:29.077 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.077 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.077 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.077 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:29.077 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:29.077 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.460070121999999, 2024-09-13T09:13:29.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:13:29.077 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.077 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.077 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.078 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:29.078 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:29.078 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:13:29.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.078 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:29.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:29.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:29.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:13:29.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:13:29.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:13:29.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:13:29.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:13:29.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:13:29.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:13:29.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:13:29.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:13:29.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:13:29.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:13:29.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:13:29.081 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.081 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.081 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.081 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.081 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.081 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:29.083 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:29.083 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:29.082+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:29.183 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:29.183+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:29.246 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:29.247 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:29.247 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:29.247 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:29.255 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:29.297 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:29.296+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:29.386 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:29.384+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:29.398 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:29.399 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:29.407 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:29.416 INFO:teuthology.orchestra.run.smithi157.stdout:4560 read 0xe4df600 thru 0xe4e4a73 (0x5474 bytes) 2024-09-13T09:13:29.425 INFO:teuthology.orchestra.run.smithi157.stdout:4561 trunc from 0xecb0200 to 0x87cd400 2024-09-13T09:13:29.485 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-09-13T09:13:29.487 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:29.485+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:29.562 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:29.562 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:29.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9035 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2342912~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.814178677000001, 2024-09-13T09:13:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070293606999999994, 2024-09-13T09:13:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9035 2024-09-13T09:13:29.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.564 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:29.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:29.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:29.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745602+0000", 2024-09-13T09:13:29.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-06 2024-09-13T09:13:29.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745607+0000", 2024-09-13T09:13:29.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0770000000000001e-06 2024-09-13T09:13:29.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745609+0000", 2024-09-13T09:13:29.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3499999999999999e-06 2024-09-13T09:13:29.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815626+0000", 2024-09-13T09:13:29.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070017398999999994 2024-09-13T09:13:29.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815672+0000", 2024-09-13T09:13:29.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5275999999999998e-05 2024-09-13T09:13:29.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815895+0000", 2024-09-13T09:13:29.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022328300000000001 2024-09-13T09:13:29.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.567 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.567 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.568 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:29.568 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.814166133000001, 2024-09-13T09:13:29.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:13:29.568 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.568 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.568 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.568 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.568 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.568 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:13:29.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.568 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:29.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:29.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:29.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:13:29.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:29.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:13:29.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:13:29.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:13:29.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:13:29.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:13:29.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:13:29.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:13:29.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:13:29.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:13:29.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:13:29.572 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.572 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.572 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.572 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.572 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:29.572 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.814157455, 2024-09-13T09:13:29.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:13:29.572 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.572 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.573 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.573 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.573 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.573 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:13:29.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.573 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:29.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:29.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:29.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:29.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:13:29.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:13:29.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:13:29.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:13:29.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:13:29.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:13:29.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:13:29.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:13:29.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:13:29.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:13:29.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:13:29.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:13:29.576 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.576 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.576 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.577 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.577 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:29.577 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.814148468999999, 2024-09-13T09:13:29.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:13:29.577 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.577 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.577 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.577 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.577 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.577 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:13:29.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.578 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:29.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:29.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:13:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:13:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:13:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:13:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:13:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:13:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:13:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:13:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:13:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:13:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:13:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:13:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.814140497, 2024-09-13T09:13:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:13:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:13:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:13:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:13:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:13:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:13:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:13:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:13:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:13:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:13:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:13:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:13:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:13:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:13:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.812265663000002, 2024-09-13T09:13:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:13:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:13:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:13:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:13:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:13:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:13:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:13:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:13:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:13:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:13:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:13:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:13:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:13:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:13:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.812255002000001, 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:29.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:29.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:29.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:29.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:13:29.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:13:29.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:13:29.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:13:29.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:13:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:13:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:13:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:13:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:13:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:13:29.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:13:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:13:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:29.595 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.812233026000001, 2024-09-13T09:13:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:13:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:13:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:29.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:13:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:13:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:13:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:13:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:13:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:13:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:13:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:13:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:13:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:13:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:13:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:13:29.599 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.812219524, 2024-09-13T09:13:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:13:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.600 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:13:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:13:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:13:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:13:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:13:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:13:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:13:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:13:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:13:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9055 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 6930432~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.810226744000001, 2024-09-13T09:13:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070177949000000003, 2024-09-13T09:13:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9055 2024-09-13T09:13:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:29.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749553+0000", 2024-09-13T09:13:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749554+0000", 2024-09-13T09:13:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.11e-06 2024-09-13T09:13:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749557+0000", 2024-09-13T09:13:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7949999999999999e-06 2024-09-13T09:13:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749559+0000", 2024-09-13T09:13:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:13:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819633+0000", 2024-09-13T09:13:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073995 2024-09-13T09:13:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819679+0000", 2024-09-13T09:13:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6006e-05 2024-09-13T09:13:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819731+0000", 2024-09-13T09:13:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1934000000000001e-05 2024-09-13T09:13:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9056 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 7159808~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.810218695, 2024-09-13T09:13:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070283126000000001, 2024-09-13T09:13:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9056 2024-09-13T09:13:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749562+0000", 2024-09-13T09:13:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1340000000000001e-06 2024-09-13T09:13:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749565+0000", 2024-09-13T09:13:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8760000000000001e-06 2024-09-13T09:13:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749567+0000", 2024-09-13T09:13:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2340000000000001e-06 2024-09-13T09:13:29.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819750+0000", 2024-09-13T09:13:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070182537000000003 2024-09-13T09:13:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819792+0000", 2024-09-13T09:13:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2141999999999997e-05 2024-09-13T09:13:29.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819844+0000", 2024-09-13T09:13:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2428e-05 2024-09-13T09:13:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.613 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.810210426000001, 2024-09-13T09:13:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:13:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:13:29.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:13:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:13:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:13:29.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:13:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:13:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:13:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:13:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:13:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:13:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:13:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:13:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:13:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.618 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.810186104, 2024-09-13T09:13:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:13:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:13:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:29.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:13:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:13:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:13:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:13:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:13:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:13:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:13:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:13:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:13:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:13:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:13:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:13:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.623 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.810175130000001, 2024-09-13T09:13:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:13:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:13:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:13:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:13:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:13:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:13:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:13:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:13:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:13:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:13:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:13:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:13:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:13:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:13:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:13:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9063 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1589248~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.808271487999999, 2024-09-13T09:13:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070391924999999994, 2024-09-13T09:13:29.628 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9063 2024-09-13T09:13:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751509+0000", 2024-09-13T09:13:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.142e-06 2024-09-13T09:13:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751513+0000", 2024-09-13T09:13:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6669999999999998e-06 2024-09-13T09:13:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751520+0000", 2024-09-13T09:13:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5910000000000001e-06 2024-09-13T09:13:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821593+0000", 2024-09-13T09:13:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073519000000001 2024-09-13T09:13:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821648+0000", 2024-09-13T09:13:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:13:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821900+0000", 2024-09-13T09:13:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025263999999999999 2024-09-13T09:13:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.808257505, 2024-09-13T09:13:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:13:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.633 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:13:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:29.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:13:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:13:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:13:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:13:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:13:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:13:29.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:13:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:13:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:13:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:13:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:13:29.637 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.808248108000001, 2024-09-13T09:13:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:13:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.638 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:13:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:13:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:13:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:13:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:13:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:13:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:13:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:13:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:13:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:13:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:13:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:13:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:13:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9069 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2965504~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:29.642 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.806288728999998, 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070184959000000005, 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9069 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753491+0000", 2024-09-13T09:13:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753492+0000", 2024-09-13T09:13:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.403e-06 2024-09-13T09:13:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753495+0000", 2024-09-13T09:13:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9390000000000002e-06 2024-09-13T09:13:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753504+0000", 2024-09-13T09:13:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5129999999999996e-06 2024-09-13T09:13:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823390+0000", 2024-09-13T09:13:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.069886632000000004 2024-09-13T09:13:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823445+0000", 2024-09-13T09:13:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4880999999999999e-05 2024-09-13T09:13:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823676+0000", 2024-09-13T09:13:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000230812 2024-09-13T09:13:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.646 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9070 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3194880~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.806273638, 2024-09-13T09:13:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070280588000000005, 2024-09-13T09:13:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9070 2024-09-13T09:13:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753506+0000", 2024-09-13T09:13:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:29.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753507+0000", 2024-09-13T09:13:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753510+0000", 2024-09-13T09:13:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0460000000000001e-06 2024-09-13T09:13:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753513+0000", 2024-09-13T09:13:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:13:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823689+0000", 2024-09-13T09:13:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070176083 2024-09-13T09:13:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823734+0000", 2024-09-13T09:13:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5056999999999999e-05 2024-09-13T09:13:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823787+0000", 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3121000000000002e-05 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9071 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.806265091, 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070386377999999999, 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:29.651 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9071 2024-09-13T09:13:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:29.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753516+0000", 2024-09-13T09:13:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1519999999999999e-06 2024-09-13T09:13:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753519+0000", 2024-09-13T09:13:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6559999999999998e-06 2024-09-13T09:13:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753522+0000", 2024-09-13T09:13:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:13:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823798+0000", 2024-09-13T09:13:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070275907999999998 2024-09-13T09:13:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823845+0000", 2024-09-13T09:13:29.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7593000000000003e-05 2024-09-13T09:13:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823901+0000", 2024-09-13T09:13:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5934999999999999e-05 2024-09-13T09:13:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:29.655 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:29.655 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:29.657 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:29.656+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:29.657 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:29.757 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:29.757+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:29.771 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:29.771 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:29.771 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:29.772 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:29.782 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:29.854 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-13T09:13:29.854 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:13:29.854 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 1, 2, 3, 4, 5] 2024-09-13T09:13:29.854 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T09:13:29.854 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:13:29.858 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:29.857+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:29.893 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:29.894 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:29.895 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T09:13:29.895 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.2 2024-09-13T09:13:29.895 INFO:tasks.thrashosds.thrasher:remote for osd 2 is ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T09:13:29.895 DEBUG:teuthology.orchestra.run.smithi044:> type ceph-objectstore-tool 2024-09-13T09:13:29.902 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:29.958 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T09:13:29.958 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op list-pgs 2024-09-13T09:13:29.960 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:29.959+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:30.060 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:30.060 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:30.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:30.060 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.529523957000002, 2024-09-13T09:13:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:13:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:13:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:13:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:13:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:13:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:13:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:13:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:13:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:13:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:13:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:13:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:13:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:13:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:13:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:13:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:13:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:13:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:13:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:13:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:13:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:13:30.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:13:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:13:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.529505921999998, 2024-09-13T09:13:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:13:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:13:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:13:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:13:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:13:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:13:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:13:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:13:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:13:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:13:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:13:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:13:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:13:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:13:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:13:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:13:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:13:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:13:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:13:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:13:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:13:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:13:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.529489251999998, 2024-09-13T09:13:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:13:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:13:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:13:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:13:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:13:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:13:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:13:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:13:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:13:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:13:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:13:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:13:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:13:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:13:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:13:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:13:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:13:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:13:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:13:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:13:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:13:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:13:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.529471620000002, 2024-09-13T09:13:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:13:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:13:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:13:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:13:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:13:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:13:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:13:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:13:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:13:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:13:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:13:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:13:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:13:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:13:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:13:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:13:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:13:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:13:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:13:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:13:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:13:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:13:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.529425314999997, 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:13:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:13:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:13:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:13:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:13:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:13:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:13:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:13:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:13:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.529400013999997, 2024-09-13T09:13:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:13:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:13:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:13:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:13:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:13:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:13:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:13:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:13:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:13:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:13:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:13:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:13:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:13:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:13:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:13:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:13:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:13:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:13:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:13:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:13:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:13:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:13:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:13:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.529381784999998, 2024-09-13T09:13:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:13:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:13:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:13:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:13:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:13:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:13:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:13:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:13:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:13:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:13:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:13:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:13:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:13:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:13:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:13:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:13:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:13:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:13:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:13:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:13:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:13:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:13:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.529334466000002, 2024-09-13T09:13:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:13:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:13:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:13:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:13:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:13:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:13:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:13:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:13:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:13:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:13:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:13:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:13:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:13:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:13:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:13:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:13:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:13:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:13:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:13:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:13:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:13:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:13:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.528820228000001, 2024-09-13T09:13:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:13:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:13:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:13:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:13:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:13:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:13:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:13:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:13:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:13:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:13:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:13:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:13:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:13:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:13:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:13:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:13:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:13:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:13:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:13:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:13:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:13:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:13:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:13:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:13:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.528785698, 2024-09-13T09:13:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:13:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:13:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:13:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:13:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.119 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:30.058+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:30.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:13:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:13:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:13:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:13:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:13:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:13:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:13:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:13:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:13:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:13:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:13:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:13:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:13:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:13:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:13:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:13:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:13:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:13:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.528767791, 2024-09-13T09:13:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:13:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:13:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:13:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:13:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:13:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:13:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:13:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:13:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:13:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:13:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:13:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:13:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:13:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:13:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:13:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:13:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:13:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:13:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:13:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:13:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:13:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:13:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.519640942999999, 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:13:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:13:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:13:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:13:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:13:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:13:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:13:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:13:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:13:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:13:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:13:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:13:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:13:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:13:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.519589363000001, 2024-09-13T09:13:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:13:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:13:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:13:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:13:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:13:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:13:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:13:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:13:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:13:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:13:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:13:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:13:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:13:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:13:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:13:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:13:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:13:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.519573086999998, 2024-09-13T09:13:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:13:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:13:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:13:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:13:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:13:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:13:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:13:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:13:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:13:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:13:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:13:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:13:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:13:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:13:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:13:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:13:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:13:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:13:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:13:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:13:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:13:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:13:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:13:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.519557313, 2024-09-13T09:13:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:13:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:13:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:13:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:13:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:13:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:13:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:13:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:13:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:13:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:13:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:13:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:13:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:13:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:13:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:13:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:13:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:13:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:13:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:13:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:13:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:13:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:13:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:13:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.519542438000002, 2024-09-13T09:13:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:13:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:13:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:13:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:13:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:13:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:13:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:13:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:13:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:13:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:13:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:13:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:13:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:13:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:13:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:13:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:13:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:13:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:13:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:13:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:13:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:13:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:13:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.519524976, 2024-09-13T09:13:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:13:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:13:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:13:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:13:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:13:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:13:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:13:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:13:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:13:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:13:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:13:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:13:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:13:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:13:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:13:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:13:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:13:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:13:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.517924344000001, 2024-09-13T09:13:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:13:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:13:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:13:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:13:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:13:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:13:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:13:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:13:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:13:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:13:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:13:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.517884293000002, 2024-09-13T09:13:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:13:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:13:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:13:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:13:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:13:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:13:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:13:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:13:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:13:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:13:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:13:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:13:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:13:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:13:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:13:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:13:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:13:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:13:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:13:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:13:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:13:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:13:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:13:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.517840530000001, 2024-09-13T09:13:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:13:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:13:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:13:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:13:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:13:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:13:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:13:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:13:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:13:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:13:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:13:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:13:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:13:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:13:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:13:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:13:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:13:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:13:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:13:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:13:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:13:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:13:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:30.180 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:30.181 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:30.182 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:30.181+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:30.189 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:13:30.284 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:30.283+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:30.290 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:30.290 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:30.290 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:30.290 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:30.300 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:30.354 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:30.354 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:30.373 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:30.373 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:30.384 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:30.383+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:30.412 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:30.412 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:30.422 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:30.486 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:30.485+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:30.536 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:30.536 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:30.545 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:30.586 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:30.584+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:30.659 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:30.659 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:30.660 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:30.660 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:30.668 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:30.687 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:30.685+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:30.771 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:30.771 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:30.771 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:30.771 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:30.771 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:30.780 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:30.828 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:30.826+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:30.884 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360812889999998, 2024-09-13T09:13:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:13:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:13:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:13:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:13:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:13:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:13:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:13:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:13:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:13:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:13:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:13:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:13:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:13:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:13:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:13:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:13:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360763833999997, 2024-09-13T09:13:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:13:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:13:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:13:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:13:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:13:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:13:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:13:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:13:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:13:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:13:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:13:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:13:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:13:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:13:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:13:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:13:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360723759000003, 2024-09-13T09:13:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:13:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:13:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:13:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:13:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:13:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:13:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:13:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:13:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:13:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:13:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:13:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:13:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:13:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:13:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:13:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:13:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360693943999998, 2024-09-13T09:13:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:13:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:13:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:13:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:13:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:13:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:13:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:13:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:13:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:13:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:13:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:13:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:13:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:13:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:13:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:13:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360664282999998, 2024-09-13T09:13:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:13:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:13:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:13:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:13:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:13:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:13:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:13:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:13:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:13:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:13:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:13:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:13:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360635076000001, 2024-09-13T09:13:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:13:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:13:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:13:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:13:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:13:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:13:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:13:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:13:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:13:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:13:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:13:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:13:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:13:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:13:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:13:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:13:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360578895000003, 2024-09-13T09:13:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:13:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:13:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:13:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:13:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:13:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:13:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360525434000003, 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:13:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:13:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:13:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:13:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:13:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:13:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:13:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:13:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:13:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:13:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:13:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:13:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:13:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:13:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:13:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:13:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:13:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.927 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.927 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:30.927 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360468603999998, 2024-09-13T09:13:30.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:13:30.927 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.927 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.927 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:13:30.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.927 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:13:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:13:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:13:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:13:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:13:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:13:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:13:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:13:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:13:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:13:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:13:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:13:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:13:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:13:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:13:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360417577, 2024-09-13T09:13:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:13:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:13:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:13:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:13:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:13:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:13:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:13:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:13:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:13:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:13:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:13:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:13:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:13:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:13:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:13:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:13:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:13:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360376387000002, 2024-09-13T09:13:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:13:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:13:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:13:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:13:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:13:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:13:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:13:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:13:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:13:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360334270999999, 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:13:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:13:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:13:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:13:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:13:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:13:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:13:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:13:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360290392000003, 2024-09-13T09:13:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:13:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:13:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:13:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:13:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:13:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:13:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:13:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:13:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360244475000002, 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:13:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:13:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:13:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:13:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360194704000001, 2024-09-13T09:13:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:13:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:13:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:13:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:13:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:13:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:13:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:13:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:13:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:13:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:13:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:13:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:13:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:13:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:13:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360149300000003, 2024-09-13T09:13:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:13:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:13:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:13:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:13:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:13:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:13:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:13:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:13:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:13:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:13:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:13:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:13:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:13:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:13:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:13:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:13:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.962 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360098661999999, 2024-09-13T09:13:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:13:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:13:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:13:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:13:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:13:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:13:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:13:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:13:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:13:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:13:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:13:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:13:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:13:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:13:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:13:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:13:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:13:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.360042819, 2024-09-13T09:13:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:13:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.967 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:13:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:30.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:13:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:13:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:13:30.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:13:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:13:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:13:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:13:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:13:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:13:30.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:13:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:13:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:13:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:13:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:13:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.359985897000001, 2024-09-13T09:13:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:13:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:13:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.972 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:30.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:13:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:13:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:13:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:13:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:13:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:13:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:13:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:13:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:13:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:13:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:13:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:13:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:13:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:13:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:30.976 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.359932334, 2024-09-13T09:13:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:13:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:13:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:13:30.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:13:30.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:13:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:13:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:13:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:13:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:13:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:13:30.980 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:30.981 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:30.982 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:30.983 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:30.981+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:31.083 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:31.081+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:31.126 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:31.126 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:31.126 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:31.126 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:31.134 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:31.163 INFO:teuthology.orchestra.run.smithi044.stdout:3.9 2024-09-13T09:13:31.163 INFO:teuthology.orchestra.run.smithi044.stdout:3.7 2024-09-13T09:13:31.163 INFO:teuthology.orchestra.run.smithi044.stdout:3.8 2024-09-13T09:13:31.163 INFO:teuthology.orchestra.run.smithi044.stdout:3.4 2024-09-13T09:13:31.163 INFO:teuthology.orchestra.run.smithi044.stdout:3.1e 2024-09-13T09:13:31.163 INFO:teuthology.orchestra.run.smithi044.stdout:3.1b 2024-09-13T09:13:31.163 INFO:teuthology.orchestra.run.smithi044.stdout:3.5 2024-09-13T09:13:31.163 INFO:teuthology.orchestra.run.smithi044.stdout:3.18 2024-09-13T09:13:31.163 INFO:teuthology.orchestra.run.smithi044.stdout:2.3 2024-09-13T09:13:31.163 INFO:teuthology.orchestra.run.smithi044.stdout:3.d 2024-09-13T09:13:31.164 INFO:teuthology.orchestra.run.smithi044.stdout:2.1 2024-09-13T09:13:31.185 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:31.183+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:31.236 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:31.236 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:31.236 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:31.236 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:31.236 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:31.244 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:31.287 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:31.287+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:31.348 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:31.348 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:31.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:31.348 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:31.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.348 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2185 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.349 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:31.349 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820551267999999, 2024-09-13T09:13:31.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063348994000000006, 2024-09-13T09:13:31.349 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.349 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.349 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.349 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.349 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.349 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2185 2024-09-13T09:13:31.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.349 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:31.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:31.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:31.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525765+0000", 2024-09-13T09:13:31.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2390000000000001e-06 2024-09-13T09:13:31.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525772+0000", 2024-09-13T09:13:31.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:13:31.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525777+0000", 2024-09-13T09:13:31.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0799999999999996e-06 2024-09-13T09:13:31.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589020+0000", 2024-09-13T09:13:31.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063242501000000007 2024-09-13T09:13:31.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589063+0000", 2024-09-13T09:13:31.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2673999999999998e-05 2024-09-13T09:13:31.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589112+0000", 2024-09-13T09:13:31.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9286000000000002e-05 2024-09-13T09:13:31.353 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.353 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.353 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.353 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2187 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.353 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:31.353 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820531721, 2024-09-13T09:13:31.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063500102000000003, 2024-09-13T09:13:31.353 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.353 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.353 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.353 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.354 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.354 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2187 2024-09-13T09:13:31.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.354 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:31.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:31.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:31.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525784+0000", 2024-09-13T09:13:31.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1629999999999999e-06 2024-09-13T09:13:31.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525792+0000", 2024-09-13T09:13:31.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:31.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525796+0000", 2024-09-13T09:13:31.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7389999999999999e-06 2024-09-13T09:13:31.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589132+0000", 2024-09-13T09:13:31.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063336119999999996 2024-09-13T09:13:31.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589187+0000", 2024-09-13T09:13:31.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4126000000000003e-05 2024-09-13T09:13:31.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589283+0000", 2024-09-13T09:13:31.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6032999999999993e-05 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2188 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820513675000001, 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063724617999999997, 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2188 2024-09-13T09:13:31.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.359 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:31.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:31.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:31.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525802+0000", 2024-09-13T09:13:31.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0439999999999998e-06 2024-09-13T09:13:31.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525809+0000", 2024-09-13T09:13:31.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.238e-06 2024-09-13T09:13:31.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525814+0000", 2024-09-13T09:13:31.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-13T09:13:31.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589327+0000", 2024-09-13T09:13:31.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063513257000000004 2024-09-13T09:13:31.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589418+0000", 2024-09-13T09:13:31.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0796000000000003e-05 2024-09-13T09:13:31.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589525+0000", 2024-09-13T09:13:31.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010693500000000001 2024-09-13T09:13:31.362 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.362 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.362 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.362 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2190 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.362 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:31.362 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820495589, 2024-09-13T09:13:31.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063863981, 2024-09-13T09:13:31.363 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.363 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.363 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.363 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.363 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.363 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2190 2024-09-13T09:13:31.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.363 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:31.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:31.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525818+0000", 2024-09-13T09:13:31.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525820+0000", 2024-09-13T09:13:31.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:13:31.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525827+0000", 2024-09-13T09:13:31.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0589999999999999e-06 2024-09-13T09:13:31.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525831+0000", 2024-09-13T09:13:31.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7870000000000001e-06 2024-09-13T09:13:31.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589562+0000", 2024-09-13T09:13:31.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063731038000000004 2024-09-13T09:13:31.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589612+0000", 2024-09-13T09:13:31.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0087999999999999e-05 2024-09-13T09:13:31.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589683+0000", 2024-09-13T09:13:31.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0121000000000002e-05 2024-09-13T09:13:31.367 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.367 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.367 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.367 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2192 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.367 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:31.367 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820478629, 2024-09-13T09:13:31.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064050299000000005, 2024-09-13T09:13:31.367 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.367 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.368 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2192 2024-09-13T09:13:31.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.368 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:31.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:31.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525835+0000", 2024-09-13T09:13:31.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525837+0000", 2024-09-13T09:13:31.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:13:31.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525845+0000", 2024-09-13T09:13:31.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3660000000000003e-06 2024-09-13T09:13:31.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525850+0000", 2024-09-13T09:13:31.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0030000000000002e-06 2024-09-13T09:13:31.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589720+0000", 2024-09-13T09:13:31.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063870176000000001 2024-09-13T09:13:31.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589801+0000", 2024-09-13T09:13:31.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0889000000000004e-05 2024-09-13T09:13:31.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589886+0000", 2024-09-13T09:13:31.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5183000000000001e-05 2024-09-13T09:13:31.372 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.372 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.372 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.372 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.372 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:31.372 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820460526, 2024-09-13T09:13:31.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:13:31.372 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.373 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.373 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:13:31.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.373 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:31.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:31.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:13:31.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:13:31.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:13:31.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:13:31.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:13:31.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:13:31.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:13:31.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:13:31.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:13:31.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:13:31.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:13:31.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:13:31.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:13:31.377 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.377 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.377 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.377 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.377 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:31.377 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820442828000001, 2024-09-13T09:13:31.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:13:31.378 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.378 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.378 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:13:31.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.378 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:31.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:31.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:31.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:13:31.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:13:31.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:13:31.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:13:31.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:13:31.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:13:31.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:13:31.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:13:31.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:13:31.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:13:31.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:13:31.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:13:31.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.382 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.382 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.382 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:31.382 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820422213000001, 2024-09-13T09:13:31.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:13:31.382 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.383 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.383 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:13:31.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.383 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:31.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:31.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:31.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:13:31.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:13:31.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:13:31.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:13:31.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:13:31.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:13:31.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:13:31.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:13:31.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:13:31.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:13:31.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:13:31.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:13:31.387 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.387 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.387 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.387 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.387 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:31.387 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820403257000001, 2024-09-13T09:13:31.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:13:31.387 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.387 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.388 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:13:31.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.388 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:31.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:31.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:13:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:13:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:13:31.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:13:31.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:13:31.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:13:31.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:13:31.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:13:31.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:13:31.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:13:31.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:13:31.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:13:31.391 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.391 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.391 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.392 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.392 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:31.392 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820385998999999, 2024-09-13T09:13:31.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:13:31.392 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.392 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.392 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.392 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.392 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.392 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:13:31.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.392 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:31.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:31.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:31.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:13:31.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:13:31.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:13:31.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:13:31.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:13:31.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:13:31.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:13:31.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:13:31.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:13:31.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:13:31.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:13:31.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:13:31.396 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.396 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.396 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.396 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.396 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:31.396 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820365985, 2024-09-13T09:13:31.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:13:31.397 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.397 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.397 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.397 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.397 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.397 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:13:31.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.397 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:31.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:31.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:31.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:13:31.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:13:31.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:13:31.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:13:31.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:13:31.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:13:31.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:13:31.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:13:31.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:13:31.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:13:31.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:13:31.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:13:31.401 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.401 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.401 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.401 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.401 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:31.401 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820340402999999, 2024-09-13T09:13:31.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:13:31.401 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.402 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.402 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.402 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.402 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.402 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:13:31.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.402 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:31.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:31.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:13:31.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:13:31.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:13:31.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:13:31.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:13:31.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:13:31.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:13:31.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:13:31.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:13:31.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:13:31.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:13:31.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:13:31.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:13:31.405 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.406 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.406 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.406 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.406 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:31.406 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820321119000001, 2024-09-13T09:13:31.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:13:31.406 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.406 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.406 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:13:31.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.407 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:31.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:31.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:31.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:13:31.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:13:31.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:13:31.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:13:31.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:13:31.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:13:31.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:13:31.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:13:31.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:13:31.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:13:31.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:13:31.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:13:31.410 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.410 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.410 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.410 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.410 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:31.410 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820303862999999, 2024-09-13T09:13:31.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:13:31.411 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.411 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.411 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.411 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.411 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.411 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:13:31.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.411 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:31.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:31.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:31.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:13:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:13:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:13:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:13:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:13:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:13:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:13:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:13:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:13:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:13:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:13:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:13:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820284383000001, 2024-09-13T09:13:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:13:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:13:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:13:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:13:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:13:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:13:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:13:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:13:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:13:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:13:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:13:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:13:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:13:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:13:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:13:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820267454, 2024-09-13T09:13:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:13:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:13:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:13:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:13:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:13:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:13:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:13:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:13:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:13:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:13:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:13:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:13:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:13:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:13:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:13:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820248305, 2024-09-13T09:13:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:13:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:13:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:13:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:13:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:13:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:13:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:13:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:13:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:13:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:13:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:13:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:13:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:13:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:13:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.820230166, 2024-09-13T09:13:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:13:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:13:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:13:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:13:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:13:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:13:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:13:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:13:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:13:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:13:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:13:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:13:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:13:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:13:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.819819670999999, 2024-09-13T09:13:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:13:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:13:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:13:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:13:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:13:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:13:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:13:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:13:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:13:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:13:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:13:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:13:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:13:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:13:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.81979677, 2024-09-13T09:13:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:13:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:13:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:13:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:13:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:13:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:13:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:13:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:13:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:13:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:13:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:13:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:13:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:13:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:13:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.443 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.443 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:31.444 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:31.445 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:31.443+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:31.499 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:13:31.509 INFO:teuthology.orchestra.run.smithi157.stdout:4562 write 0x7519600 thru 0x75238e8 (0xa2e9 bytes) 2024-09-13T09:13:31.511 INFO:teuthology.orchestra.run.smithi157.stdout:4565 read 0x3082000 thru 0x308c875 (0xa876 bytes) 2024-09-13T09:13:31.511 INFO:teuthology.orchestra.run.smithi157.stdout:4566 punch from 0x1aaba00 to 0x1aae000, (0x2600 bytes) 2024-09-13T09:13:31.512 INFO:teuthology.orchestra.run.smithi157.stdout:4567 read 0x4119200 thru 0x41228f5 (0x96f6 bytes) 2024-09-13T09:13:31.512 INFO:teuthology.orchestra.run.smithi157.stdout:4569 punch from 0x3285a00 to 0x328ca00, (0x7000 bytes) 2024-09-13T09:13:31.530 INFO:teuthology.orchestra.run.smithi157.stdout:4570 write 0xc49be00 thru 0xc4a1252 (0x5453 bytes) 2024-09-13T09:13:31.530 INFO:teuthology.orchestra.run.smithi157.stdout:4571 read 0xa821000 thru 0xa82c4f6 (0xb4f7 bytes) 2024-09-13T09:13:31.530 INFO:teuthology.orchestra.run.smithi157.stdout:4573 read 0x1887400 thru 0x188dda3 (0x69a4 bytes) 2024-09-13T09:13:31.531 INFO:teuthology.orchestra.run.smithi157.stdout:4574 write 0x5de7400 thru 0x5df02b9 (0x8eba bytes) 2024-09-13T09:13:31.532 INFO:teuthology.orchestra.run.smithi157.stdout:4575 read 0x7289400 thru 0x7290485 (0x7086 bytes) 2024-09-13T09:13:31.533 INFO:teuthology.orchestra.run.smithi157.stdout:4578 write 0xbd1fa00 thru 0xbd29ad0 (0xa0d1 bytes) 2024-09-13T09:13:31.533 INFO:teuthology.orchestra.run.smithi157.stdout:4579 read 0x77ea200 thru 0x77f8b51 (0xe952 bytes) 2024-09-13T09:13:31.533 INFO:teuthology.orchestra.run.smithi157.stdout:4580 write 0xc061e00 thru 0xc06656b (0x476c bytes) 2024-09-13T09:13:31.533 INFO:teuthology.orchestra.run.smithi157.stdout:4581 write 0x16c3400 thru 0x16cb89a (0x849b bytes) 2024-09-13T09:13:31.534 INFO:teuthology.orchestra.run.smithi157.stdout:4582 write 0x391f600 thru 0x392ab5c (0xb55d bytes) 2024-09-13T09:13:31.535 INFO:teuthology.orchestra.run.smithi157.stdout:4585 read 0x62ffe00 thru 0x6300efb (0x10fc bytes) 2024-09-13T09:13:31.536 INFO:teuthology.orchestra.run.smithi157.stdout:4586 write 0x2912c00 thru 0x2913f99 (0x139a bytes) 2024-09-13T09:13:31.537 INFO:teuthology.orchestra.run.smithi157.stdout:4587 punch from 0x93c0c00 to 0x93c4a00, (0x3e00 bytes) 2024-09-13T09:13:31.538 INFO:teuthology.orchestra.run.smithi157.stdout:4588 write 0x49e1c00 thru 0x49e8111 (0x6512 bytes) 2024-09-13T09:13:31.539 INFO:teuthology.orchestra.run.smithi157.stdout:4589 write 0xb058a00 thru 0xb05fb27 (0x7128 bytes) 2024-09-13T09:13:31.539 INFO:teuthology.orchestra.run.smithi157.stdout:4591 write 0x274d000 thru 0x275baaf (0xeab0 bytes) 2024-09-13T09:13:31.540 INFO:teuthology.orchestra.run.smithi157.stdout:4592 punch from 0x7d5b800 to 0x7d61200, (0x5a00 bytes) 2024-09-13T09:13:31.540 INFO:teuthology.orchestra.run.smithi157.stdout:4595 write 0x726e200 thru 0x72728d0 (0x46d1 bytes) 2024-09-13T09:13:31.545 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:31.544+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:31.554 INFO:teuthology.orchestra.run.smithi157.stdout:4597 trunc from 0xc4a1400 to 0x3de6c00 2024-09-13T09:13:31.578 INFO:teuthology.orchestra.run.smithi157.stdout:4598 punch from 0x144b200 to 0x144fa00, (0x4800 bytes) 2024-09-13T09:13:31.603 INFO:teuthology.orchestra.run.smithi157.stdout:4599 write 0xc71b800 thru 0xc7274e5 (0xbce6 bytes) 2024-09-13T09:13:31.604 INFO:teuthology.orchestra.run.smithi157.stdout:4600 read 0x2767200 thru 0x2773705 (0xc506 bytes) 2024-09-13T09:13:31.605 INFO:teuthology.orchestra.run.smithi157.stdout:4601 punch from 0x66f8000 to 0x66f8600, (0x600 bytes) 2024-09-13T09:13:31.605 INFO:teuthology.orchestra.run.smithi157.stdout:4604 punch from 0x6171400 to 0x6180800, (0xf400 bytes) 2024-09-13T09:13:31.605 INFO:teuthology.orchestra.run.smithi157.stdout:4608 punch from 0xaf9c800 to 0xafa5400, (0x8c00 bytes) 2024-09-13T09:13:31.605 INFO:teuthology.orchestra.run.smithi157.stdout:4609 write 0x93a6000 thru 0x93b4fb7 (0xefb8 bytes) 2024-09-13T09:13:31.608 INFO:teuthology.orchestra.run.smithi157.stdout:4610 trunc from 0xc727600 to 0xa249a00 2024-09-13T09:13:31.610 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:31.610 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:31.611 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:31.611 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:31.622 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:31.627 INFO:teuthology.orchestra.run.smithi157.stdout:4613 write 0x291ac00 thru 0x2926282 (0xb683 bytes) 2024-09-13T09:13:31.640 INFO:teuthology.orchestra.run.smithi157.stdout:4614 trunc from 0xa249a00 to 0x41a400 2024-09-13T09:13:31.645 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:31.643+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:31.683 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op export --pgid 3.18 --file /var/log/ceph/exp.3.18.2 2024-09-13T09:13:31.696 INFO:teuthology.orchestra.run.smithi157.stdout:4615 read 0x1d0400 thru 0x1df4f1 (0xf0f2 bytes) 2024-09-13T09:13:31.699 INFO:teuthology.orchestra.run.smithi157.stdout:4616 read 0x119400 thru 0x11d757 (0x4358 bytes) 2024-09-13T09:13:31.699 INFO:teuthology.orchestra.run.smithi157.stdout:4617 read 0x39fe00 thru 0x3afc96 (0xfe97 bytes) 2024-09-13T09:13:31.700 INFO:teuthology.orchestra.run.smithi157.stdout:4618 read 0x30e400 thru 0x313928 (0x5529 bytes) 2024-09-13T09:13:31.714 INFO:teuthology.orchestra.run.smithi157.stdout:4619 write 0x541d000 thru 0x5424014 (0x7015 bytes) 2024-09-13T09:13:31.715 INFO:teuthology.orchestra.run.smithi157.stdout:4621 read 0x8f4c00 thru 0x8f671e (0x1b1f bytes) 2024-09-13T09:13:31.715 INFO:teuthology.orchestra.run.smithi157.stdout:4622 punch from 0xa05800 to 0xa07c00, (0x2400 bytes) 2024-09-13T09:13:31.715 INFO:teuthology.orchestra.run.smithi157.stdout:4624 write 0x2cfac00 thru 0x2d01ab1 (0x6eb2 bytes) 2024-09-13T09:13:31.716 INFO:teuthology.orchestra.run.smithi157.stdout:4626 punch from 0x1295c00 to 0x12a2400, (0xc800 bytes) 2024-09-13T09:13:31.716 INFO:teuthology.orchestra.run.smithi157.stdout:4628 punch from 0x3ccda00 to 0x3cce400, (0xa00 bytes) 2024-09-13T09:13:31.716 INFO:teuthology.orchestra.run.smithi157.stdout:4630 read 0x168fa00 thru 0x1693e92 (0x4493 bytes) 2024-09-13T09:13:31.732 INFO:teuthology.orchestra.run.smithi157.stdout:4631 trunc from 0x5424200 to 0xda9ce00 2024-09-13T09:13:31.740 INFO:teuthology.orchestra.run.smithi157.stdout:4633 write 0x4204a00 thru 0x420b887 (0x6e88 bytes) 2024-09-13T09:13:31.741 INFO:teuthology.orchestra.run.smithi157.stdout:4634 read 0x5c32600 thru 0x5c42104 (0xfb05 bytes) 2024-09-13T09:13:31.747 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:31.746+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:31.749 INFO:teuthology.orchestra.run.smithi157.stdout:4635 trunc from 0xda9ce00 to 0x7380e00 2024-09-13T09:13:31.763 INFO:teuthology.orchestra.run.smithi157.stdout:4637 read 0x4cb7a00 thru 0x4cb7e3d (0x43e bytes) 2024-09-13T09:13:31.764 INFO:teuthology.orchestra.run.smithi157.stdout:4638 read 0x43c2a00 thru 0x43ce206 (0xb807 bytes) 2024-09-13T09:13:31.765 INFO:teuthology.orchestra.run.smithi157.stdout:4640 read 0x4a8d600 thru 0x4a93b7f (0x6580 bytes) 2024-09-13T09:13:31.765 INFO:teuthology.orchestra.run.smithi157.stdout:4641 read 0x474e00 thru 0x47d725 (0x8926 bytes) 2024-09-13T09:13:31.778 INFO:teuthology.orchestra.run.smithi157.stdout:4644 write 0xab6a600 thru 0xab715e5 (0x6fe6 bytes) 2024-09-13T09:13:31.779 INFO:teuthology.orchestra.run.smithi157.stdout:4646 read 0x2700a00 thru 0x27018d6 (0xed7 bytes) 2024-09-13T09:13:31.779 INFO:teuthology.orchestra.run.smithi157.stdout:4647 write 0x9862200 thru 0x987078b (0xe58c bytes) 2024-09-13T09:13:31.779 INFO:teuthology.orchestra.run.smithi157.stdout:4649 read 0xa7a5e00 thru 0xa7ab53d (0x573e bytes) 2024-09-13T09:13:31.780 INFO:teuthology.orchestra.run.smithi157.stdout:4650 write 0x238da00 thru 0x238efd1 (0x15d2 bytes) 2024-09-13T09:13:31.780 INFO:teuthology.orchestra.run.smithi157.stdout:4653 punch from 0x5869c00 to 0x586d800, (0x3c00 bytes) 2024-09-13T09:13:31.780 INFO:teuthology.orchestra.run.smithi157.stdout:4655 read 0x3897a00 thru 0x389d8d7 (0x5ed8 bytes) 2024-09-13T09:13:31.783 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:31.783 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:31.783 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:31.783 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:31.783 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:31.793 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:31.800 INFO:teuthology.orchestra.run.smithi157.stdout:4656 write 0xe8fe600 thru 0xe908db8 (0xa7b9 bytes) 2024-09-13T09:13:31.806 INFO:teuthology.orchestra.run.smithi157.stdout:4659 trunc from 0xe908e00 to 0xa4b8600 2024-09-13T09:13:31.824 INFO:teuthology.orchestra.run.smithi157.stdout:4660 read 0x6a80e00 thru 0x6a90067 (0xf268 bytes) 2024-09-13T09:13:31.826 INFO:teuthology.orchestra.run.smithi157.stdout:4661 punch from 0x5d84c00 to 0x5d8c400, (0x7800 bytes) 2024-09-13T09:13:31.826 INFO:teuthology.orchestra.run.smithi157.stdout:4663 read 0x7dad000 thru 0x7db4527 (0x7528 bytes) 2024-09-13T09:13:31.826 INFO:teuthology.orchestra.run.smithi157.stdout:4664 punch from 0x4543400 to 0x4549800, (0x6400 bytes) 2024-09-13T09:13:31.837 INFO:teuthology.orchestra.run.smithi157.stdout:4666 write 0xb580000 thru 0xb586709 (0x670a bytes) 2024-09-13T09:13:31.838 INFO:teuthology.orchestra.run.smithi157.stdout:4667 read 0xb23a200 thru 0xb23c161 (0x1f62 bytes) 2024-09-13T09:13:31.847 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:31.845+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:31.849 INFO:teuthology.orchestra.run.smithi157.stdout:4668 trunc from 0xb586800 to 0x1d29400 2024-09-13T09:13:31.897 INFO:teuthology.orchestra.run.smithi157.stdout:4669 write 0x8b43000 thru 0x8b51abb (0xeabc bytes) 2024-09-13T09:13:31.898 INFO:teuthology.orchestra.run.smithi157.stdout:4672 punch from 0x494f000 to 0x495d800, (0xe800 bytes) 2024-09-13T09:13:31.898 INFO:teuthology.orchestra.run.smithi157.stdout:4673 trunc from 0x8b51c00 to 0x8ee0400 2024-09-13T09:13:31.921 INFO:teuthology.orchestra.run.smithi157.stdout:4674 write 0xb013c00 thru 0xb0210b8 (0xd4b9 bytes) 2024-09-13T09:13:31.921 INFO:teuthology.orchestra.run.smithi157.stdout:4675 write 0x91d3000 thru 0x91dcf7e (0x9f7f bytes) 2024-09-13T09:13:31.921 INFO:teuthology.orchestra.run.smithi157.stdout:4676 read 0x6fe3e00 thru 0x6fec929 (0x8b2a bytes) 2024-09-13T09:13:31.922 INFO:teuthology.orchestra.run.smithi157.stdout:4677 read 0x8fc2400 thru 0x8fc6cca (0x48cb bytes) 2024-09-13T09:13:31.922 INFO:teuthology.orchestra.run.smithi157.stdout:4678 write 0xd62000 thru 0xd71e2c (0xfe2d bytes) 2024-09-13T09:13:31.923 INFO:teuthology.orchestra.run.smithi157.stdout:4679 write 0x1ebc600 thru 0x1ec8195 (0xbb96 bytes) 2024-09-13T09:13:31.923 INFO:teuthology.orchestra.run.smithi157.stdout:4682 punch from 0xa5c6e00 to 0xa5cc800, (0x5a00 bytes) 2024-09-13T09:13:31.923 INFO:teuthology.orchestra.run.smithi157.stdout:4683 read 0x73ec400 thru 0x73f0451 (0x4052 bytes) 2024-09-13T09:13:31.924 INFO:teuthology.orchestra.run.smithi157.stdout:4684 read 0x256ec00 thru 0x256f71a (0xb1b bytes) 2024-09-13T09:13:31.936 INFO:teuthology.orchestra.run.smithi157.stdout:4686 write 0xc978e00 thru 0xc97d139 (0x433a bytes) 2024-09-13T09:13:31.936 INFO:teuthology.orchestra.run.smithi157.stdout:4687 read 0xaa8ee00 thru 0xaa966e5 (0x78e6 bytes) 2024-09-13T09:13:31.938 INFO:teuthology.orchestra.run.smithi157.stdout:4688 read 0xb41ba00 thru 0xb422703 (0x6d04 bytes) 2024-09-13T09:13:31.938 INFO:teuthology.orchestra.run.smithi157.stdout:4690 read 0x919dc00 thru 0x91a386f (0x5c70 bytes) 2024-09-13T09:13:31.938 INFO:teuthology.orchestra.run.smithi157.stdout:4692 read 0xadd9c00 thru 0xaddbd7c (0x217d bytes) 2024-09-13T09:13:31.948 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:31.947+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:31.952 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9035 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2342912~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.204138749999998, 2024-09-13T09:13:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070293606999999994, 2024-09-13T09:13:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9035 2024-09-13T09:13:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745602+0000", 2024-09-13T09:13:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-06 2024-09-13T09:13:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745607+0000", 2024-09-13T09:13:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0770000000000001e-06 2024-09-13T09:13:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745609+0000", 2024-09-13T09:13:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3499999999999999e-06 2024-09-13T09:13:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815626+0000", 2024-09-13T09:13:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070017398999999994 2024-09-13T09:13:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815672+0000", 2024-09-13T09:13:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5275999999999998e-05 2024-09-13T09:13:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815895+0000", 2024-09-13T09:13:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022328300000000001 2024-09-13T09:13:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.204126205999998, 2024-09-13T09:13:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:13:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:13:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:13:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:13:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:13:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:13:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:13:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:13:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:13:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:13:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:13:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:13:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.204117527999998, 2024-09-13T09:13:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:13:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:13:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:13:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:13:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:13:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:13:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:13:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:13:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:13:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:13:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:13:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:13:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.204108542, 2024-09-13T09:13:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:13:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:13:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:13:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:13:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:13:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:13:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:13:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:13:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:13:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:13:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:13:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:13:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:13:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:13:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.204100570000001, 2024-09-13T09:13:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:13:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:13:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:13:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:13:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:13:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:13:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:13:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:13:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:13:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:13:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:13:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:13:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:13:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:13:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.202225736000003, 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:13:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:13:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:13:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:13:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:13:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:13:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.202215074999998, 2024-09-13T09:13:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:13:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:13:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:13:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:13:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:13:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:13:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:13:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:13:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:13:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:13:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:13:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:13:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:13:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:13:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.202193098999999, 2024-09-13T09:13:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:13:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:13:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:13:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:13:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:13:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:13:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:13:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:13:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:13:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:13:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:13:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:13:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:13:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:13:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.202179596999997, 2024-09-13T09:13:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:13:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:13:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:13:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:13:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:13:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:13:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:13:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:13:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:13:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:13:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:13:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:13:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:13:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:13:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9056 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 7159808~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.200178768000001, 2024-09-13T09:13:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070283126000000001, 2024-09-13T09:13:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9056 2024-09-13T09:13:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749562+0000", 2024-09-13T09:13:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1340000000000001e-06 2024-09-13T09:13:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749565+0000", 2024-09-13T09:13:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8760000000000001e-06 2024-09-13T09:13:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749567+0000", 2024-09-13T09:13:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2340000000000001e-06 2024-09-13T09:13:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819750+0000", 2024-09-13T09:13:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070182537000000003 2024-09-13T09:13:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819792+0000", 2024-09-13T09:13:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2141999999999997e-05 2024-09-13T09:13:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819844+0000", 2024-09-13T09:13:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2428e-05 2024-09-13T09:13:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.200170499000002, 2024-09-13T09:13:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:13:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:13:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:13:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:13:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:13:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:13:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:13:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:13:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:13:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:13:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:13:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:13:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:13:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:13:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.200146177000001, 2024-09-13T09:13:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:13:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:13:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:13:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:13:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:13:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:13:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:13:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:13:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:13:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:13:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:13:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:13:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:13:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:13:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.200135203000002, 2024-09-13T09:13:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:13:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:13:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:13:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:13:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:13:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:13:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:13:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:13:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:13:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:13:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:13:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:13:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:13:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:13:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:13:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9063 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1589248~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.198231561, 2024-09-13T09:13:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070391924999999994, 2024-09-13T09:13:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9063 2024-09-13T09:13:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751509+0000", 2024-09-13T09:13:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.142e-06 2024-09-13T09:13:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751513+0000", 2024-09-13T09:13:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6669999999999998e-06 2024-09-13T09:13:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751520+0000", 2024-09-13T09:13:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5910000000000001e-06 2024-09-13T09:13:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821593+0000", 2024-09-13T09:13:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073519000000001 2024-09-13T09:13:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821648+0000", 2024-09-13T09:13:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:13:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821900+0000", 2024-09-13T09:13:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025263999999999999 2024-09-13T09:13:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.198217577999998, 2024-09-13T09:13:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:13:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:13:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:13:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:13:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:13:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:13:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:13:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:13:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:13:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:13:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:13:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:13:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:13:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.198208180999998, 2024-09-13T09:13:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:13:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:13:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:13:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:13:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:13:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:13:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:13:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:13:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:13:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:13:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:13:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:13:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:13:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:13:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9071 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.196225163999998, 2024-09-13T09:13:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070386377999999999, 2024-09-13T09:13:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9071 2024-09-13T09:13:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753516+0000", 2024-09-13T09:13:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1519999999999999e-06 2024-09-13T09:13:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753519+0000", 2024-09-13T09:13:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6559999999999998e-06 2024-09-13T09:13:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753522+0000", 2024-09-13T09:13:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:13:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823798+0000", 2024-09-13T09:13:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070275907999999998 2024-09-13T09:13:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823845+0000", 2024-09-13T09:13:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7593000000000003e-05 2024-09-13T09:13:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823901+0000", 2024-09-13T09:13:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5934999999999999e-05 2024-09-13T09:13:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3181 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T09:13:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.9278440049999999, 2024-09-13T09:13:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.480957603, 2024-09-13T09:13:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:13:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:13:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3181 2024-09-13T09:13:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021894+0000", 2024-09-13T09:13:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:13:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021903+0000", 2024-09-13T09:13:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4109999999999996e-06 2024-09-13T09:13:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021919+0000", 2024-09-13T09:13:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6225000000000001e-05 2024-09-13T09:13:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021926+0000", 2024-09-13T09:13:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4120000000000004e-06 2024-09-13T09:13:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021975+0000", 2024-09-13T09:13:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8526000000000003e-05 2024-09-13T09:13:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022049+0000", 2024-09-13T09:13:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4018999999999997e-05 2024-09-13T09:13:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:13:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022230+0000", 2024-09-13T09:13:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018085599999999999 2024-09-13T09:13:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:13:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022796+0000", 2024-09-13T09:13:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056638999999999995 2024-09-13T09:13:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499540+0000", 2024-09-13T09:13:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.47674342800000002 2024-09-13T09:13:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499547+0000", 2024-09-13T09:13:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:13:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502794+0000", 2024-09-13T09:13:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.003246818 2024-09-13T09:13:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502854+0000", 2024-09-13T09:13:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9925e-05 2024-09-13T09:13:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3182 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 139818970156544 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:13:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.45460945, 2024-09-13T09:13:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0089058390000001, 2024-09-13T09:13:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:13:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:13:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3182 2024-09-13T09:13:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495128+0000", 2024-09-13T09:13:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:13:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495139+0000", 2024-09-13T09:13:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0599e-05 2024-09-13T09:13:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495157+0000", 2024-09-13T09:13:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7830999999999999e-05 2024-09-13T09:13:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495167+0000", 2024-09-13T09:13:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0689999999999999e-05 2024-09-13T09:13:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495229+0000", 2024-09-13T09:13:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1898999999999998e-05 2024-09-13T09:13:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:13:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495243+0000", 2024-09-13T09:13:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3349e-05 2024-09-13T09:13:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499575+0000", 2024-09-13T09:13:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043324410000000002 2024-09-13T09:13:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499582+0000", 2024-09-13T09:13:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0849999999999999e-06 2024-09-13T09:13:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502872+0000", 2024-09-13T09:13:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0032895150000001 2024-09-13T09:13:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503016+0000", 2024-09-13T09:13:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014394200000000001 2024-09-13T09:13:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504036+0000", 2024-09-13T09:13:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010208680000000001 2024-09-13T09:13:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:19151 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 140383089882784 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:13:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.454597991, 2024-09-13T09:13:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.009071287, 2024-09-13T09:13:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:13:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:13:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19151 2024-09-13T09:13:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495140+0000", 2024-09-13T09:13:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:13:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495149+0000", 2024-09-13T09:13:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0759999999999992e-06 2024-09-13T09:13:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495165+0000", 2024-09-13T09:13:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5322e-05 2024-09-13T09:13:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495173+0000", 2024-09-13T09:13:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0979999999999994e-06 2024-09-13T09:13:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495318+0000", 2024-09-13T09:13:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014553 2024-09-13T09:13:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:13:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495325+0000", 2024-09-13T09:13:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:13:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499598+0000", 2024-09-13T09:13:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004273064 2024-09-13T09:13:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499602+0000", 2024-09-13T09:13:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8539999999999999e-06 2024-09-13T09:13:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503507+0000", 2024-09-13T09:13:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0039045520000001 2024-09-13T09:13:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503583+0000", 2024-09-13T09:13:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6283000000000001e-05 2024-09-13T09:13:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504213+0000", 2024-09-13T09:13:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00063044700000000002 2024-09-13T09:13:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:32.062 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:32.065 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:32.063+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:32.065 INFO:teuthology.orchestra.run.smithi157.stdout:4694 trunc from 0xc97d200 to 0x134a00 2024-09-13T09:13:32.065 INFO:teuthology.orchestra.run.smithi157.stdout:4696 write 0x8622200 thru 0x86301da (0xdfdb bytes) 2024-09-13T09:13:32.065 INFO:teuthology.orchestra.run.smithi157.stdout:4697 write 0xa690c00 thru 0xa699298 (0x8699 bytes) 2024-09-13T09:13:32.065 INFO:teuthology.orchestra.run.smithi157.stdout:4698 read 0x44d2800 thru 0x44d2c94 (0x495 bytes) 2024-09-13T09:13:32.066 INFO:teuthology.orchestra.run.smithi157.stdout:4701 write 0xac41600 thru 0xac4685b (0x525c bytes) 2024-09-13T09:13:32.066 INFO:teuthology.orchestra.run.smithi157.stdout:4702 read 0x12f3a00 thru 0x12fdd4c (0xa34d bytes) 2024-09-13T09:13:32.066 INFO:teuthology.orchestra.run.smithi157.stdout:4704 write 0x19f6800 thru 0x1a02468 (0xbc69 bytes) 2024-09-13T09:13:32.066 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:32.081 INFO:teuthology.orchestra.run.smithi157.stdout:4705 write 0xe058a00 thru 0xe060b0f (0x8110 bytes) 2024-09-13T09:13:32.082 INFO:teuthology.orchestra.run.smithi157.stdout:4707 write 0xcbf5800 thru 0xcc02cb9 (0xd4ba bytes) 2024-09-13T09:13:32.090 INFO:teuthology.orchestra.run.smithi157.stdout:4708 trunc from 0xe060c00 to 0x7425400 2024-09-13T09:13:32.115 INFO:teuthology.orchestra.run.smithi157.stdout:4709 trunc from 0x7425400 to 0x44a1400 2024-09-13T09:13:32.126 INFO:teuthology.orchestra.run.smithi157.stdout:4710 read 0x3e25a00 thru 0x3e31e11 (0xc412 bytes) 2024-09-13T09:13:32.132 INFO:teuthology.orchestra.run.smithi157.stdout:4712 punch from 0x1402400 to 0x1405c00, (0x3800 bytes) 2024-09-13T09:13:32.132 INFO:teuthology.orchestra.run.smithi157.stdout:4714 read 0x3066000 thru 0x306dd81 (0x7d82 bytes) 2024-09-13T09:13:32.133 INFO:teuthology.orchestra.run.smithi157.stdout:4715 read 0x369b200 thru 0x36a3697 (0x8498 bytes) 2024-09-13T09:13:32.133 INFO:teuthology.orchestra.run.smithi157.stdout:4716 write 0x1a23600 thru 0x1a31c68 (0xe669 bytes) 2024-09-13T09:13:32.143 INFO:teuthology.orchestra.run.smithi157.stdout:4719 trunc from 0x44a1400 to 0xccb8e00 2024-09-13T09:13:32.152 INFO:teuthology.orchestra.run.smithi157.stdout:4720 write 0x5470a00 thru 0x5477346 (0x6947 bytes) 2024-09-13T09:13:32.160 INFO:teuthology.orchestra.run.smithi157.stdout:4723 write 0x5363a00 thru 0x536d377 (0x9978 bytes) 2024-09-13T09:13:32.160 INFO:teuthology.orchestra.run.smithi157.stdout:4724 write 0x5aa5600 thru 0x5aaa145 (0x4b46 bytes) 2024-09-13T09:13:32.161 INFO:teuthology.orchestra.run.smithi157.stdout:4725 read 0x127b800 thru 0x127d41c (0x1c1d bytes) 2024-09-13T09:13:32.161 INFO:teuthology.orchestra.run.smithi157.stdout:4726 read 0x739c400 thru 0x73a1a87 (0x5688 bytes) 2024-09-13T09:13:32.161 INFO:teuthology.orchestra.run.smithi157.stdout:4727 write 0x49db000 thru 0x49e86d2 (0xd6d3 bytes) 2024-09-13T09:13:32.161 INFO:teuthology.orchestra.run.smithi157.stdout:4729 punch from 0xa5a5200 to 0xa5af200, (0xa000 bytes) 2024-09-13T09:13:32.162 INFO:teuthology.orchestra.run.smithi157.stdout:4730 write 0x852400 thru 0x85f5bf (0xd1c0 bytes) 2024-09-13T09:13:32.164 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:32.163+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:32.172 INFO:teuthology.orchestra.run.smithi157.stdout:4731 trunc from 0xccb8e00 to 0x3f1c600 2024-09-13T09:13:32.179 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:32.179 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:32.189 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:32.205 INFO:teuthology.orchestra.run.smithi157.stdout:4735 write 0xb8f7c00 thru 0xb906b14 (0xef15 bytes) 2024-09-13T09:13:32.216 INFO:teuthology.orchestra.run.smithi157.stdout:4737 trunc from 0xb906c00 to 0x84dda00 2024-09-13T09:13:32.255 INFO:teuthology.orchestra.run.smithi157.stdout:4741 write 0xa22e600 thru 0xa235d83 (0x7784 bytes) 2024-09-13T09:13:32.256 INFO:teuthology.orchestra.run.smithi157.stdout:4742 read 0xfee200 thru 0xff3fd9 (0x5dda bytes) 2024-09-13T09:13:32.256 INFO:teuthology.orchestra.run.smithi157.stdout:4743 read 0x8aeae00 thru 0x8af8c97 (0xde98 bytes) 2024-09-13T09:13:32.257 INFO:teuthology.orchestra.run.smithi157.stdout:4744 write 0x9963600 thru 0x996c10c (0x8b0d bytes) 2024-09-13T09:13:32.257 INFO:teuthology.orchestra.run.smithi157.stdout:4748 read 0x1167000 thru 0x116777a (0x77b bytes) 2024-09-13T09:13:32.257 INFO:teuthology.orchestra.run.smithi157.stdout:4749 punch from 0x1c96400 to 0x1ca0800, (0xa400 bytes) 2024-09-13T09:13:32.258 INFO:teuthology.orchestra.run.smithi157.stdout:4750 read 0x82c600 thru 0x83116b (0x4b6c bytes) 2024-09-13T09:13:32.260 INFO:teuthology.orchestra.run.smithi157.stdout:4751 read 0x687200 thru 0x68db73 (0x6974 bytes) 2024-09-13T09:13:32.260 INFO:teuthology.orchestra.run.smithi157.stdout:4752 read 0x755aa00 thru 0x7561cb6 (0x72b7 bytes) 2024-09-13T09:13:32.261 INFO:teuthology.orchestra.run.smithi157.stdout:4755 write 0x10b3a00 thru 0x10c19f8 (0xdff9 bytes) 2024-09-13T09:13:32.261 INFO:teuthology.orchestra.run.smithi157.stdout:4756 read 0x22c7800 thru 0x22cadf7 (0x35f8 bytes) 2024-09-13T09:13:32.261 INFO:teuthology.orchestra.run.smithi157.stdout:4758 read 0x8080c00 thru 0x8083d48 (0x3149 bytes) 2024-09-13T09:13:32.264 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:32.264+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:32.269 INFO:teuthology.orchestra.run.smithi157.stdout:4759 trunc from 0xa235e00 to 0x4207a00 2024-09-13T09:13:32.293 INFO:teuthology.orchestra.run.smithi157.stdout:4760 read 0x2a3fa00 thru 0x2a402c5 (0x8c6 bytes) 2024-09-13T09:13:32.300 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:32.300 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:32.309 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:32.319 INFO:teuthology.orchestra.run.smithi157.stdout:4763 write 0xd982000 thru 0xd9890e6 (0x70e7 bytes) 2024-09-13T09:13:32.337 INFO:teuthology.orchestra.run.smithi157.stdout:4764 write 0xea10400 thru 0xea1f8bf (0xf4c0 bytes) 2024-09-13T09:13:32.342 INFO:teuthology.orchestra.run.smithi157.stdout:4765 trunc from 0xea1fa00 to 0xa5a2e00 2024-09-13T09:13:32.365 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:32.363+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:32.369 INFO:teuthology.orchestra.run.smithi157.stdout:4768 write 0x8fd4000 thru 0x8fe1484 (0xd485 bytes) 2024-09-13T09:13:32.370 INFO:teuthology.orchestra.run.smithi157.stdout:4770 punch from 0x6c4ca00 to 0x6c50000, (0x3600 bytes) 2024-09-13T09:13:32.370 INFO:teuthology.orchestra.run.smithi157.stdout:4771 read 0x6099a00 thru 0x609dfb7 (0x45b8 bytes) 2024-09-13T09:13:32.385 INFO:teuthology.orchestra.run.smithi157.stdout:4772 write 0xb381800 thru 0xb38e7e2 (0xcfe3 bytes) 2024-09-13T09:13:32.386 INFO:teuthology.orchestra.run.smithi157.stdout:4773 read 0x5155800 thru 0x515a73b (0x4f3c bytes) 2024-09-13T09:13:32.391 INFO:teuthology.orchestra.run.smithi157.stdout:4774 trunc from 0xb38e800 to 0x6dd5400 2024-09-13T09:13:32.419 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:32.419 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:32.419 INFO:teuthology.orchestra.run.smithi157.stdout:4776 punch from 0x5846200 to 0x5847200, (0x1000 bytes) 2024-09-13T09:13:32.428 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:32.428 INFO:teuthology.orchestra.run.smithi157.stdout:4779 trunc from 0x6dd5400 to 0xed03200 2024-09-13T09:13:32.441 INFO:teuthology.orchestra.run.smithi157.stdout:4780 write 0x638ce00 thru 0x6392a41 (0x5c42 bytes) 2024-09-13T09:13:32.458 INFO:teuthology.orchestra.run.smithi157.stdout:4781 trunc from 0xed03200 to 0xef7000 2024-09-13T09:13:32.465 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:32.465+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:32.489 INFO:teuthology.orchestra.run.smithi157.stdout:4783 punch from 0xe30000 to 0xe3aa00, (0xaa00 bytes) 2024-09-13T09:13:32.503 INFO:teuthology.orchestra.run.smithi157.stdout:4785 write 0x1c78e00 thru 0x1c7d47c (0x467d bytes) 2024-09-13T09:13:32.504 INFO:teuthology.orchestra.run.smithi157.stdout:4786 trunc from 0x1c7d600 to 0x1e13e00 2024-09-13T09:13:32.512 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-09-13T09:13:32.519 INFO:teuthology.orchestra.run.smithi157.stdout:4787 punch from 0xb76000 to 0xb7d200, (0x7200 bytes) 2024-09-13T09:13:32.529 INFO:teuthology.orchestra.run.smithi157.stdout:4789 trunc from 0x1e13e00 to 0xa72c600 2024-09-13T09:13:32.543 INFO:teuthology.orchestra.run.smithi157.stdout:4791 trunc from 0xa72c600 to 0x9d63800 2024-09-13T09:13:32.561 INFO:teuthology.orchestra.run.smithi157.stdout:4792 trunc from 0x9d63800 to 0x9c78200 2024-09-13T09:13:32.567 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:32.566+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:32.576 INFO:teuthology.orchestra.run.smithi157.stdout:4793 punch from 0x50b7800 to 0x50b8c00, (0x1400 bytes) 2024-09-13T09:13:32.576 INFO:teuthology.orchestra.run.smithi157.stdout:4795 punch from 0x7b49000 to 0x7b4a600, (0x1600 bytes) 2024-09-13T09:13:32.576 INFO:teuthology.orchestra.run.smithi157.stdout:4798 write 0x8447400 thru 0x8449f97 (0x2b98 bytes) 2024-09-13T09:13:32.577 INFO:teuthology.orchestra.run.smithi157.stdout:4799 punch from 0x4945800 to 0x4953400, (0xdc00 bytes) 2024-09-13T09:13:32.578 INFO:teuthology.orchestra.run.smithi157.stdout:4800 trunc from 0x9c78200 to 0x8f28600 2024-09-13T09:13:32.585 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:32.585 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:32.593 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:32.594 INFO:teuthology.orchestra.run.smithi157.stdout:4803 write 0x16a6200 thru 0x16b371d (0xd51e bytes) 2024-09-13T09:13:32.616 INFO:teuthology.orchestra.run.smithi157.stdout:4806 write 0xd54c200 thru 0xd558e75 (0xcc76 bytes) 2024-09-13T09:13:32.616 INFO:teuthology.orchestra.run.smithi157.stdout:4807 read 0x453ea00 thru 0x4542e54 (0x4455 bytes) 2024-09-13T09:13:32.616 INFO:teuthology.orchestra.run.smithi157.stdout:4812 read 0x84f4600 thru 0x85045e5 (0xffe6 bytes) 2024-09-13T09:13:32.617 INFO:teuthology.orchestra.run.smithi157.stdout:4813 write 0xd323c00 thru 0xd3298a1 (0x5ca2 bytes) 2024-09-13T09:13:32.617 INFO:teuthology.orchestra.run.smithi157.stdout:4814 write 0x6be0800 thru 0x6be66f3 (0x5ef4 bytes) 2024-09-13T09:13:32.617 INFO:teuthology.orchestra.run.smithi157.stdout:4816 read 0x11efe00 thru 0x11fb120 (0xb321 bytes) 2024-09-13T09:13:32.618 INFO:teuthology.orchestra.run.smithi157.stdout:4817 punch from 0x84f4a00 to 0x84f6e00, (0x2400 bytes) 2024-09-13T09:13:32.618 INFO:teuthology.orchestra.run.smithi157.stdout:4821 write 0x28eda00 thru 0x28fd313 (0xf914 bytes) 2024-09-13T09:13:32.629 INFO:teuthology.orchestra.run.smithi157.stdout:4822 trunc from 0xd559000 to 0x478da00 2024-09-13T09:13:32.657 INFO:teuthology.orchestra.run.smithi157.stdout:4825 write 0x150000 thru 0x1530eb (0x30ec bytes) 2024-09-13T09:13:32.659 INFO:teuthology.orchestra.run.smithi157.stdout:4826 trunc from 0x478da00 to 0x2f2dc00 2024-09-13T09:13:32.668 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:32.666+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:32.704 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:32.704 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:32.709 INFO:teuthology.orchestra.run.smithi157.stdout:4827 write 0xee18e00 thru 0xee27672 (0xe873 bytes) 2024-09-13T09:13:32.709 INFO:teuthology.orchestra.run.smithi157.stdout:4829 punch from 0x9407200 to 0x940f800, (0x8600 bytes) 2024-09-13T09:13:32.709 INFO:teuthology.orchestra.run.smithi157.stdout:4830 read 0x7525600 thru 0x753471b (0xf11c bytes) 2024-09-13T09:13:32.710 INFO:teuthology.orchestra.run.smithi157.stdout:4831 read 0x5de2e00 thru 0x5de87fa (0x59fb bytes) 2024-09-13T09:13:32.711 INFO:teuthology.orchestra.run.smithi157.stdout:4834 read 0xc29f400 thru 0xc2ad420 (0xe021 bytes) 2024-09-13T09:13:32.712 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:32.713 INFO:teuthology.orchestra.run.smithi157.stdout:4836 write 0x7ebf200 thru 0x7ec6ee2 (0x7ce3 bytes) 2024-09-13T09:13:32.713 INFO:teuthology.orchestra.run.smithi157.stdout:4839 write 0xe381200 thru 0xe387224 (0x6025 bytes) 2024-09-13T09:13:32.714 INFO:teuthology.orchestra.run.smithi157.stdout:4841 read 0xe5e9a00 thru 0xe5f60b4 (0xc6b5 bytes) 2024-09-13T09:13:32.714 INFO:teuthology.orchestra.run.smithi157.stdout:4842 read 0x719cc00 thru 0x71a97a7 (0xcba8 bytes) 2024-09-13T09:13:32.714 INFO:teuthology.orchestra.run.smithi157.stdout:4845 write 0x932a600 thru 0x932fbbb (0x55bc bytes) 2024-09-13T09:13:32.715 INFO:teuthology.orchestra.run.smithi157.stdout:4848 write 0xb4cae00 thru 0xb4d990b (0xeb0c bytes) 2024-09-13T09:13:32.715 INFO:teuthology.orchestra.run.smithi157.stdout:4850 punch from 0x40a800 to 0x411800, (0x7000 bytes) 2024-09-13T09:13:32.716 INFO:teuthology.orchestra.run.smithi157.stdout:4851 read 0x63ab600 thru 0x63b9afb (0xe4fc bytes) 2024-09-13T09:13:32.729 INFO:teuthology.orchestra.run.smithi157.stdout:4852 trunc from 0xee27800 to 0x4455000 2024-09-13T09:13:32.755 INFO:teuthology.orchestra.run.smithi157.stdout:4855 trunc from 0x4455000 to 0x3b22200 2024-09-13T09:13:32.769 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:32.768+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:32.771 INFO:teuthology.orchestra.run.smithi157.stdout:4856 punch from 0x339fa00 to 0x33a3600, (0x3c00 bytes) 2024-09-13T09:13:32.772 INFO:teuthology.orchestra.run.smithi157.stdout:4857 read 0xed6c00 thru 0xee4e91 (0xe292 bytes) 2024-09-13T09:13:32.794 INFO:teuthology.orchestra.run.smithi157.stdout:4858 write 0xa53e200 thru 0xa53f932 (0x1733 bytes) 2024-09-13T09:13:32.795 INFO:teuthology.orchestra.run.smithi157.stdout:4861 punch from 0x9109600 to 0x9111600, (0x8000 bytes) 2024-09-13T09:13:32.795 INFO:teuthology.orchestra.run.smithi157.stdout:4862 clone 29 order 20 su 65536 sc 6 2024-09-13T09:13:32.837 INFO:teuthology.orchestra.run.smithi044.stderr:Exporting 3.18 info 3.18( v 83'380 (0'0,83'380] local-lis/les=21/22 n=42 ec=21/21 lis/c=21/21 les/c/f=22/22/0 sis=21) 2024-09-13T09:13:32.842 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:18750c32:::rbd_data.1123af309369.0000000000000038:1f# 2024-09-13T09:13:32.844 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:18750c32:::rbd_data.1123af309369.0000000000000038:head# 2024-09-13T09:13:32.844 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:188b27a2:::rbd_data.1123ee68625c.0000000000000023:head# 2024-09-13T09:13:32.844 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:18fa2bd0:::rbd_header.1123a0dcea4a:head# 2024-09-13T09:13:32.844 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:190c77f9:::rbd_data.1123af309369.00000000000000b4:1f# 2024-09-13T09:13:32.844 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:190c77f9:::rbd_data.1123af309369.00000000000000b4:head# 2024-09-13T09:13:32.844 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:193bbcf1:::rbd_id.image_client.0-clone20:head# 2024-09-13T09:13:32.845 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:193f53c0:::rbd_data.11233da12af.000000000000000c:head# 2024-09-13T09:13:32.845 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1958c3bf:::rbd_data.112326fb43d3.0000000000000038:11# 2024-09-13T09:13:32.846 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1958c3bf:::rbd_data.112326fb43d3.0000000000000038:head# 2024-09-13T09:13:32.846 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:196dfe53:::rbd_data.1123ee68625c.0000000000000020:head# 2024-09-13T09:13:32.846 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1981c38e:::rbd_data.1123c35b2348.0000000000000039:head# 2024-09-13T09:13:32.846 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:19fd4e96:::rbd_data.11233da12af.0000000000000023:head# 2024-09-13T09:13:32.848 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1a35bbab:::rbd_data.1123af309369.0000000000000182:1f# 2024-09-13T09:13:32.849 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1a35bbab:::rbd_data.1123af309369.0000000000000182:head# 2024-09-13T09:13:32.849 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1a99e484:::rbd_data.1123ee68625c.0000000000000025:head# 2024-09-13T09:13:32.849 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1a9b4417:::rbd_data.1123af309369.0000000000000231:1f# 2024-09-13T09:13:32.849 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1a9b4417:::rbd_data.1123af309369.0000000000000231:head# 2024-09-13T09:13:32.849 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1b0fc23a:::rbd_data.1123af309369.0000000000000035:1f# 2024-09-13T09:13:32.850 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1b0fc23a:::rbd_data.1123af309369.0000000000000035:head# 2024-09-13T09:13:32.850 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1b8df09d:::rbd_data.1123b8b4a28a.000000000000003c:head# 2024-09-13T09:13:32.851 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1bb498e5:::rbd_data.1123b428d19a.0000000000000019:16# 2024-09-13T09:13:32.851 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1bb498e5:::rbd_data.1123b428d19a.0000000000000019:head# 2024-09-13T09:13:32.851 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1be76995:::rbd_data.1123c35b2348.0000000000000007:head# 2024-09-13T09:13:32.852 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1bf4adc9:::rbd_data.1123af309369.0000000000000158:1f# 2024-09-13T09:13:32.853 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1bf4adc9:::rbd_data.1123af309369.0000000000000158:head# 2024-09-13T09:13:32.853 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1bfc759a:::rbd_data.1123276d9426.0000000000000011:19# 2024-09-13T09:13:32.854 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1bfc759a:::rbd_data.1123276d9426.0000000000000011:head# 2024-09-13T09:13:32.854 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1c152610:::rbd_data.11239759ab94.0000000000000085:1b# 2024-09-13T09:13:32.855 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1c152610:::rbd_data.11239759ab94.0000000000000085:head# 2024-09-13T09:13:32.855 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1c248fb4:::rbd_data.1123ee68625c.000000000000002a:head# 2024-09-13T09:13:32.855 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1c27f267:::rbd_data.11239759ab94.0000000000000014:1b# 2024-09-13T09:13:32.856 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1c27f267:::rbd_data.11239759ab94.0000000000000014:head# 2024-09-13T09:13:32.856 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1c607c64:::rbd_data.1123276d9426.000000000000001a:19# 2024-09-13T09:13:32.858 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1c607c64:::rbd_data.1123276d9426.000000000000001a:head# 2024-09-13T09:13:32.858 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1c92b5b7:::rbd_data.1123b8d418eb.0000000000000008:b# 2024-09-13T09:13:32.859 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1c92b5b7:::rbd_data.1123b8d418eb.0000000000000008:head# 2024-09-13T09:13:32.860 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1e0409d3:::rbd_header.11239c07d29f:head# 2024-09-13T09:13:32.860 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1e8b6a6d:::rbd_header.112360c1ab8:head# 2024-09-13T09:13:32.860 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1f50bcb3:::rbd_data.1123b428d19a.0000000000000005:16# 2024-09-13T09:13:32.862 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1f50bcb3:::rbd_data.1123b428d19a.0000000000000005:head# 2024-09-13T09:13:32.863 INFO:teuthology.orchestra.run.smithi044.stderr:Read #3:1f517812:::rbd_data.1123ee68625c.000000000000002c:head# 2024-09-13T09:13:32.863 INFO:teuthology.orchestra.run.smithi044.stderr:Export successful 2024-09-13T09:13:32.868 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:32.868 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:32.869 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:32.867+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:32.877 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:32.970 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:32.969+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:32.980 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:32.980 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:32.980 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:32.980 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:32.988 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:33.071 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:33.070+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:33.089 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:33.090 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:33.090 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:33.090 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:33.090 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:33.098 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:33.182 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:33.182+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:33.243 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:33.243 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:33.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:33.243 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:33.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.243 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.243 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:33.244 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.719459634000003, 2024-09-13T09:13:33.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:13:33.244 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.244 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.244 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.244 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.244 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.244 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:13:33.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.244 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:33.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:33.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:33.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:13:33.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:13:33.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:13:33.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:13:33.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:13:33.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:13:33.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:13:33.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:13:33.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:13:33.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:13:33.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:13:33.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:13:33.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:13:33.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:13:33.248 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.248 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.248 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.248 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.248 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:33.249 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.719410578000002, 2024-09-13T09:13:33.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:13:33.249 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.249 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.249 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.249 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.249 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.249 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:13:33.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.249 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:33.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:33.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:33.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:13:33.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:13:33.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:13:33.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:13:33.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:13:33.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:13:33.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:13:33.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:13:33.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:13:33.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:13:33.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:13:33.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:13:33.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:13:33.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:13:33.256 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.257 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.257 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.257 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.257 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:33.257 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.719370503, 2024-09-13T09:13:33.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:13:33.258 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.258 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.258 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.258 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.258 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.258 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:13:33.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.258 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:33.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:33.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:33.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:13:33.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:13:33.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:13:33.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:13:33.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:13:33.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:13:33.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:13:33.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:13:33.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:13:33.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:13:33.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:13:33.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:13:33.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:13:33.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:13:33.262 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.262 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.262 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.263 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.263 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:33.263 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.719340688000003, 2024-09-13T09:13:33.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:13:33.263 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.263 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.263 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.263 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.263 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.263 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:13:33.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.263 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:33.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:33.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:33.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:13:33.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:13:33.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:13:33.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:33.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:13:33.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:13:33.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:13:33.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:13:33.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:13:33.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:13:33.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:13:33.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:13:33.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:13:33.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:13:33.267 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.267 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.267 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.268 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.268 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:33.268 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.719311027000003, 2024-09-13T09:13:33.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:13:33.268 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.268 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.268 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.268 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.268 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.268 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:13:33.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.269 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:33.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:33.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:33.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:13:33.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:13:33.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:13:33.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:13:33.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:13:33.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:13:33.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:13:33.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:13:33.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:13:33.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:13:33.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:13:33.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:13:33.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:13:33.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:13:33.273 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.273 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.273 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.273 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.273 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:33.273 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.719281819999999, 2024-09-13T09:13:33.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:13:33.273 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.273 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.274 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.274 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.274 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.274 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:13:33.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.274 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:33.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:33.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:33.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:13:33.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:13:33.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:13:33.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:13:33.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:13:33.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:13:33.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:13:33.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:13:33.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:13:33.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:13:33.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:13:33.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:13:33.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:13:33.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:13:33.278 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.278 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.278 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.278 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.719225639000001, 2024-09-13T09:13:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:13:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:13:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:33.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:33.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:33.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:13:33.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:13:33.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:13:33.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:13:33.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:13:33.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:13:33.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:13:33.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:13:33.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:13:33.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:13:33.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:13:33.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:13:33.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:13:33.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:13:33.283 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.283 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.283 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.283 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.283 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:33.284 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.719172178000001, 2024-09-13T09:13:33.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:13:33.284 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.284 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.284 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.284 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.284 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.284 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:13:33.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.284 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:33.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:33.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:13:33.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:13:33.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:13:33.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:13:33.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:13:33.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:13:33.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:13:33.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:13:33.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:13:33.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:13:33.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:13:33.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:13:33.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:13:33.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:13:33.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:13:33.288 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.288 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.288 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.288 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.288 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:33.288 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.719115348000003, 2024-09-13T09:13:33.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:13:33.288 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.288 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.288 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.288 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.289 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.289 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:13:33.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.289 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:33.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:33.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:13:33.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:13:33.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:13:33.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:13:33.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:13:33.292 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.292 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.292 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.292 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.293 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:33.293 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.719064320999998, 2024-09-13T09:13:33.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:13:33.293 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.293 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.293 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.293 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.293 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.293 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:13:33.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.293 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:13:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:13:33.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:13:33.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:13:33.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:13:33.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:13:33.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:13:33.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:13:33.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:13:33.297 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.297 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.297 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.297 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.297 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:33.297 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.719023131, 2024-09-13T09:13:33.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:13:33.297 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.297 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.297 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.297 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:13:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:33.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:13:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:13:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:13:33.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:13:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:13:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:13:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:13:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:13:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:13:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:13:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:13:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:13:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:13:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:13:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.718981014999997, 2024-09-13T09:13:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:13:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:13:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:13:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:13:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:13:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:13:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:13:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:13:33.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:13:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:13:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:13:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:13:33.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:13:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:13:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:13:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:13:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.718937136000001, 2024-09-13T09:13:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:13:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:13:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:33.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:13:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:13:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:13:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:13:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:13:33.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:13:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:13:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:13:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:13:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:13:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:13:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:13:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:13:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:13:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.311 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.718891219, 2024-09-13T09:13:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:13:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:13:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:13:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:13:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:13:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:13:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:13:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:13:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:13:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:13:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:13:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:13:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:13:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:13:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:13:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:13:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:13:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.718841447999999, 2024-09-13T09:13:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:13:33.316 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:13:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:13:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:13:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:13:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:13:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:13:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:13:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:13:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:13:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:13:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:13:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:13:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:13:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:13:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:13:33.320 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.718796044000001, 2024-09-13T09:13:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:13:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.321 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:13:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:33.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:13:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:13:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:13:33.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:13:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:13:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:13:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:13:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:13:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:13:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:13:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:13:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:13:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:13:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:13:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.718745405999996, 2024-09-13T09:13:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:13:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:13:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:13:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:13:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:13:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:13:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:13:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:13:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:13:33.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:13:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:13:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:13:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:13:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:13:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:13:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:13:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:13:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:33.330 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.718689562999998, 2024-09-13T09:13:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:13:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:13:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:13:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:13:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:13:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:13:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:13:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:13:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:13:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:13:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:13:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:13:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:13:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:13:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:13:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:13:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.718632640999999, 2024-09-13T09:13:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:13:33.335 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:13:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:13:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:13:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:13:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:13:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:13:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:13:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:13:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:13:33.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:13:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:13:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:13:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:13:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:13:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:13:33.339 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.718579077999998, 2024-09-13T09:13:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:13:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.340 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:13:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:13:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:13:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:13:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:13:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:13:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:13:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:13:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:13:33.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:13:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:13:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:13:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:13:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:13:33.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:13:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.345 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.345 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:33.347 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:33.347 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:33.345+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:33.377 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --force --op remove --pgid 3.18 2024-09-13T09:13:33.448 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:33.447+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:33.494 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:33.494 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:33.494 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:33.494 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:33.502 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:33.547 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:33.546+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:33.610 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone28 from 0xa53fa00 (overlap 0x134a00) to 0x0 2024-09-13T09:13:33.648 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:33.646+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:33.652 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:33.652 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:33.652 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:33.652 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:33.653 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:33.661 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:33.749 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:33.748+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:33.768 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:33.769 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:33.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:33.769 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:33.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.769 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2185 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.769 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:33.769 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.241048357, 2024-09-13T09:13:33.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063348994000000006, 2024-09-13T09:13:33.769 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.769 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.769 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.769 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.770 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.770 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2185 2024-09-13T09:13:33.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.770 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:33.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:33.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:33.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525765+0000", 2024-09-13T09:13:33.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2390000000000001e-06 2024-09-13T09:13:33.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525772+0000", 2024-09-13T09:13:33.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:13:33.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525777+0000", 2024-09-13T09:13:33.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0799999999999996e-06 2024-09-13T09:13:33.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589020+0000", 2024-09-13T09:13:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063242501000000007 2024-09-13T09:13:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589063+0000", 2024-09-13T09:13:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2673999999999998e-05 2024-09-13T09:13:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589112+0000", 2024-09-13T09:13:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9286000000000002e-05 2024-09-13T09:13:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.773 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.774 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.774 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2187 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.774 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:33.774 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.24102881, 2024-09-13T09:13:33.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063500102000000003, 2024-09-13T09:13:33.774 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.774 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.774 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.774 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.774 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.774 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2187 2024-09-13T09:13:33.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.775 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:33.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:33.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:33.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525784+0000", 2024-09-13T09:13:33.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1629999999999999e-06 2024-09-13T09:13:33.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525792+0000", 2024-09-13T09:13:33.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:33.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525796+0000", 2024-09-13T09:13:33.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7389999999999999e-06 2024-09-13T09:13:33.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589132+0000", 2024-09-13T09:13:33.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063336119999999996 2024-09-13T09:13:33.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589187+0000", 2024-09-13T09:13:33.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4126000000000003e-05 2024-09-13T09:13:33.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589283+0000", 2024-09-13T09:13:33.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6032999999999993e-05 2024-09-13T09:13:33.778 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.778 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.778 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.778 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2188 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.778 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:33.778 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.241010763999999, 2024-09-13T09:13:33.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063724617999999997, 2024-09-13T09:13:33.779 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.779 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.779 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.779 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.779 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.779 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2188 2024-09-13T09:13:33.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.779 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:33.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:33.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:33.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525802+0000", 2024-09-13T09:13:33.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0439999999999998e-06 2024-09-13T09:13:33.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525809+0000", 2024-09-13T09:13:33.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.238e-06 2024-09-13T09:13:33.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525814+0000", 2024-09-13T09:13:33.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-13T09:13:33.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589327+0000", 2024-09-13T09:13:33.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063513257000000004 2024-09-13T09:13:33.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589418+0000", 2024-09-13T09:13:33.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0796000000000003e-05 2024-09-13T09:13:33.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589525+0000", 2024-09-13T09:13:33.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010693500000000001 2024-09-13T09:13:33.782 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.782 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.783 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.783 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2190 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.783 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:33.783 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240992678000001, 2024-09-13T09:13:33.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063863981, 2024-09-13T09:13:33.783 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.783 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.783 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.783 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.783 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.783 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2190 2024-09-13T09:13:33.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.784 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:33.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:33.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525818+0000", 2024-09-13T09:13:33.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525820+0000", 2024-09-13T09:13:33.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:13:33.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525827+0000", 2024-09-13T09:13:33.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0589999999999999e-06 2024-09-13T09:13:33.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525831+0000", 2024-09-13T09:13:33.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7870000000000001e-06 2024-09-13T09:13:33.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589562+0000", 2024-09-13T09:13:33.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063731038000000004 2024-09-13T09:13:33.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589612+0000", 2024-09-13T09:13:33.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0087999999999999e-05 2024-09-13T09:13:33.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589683+0000", 2024-09-13T09:13:33.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0121000000000002e-05 2024-09-13T09:13:33.787 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.787 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.787 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.787 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2192 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.787 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:33.787 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240975718000001, 2024-09-13T09:13:33.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064050299000000005, 2024-09-13T09:13:33.788 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.788 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.788 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2192 2024-09-13T09:13:33.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.788 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:33.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:33.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525835+0000", 2024-09-13T09:13:33.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525837+0000", 2024-09-13T09:13:33.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:13:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525845+0000", 2024-09-13T09:13:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3660000000000003e-06 2024-09-13T09:13:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525850+0000", 2024-09-13T09:13:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0030000000000002e-06 2024-09-13T09:13:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589720+0000", 2024-09-13T09:13:33.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063870176000000001 2024-09-13T09:13:33.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589801+0000", 2024-09-13T09:13:33.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0889000000000004e-05 2024-09-13T09:13:33.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589886+0000", 2024-09-13T09:13:33.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5183000000000001e-05 2024-09-13T09:13:33.791 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.791 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.791 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.792 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.792 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:33.792 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240957614999999, 2024-09-13T09:13:33.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:13:33.792 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.792 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.792 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.792 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.792 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.792 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:13:33.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.792 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:33.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:33.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:13:33.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:13:33.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:13:33.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:13:33.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:13:33.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:13:33.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:13:33.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:13:33.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:13:33.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:13:33.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:13:33.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:13:33.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:13:33.796 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.796 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.796 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.796 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.796 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:33.796 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240939916999999, 2024-09-13T09:13:33.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:13:33.796 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.796 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.796 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.796 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.796 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:13:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:13:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:13:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:13:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:13:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:13:33.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:13:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:13:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:13:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240919302000002, 2024-09-13T09:13:33.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:13:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:13:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:13:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:13:33.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:13:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:13:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:13:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:13:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:13:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:13:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:13:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:13:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:13:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:13:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.804 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240900346, 2024-09-13T09:13:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:13:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:13:33.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:13:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:13:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:13:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:13:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:13:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:13:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:13:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:13:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:13:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:13:33.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:13:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:13:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240883088, 2024-09-13T09:13:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:13:33.809 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:13:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:13:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:13:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:13:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:13:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:13:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:13:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:13:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:13:33.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:13:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:13:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:13:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:13:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240863074, 2024-09-13T09:13:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:13:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:13:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:13:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:13:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:13:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:13:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:13:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:13:33.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:13:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:13:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:13:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:13:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:13:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:13:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240837492000001, 2024-09-13T09:13:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:13:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.818 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:13:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:13:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:13:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:13:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:13:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:13:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:13:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:13:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:13:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:13:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:13:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:13:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:13:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:13:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.822 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240818208, 2024-09-13T09:13:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:13:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:13:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:13:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:13:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:13:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:13:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:13:33.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:13:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:13:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:13:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:13:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:13:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:13:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:13:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240800952000001, 2024-09-13T09:13:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:13:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:13:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:13:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:13:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:13:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:13:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:13:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:13:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.831 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240781471999998, 2024-09-13T09:13:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:13:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:13:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:13:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:13:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:13:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:13:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:13:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:13:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:13:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240764543000001, 2024-09-13T09:13:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:13:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:13:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:13:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:13:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:13:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:13:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:13:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:13:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:13:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:13:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:13:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:13:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:13:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:13:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:13:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240745394000001, 2024-09-13T09:13:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:13:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:13:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:13:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:13:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:13:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:13:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:13:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:13:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:13:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:13:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:13:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:13:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:13:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:13:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240727254999999, 2024-09-13T09:13:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:13:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.844 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:13:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:13:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:13:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:13:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:13:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:13:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:13:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:13:33.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:13:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:13:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:13:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:13:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:13:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240316759999999, 2024-09-13T09:13:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:13:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:13:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.849 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:13:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:13:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:13:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:13:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:13:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:13:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:13:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:13:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:13:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:13:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:13:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:13:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.240293859000001, 2024-09-13T09:13:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:13:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:33.853 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:13:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:13:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:13:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:13:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:33.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:13:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:13:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:33.857 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:33.857 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:33.858 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:33.859 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:33.857+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:33.959 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:33.958+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:33.960 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-13T09:13:34.033 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:34.034 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:34.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.034 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1405 3.c 3.c954f9ac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.034 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.027384+0000", 2024-09-13T09:13:34.034 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0037195660000000001, 2024-09-13T09:13:34.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0037424619999999998, 2024-09-13T09:13:34.034 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.034 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.034 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.034 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.034 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.035 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1405 2024-09-13T09:13:34.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.035 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.027384+0000", 2024-09-13T09:13:34.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.027384+0000", 2024-09-13T09:13:34.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.027383+0000", 2024-09-13T09:13:34.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.027386+0000", 2024-09-13T09:13:34.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3989999999999998e-06 2024-09-13T09:13:34.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.027391+0000", 2024-09-13T09:13:34.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3310000000000003e-06 2024-09-13T09:13:34.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.027396+0000", 2024-09-13T09:13:34.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9559999999999996e-06 2024-09-13T09:13:34.037 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.037 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.037 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.037 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1464 3.f 3.8945376f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.038 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.030427+0000", 2024-09-13T09:13:34.038 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00067616099999999999, 2024-09-13T09:13:34.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00076130600000000001, 2024-09-13T09:13:34.038 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.038 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.038 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1464 2024-09-13T09:13:34.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.038 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030427+0000", 2024-09-13T09:13:34.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030427+0000", 2024-09-13T09:13:34.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030427+0000", 2024-09-13T09:13:34.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030429+0000", 2024-09-13T09:13:34.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0829999999999998e-06 2024-09-13T09:13:34.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030436+0000", 2024-09-13T09:13:34.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2729999999999997e-06 2024-09-13T09:13:34.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030441+0000", 2024-09-13T09:13:34.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2939999999999996e-06 2024-09-13T09:13:34.040 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.040 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.041 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.041 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1465 3.c 3.43bb89ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.041 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.030446+0000", 2024-09-13T09:13:34.041 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00065776799999999998, 2024-09-13T09:13:34.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00080068299999999995, 2024-09-13T09:13:34.041 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.041 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.041 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1465 2024-09-13T09:13:34.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.041 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030446+0000", 2024-09-13T09:13:34.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030446+0000", 2024-09-13T09:13:34.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030445+0000", 2024-09-13T09:13:34.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030447+0000", 2024-09-13T09:13:34.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1100000000000001e-06 2024-09-13T09:13:34.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030460+0000", 2024-09-13T09:13:34.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2278e-05 2024-09-13T09:13:34.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030464+0000", 2024-09-13T09:13:34.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.882e-06 2024-09-13T09:13:34.044 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.044 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.044 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.044 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1412 3.c 3.7e5eaa2c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.044 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.028606+0000", 2024-09-13T09:13:34.044 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0024973009999999999, 2024-09-13T09:13:34.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0026947519999999999, 2024-09-13T09:13:34.044 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.044 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.045 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.045 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.045 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.045 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1412 2024-09-13T09:13:34.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.045 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.028606+0000", 2024-09-13T09:13:34.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.028606+0000", 2024-09-13T09:13:34.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.028605+0000", 2024-09-13T09:13:34.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.028609+0000", 2024-09-13T09:13:34.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1409999999999998e-06 2024-09-13T09:13:34.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.028679+0000", 2024-09-13T09:13:34.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9363999999999995e-05 2024-09-13T09:13:34.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.028738+0000", 2024-09-13T09:13:34.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9224000000000001e-05 2024-09-13T09:13:34.047 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.047 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.047 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.047 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1466 3.c 3.c954f9ac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.048 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.030469+0000", 2024-09-13T09:13:34.048 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00063446100000000001, 2024-09-13T09:13:34.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00088699000000000004, 2024-09-13T09:13:34.048 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.048 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.048 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.048 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.048 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.048 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1466 2024-09-13T09:13:34.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.048 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030469+0000", 2024-09-13T09:13:34.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030469+0000", 2024-09-13T09:13:34.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030468+0000", 2024-09-13T09:13:34.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030475+0000", 2024-09-13T09:13:34.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4559999999999997e-06 2024-09-13T09:13:34.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030481+0000", 2024-09-13T09:13:34.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5690000000000001e-06 2024-09-13T09:13:34.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030486+0000", 2024-09-13T09:13:34.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4580000000000003e-06 2024-09-13T09:13:34.051 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.051 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.051 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.051 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1467 3.c 3.c954f9ac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.051 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.030491+0000", 2024-09-13T09:13:34.051 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.000612508, 2024-09-13T09:13:34.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00091956999999999998, 2024-09-13T09:13:34.051 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.051 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.051 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.051 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.052 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.052 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1467 2024-09-13T09:13:34.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.052 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030491+0000", 2024-09-13T09:13:34.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030491+0000", 2024-09-13T09:13:34.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030490+0000", 2024-09-13T09:13:34.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030492+0000", 2024-09-13T09:13:34.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0609999999999998e-06 2024-09-13T09:13:34.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030498+0000", 2024-09-13T09:13:34.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4090000000000002e-06 2024-09-13T09:13:34.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030502+0000", 2024-09-13T09:13:34.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1160000000000001e-06 2024-09-13T09:13:34.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.054 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.054 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1469 3.c 3.d794f92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.054 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.030914+0000", 2024-09-13T09:13:34.054 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00018923099999999999, 2024-09-13T09:13:34.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00055623100000000002, 2024-09-13T09:13:34.055 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.055 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.055 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1469 2024-09-13T09:13:34.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.055 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030914+0000", 2024-09-13T09:13:34.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030914+0000", 2024-09-13T09:13:34.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030913+0000", 2024-09-13T09:13:34.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:13:34.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030918+0000", 2024-09-13T09:13:34.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7310000000000001e-06 2024-09-13T09:13:34.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030931+0000", 2024-09-13T09:13:34.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2835e-05 2024-09-13T09:13:34.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030938+0000", 2024-09-13T09:13:34.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3350000000000001e-06 2024-09-13T09:13:34.057 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.057 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.058 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.058 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1471 3.f 3:f6eca291:::rbd_data.1123c35b2348.0000000000000006:1e [read 483328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.058 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.030944+0000", 2024-09-13T09:13:34.058 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00015918099999999999, 2024-09-13T09:13:34.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00058588900000000003, 2024-09-13T09:13:34.058 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.058 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:34.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.058 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1471 2024-09-13T09:13:34.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.059 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030944+0000", 2024-09-13T09:13:34.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030944+0000", 2024-09-13T09:13:34.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030944+0000", 2024-09-13T09:13:34.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030946+0000", 2024-09-13T09:13:34.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.751e-06 2024-09-13T09:13:34.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030953+0000", 2024-09-13T09:13:34.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6229999999999999e-06 2024-09-13T09:13:34.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030957+0000", 2024-09-13T09:13:34.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4059999999999998e-06 2024-09-13T09:13:34.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.031493+0000", 2024-09-13T09:13:34.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00053594899999999997 2024-09-13T09:13:34.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.061 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.062 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1478 3.f 3.dbc7290f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.062 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.031063+0000", 2024-09-13T09:13:34.062 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.0575999999999999e-05, 2024-09-13T09:13:34.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00052902500000000002, 2024-09-13T09:13:34.062 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.062 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.062 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1478 2024-09-13T09:13:34.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.062 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.031063+0000", 2024-09-13T09:13:34.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.031063+0000", 2024-09-13T09:13:34.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.031062+0000", 2024-09-13T09:13:34.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.031069+0000", 2024-09-13T09:13:34.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7959999999999998e-06 2024-09-13T09:13:34.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.031077+0000", 2024-09-13T09:13:34.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7919999999999993e-06 2024-09-13T09:13:34.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.031082+0000", 2024-09-13T09:13:34.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7420000000000003e-06 2024-09-13T09:13:34.065 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.065 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.065 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.065 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1429 3.c 3.43bb89ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.065 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.029435+0000", 2024-09-13T09:13:34.065 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.001668028, 2024-09-13T09:13:34.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022129049999999998, 2024-09-13T09:13:34.065 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.065 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.066 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.066 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.066 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.066 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1429 2024-09-13T09:13:34.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.066 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029435+0000", 2024-09-13T09:13:34.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029435+0000", 2024-09-13T09:13:34.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029435+0000", 2024-09-13T09:13:34.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029438+0000", 2024-09-13T09:13:34.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8540000000000001e-06 2024-09-13T09:13:34.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029444+0000", 2024-09-13T09:13:34.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7070000000000003e-06 2024-09-13T09:13:34.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029449+0000", 2024-09-13T09:13:34.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0189999999999997e-06 2024-09-13T09:13:34.068 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.068 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.068 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.068 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1482 3.c 3.43bb89ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.068 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.031087+0000", 2024-09-13T09:13:34.069 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.5829999999999999e-05, 2024-09-13T09:13:34.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00061517500000000003, 2024-09-13T09:13:34.069 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.069 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.069 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1482 2024-09-13T09:13:34.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.069 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.031087+0000", 2024-09-13T09:13:34.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.031087+0000", 2024-09-13T09:13:34.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.031087+0000", 2024-09-13T09:13:34.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.031089+0000", 2024-09-13T09:13:34.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7659999999999999e-06 2024-09-13T09:13:34.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.031093+0000", 2024-09-13T09:13:34.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4619999999999998e-06 2024-09-13T09:13:34.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.031098+0000", 2024-09-13T09:13:34.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:13:34.072 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.072 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.072 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.072 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1435 3.c 3.43bb89ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.072 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.029502+0000", 2024-09-13T09:13:34.072 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00160166, 2024-09-13T09:13:34.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022554960000000001, 2024-09-13T09:13:34.072 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.072 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.073 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.073 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1435 2024-09-13T09:13:34.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.073 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029502+0000", 2024-09-13T09:13:34.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029502+0000", 2024-09-13T09:13:34.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029501+0000", 2024-09-13T09:13:34.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029504+0000", 2024-09-13T09:13:34.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:13:34.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029510+0000", 2024-09-13T09:13:34.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.224e-06 2024-09-13T09:13:34.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029515+0000", 2024-09-13T09:13:34.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6290000000000001e-06 2024-09-13T09:13:34.075 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.075 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.076 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.076 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1436 3.c 3.c954f9ac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.076 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.029519+0000", 2024-09-13T09:13:34.076 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.001584352, 2024-09-13T09:13:34.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022918639999999998, 2024-09-13T09:13:34.076 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.076 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.076 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1436 2024-09-13T09:13:34.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.077 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029519+0000", 2024-09-13T09:13:34.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029519+0000", 2024-09-13T09:13:34.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029519+0000", 2024-09-13T09:13:34.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029521+0000", 2024-09-13T09:13:34.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.362e-06 2024-09-13T09:13:34.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029528+0000", 2024-09-13T09:13:34.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0210000000000002e-06 2024-09-13T09:13:34.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029533+0000", 2024-09-13T09:13:34.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6530000000000002e-06 2024-09-13T09:13:34.079 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.079 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.079 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.079 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1438 3.c 3.d794f92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.079 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.029560+0000", 2024-09-13T09:13:34.079 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0015434909999999999, 2024-09-13T09:13:34.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0023071099999999998, 2024-09-13T09:13:34.079 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.079 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.080 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1438 2024-09-13T09:13:34.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.080 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029560+0000", 2024-09-13T09:13:34.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029560+0000", 2024-09-13T09:13:34.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029559+0000", 2024-09-13T09:13:34.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029562+0000", 2024-09-13T09:13:34.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6220000000000001e-06 2024-09-13T09:13:34.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029569+0000", 2024-09-13T09:13:34.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1380000000000001e-06 2024-09-13T09:13:34.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029574+0000", 2024-09-13T09:13:34.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.685e-06 2024-09-13T09:13:34.082 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.082 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.082 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.082 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1443 3.c 3.43bb89ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.082 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.029864+0000", 2024-09-13T09:13:34.082 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.001239022, 2024-09-13T09:13:34.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0020565219999999999, 2024-09-13T09:13:34.083 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.083 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.083 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.083 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.083 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.083 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1443 2024-09-13T09:13:34.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.083 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029864+0000", 2024-09-13T09:13:34.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029864+0000", 2024-09-13T09:13:34.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029864+0000", 2024-09-13T09:13:34.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029886+0000", 2024-09-13T09:13:34.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2405e-05 2024-09-13T09:13:34.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029957+0000", 2024-09-13T09:13:34.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0648999999999994e-05 2024-09-13T09:13:34.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029963+0000", 2024-09-13T09:13:34.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2870000000000001e-06 2024-09-13T09:13:34.085 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.085 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.085 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.086 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1445 3.c 3.d794f92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.086 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.029989+0000", 2024-09-13T09:13:34.086 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.001113979, 2024-09-13T09:13:34.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0019856069999999999, 2024-09-13T09:13:34.086 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.086 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.086 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.086 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.086 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.086 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1445 2024-09-13T09:13:34.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.086 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029989+0000", 2024-09-13T09:13:34.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029989+0000", 2024-09-13T09:13:34.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029989+0000", 2024-09-13T09:13:34.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.029991+0000", 2024-09-13T09:13:34.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4099999999999998e-06 2024-09-13T09:13:34.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030059+0000", 2024-09-13T09:13:34.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7895e-05 2024-09-13T09:13:34.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030065+0000", 2024-09-13T09:13:34.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:13:34.089 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.089 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.089 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.089 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1449 3.c 3.c954f9ac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.089 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.030201+0000", 2024-09-13T09:13:34.089 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00090215200000000003, 2024-09-13T09:13:34.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0018274579999999999, 2024-09-13T09:13:34.089 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.090 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.090 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.090 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.090 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.090 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1449 2024-09-13T09:13:34.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.090 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030201+0000", 2024-09-13T09:13:34.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030201+0000", 2024-09-13T09:13:34.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030201+0000", 2024-09-13T09:13:34.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030204+0000", 2024-09-13T09:13:34.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:13:34.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030210+0000", 2024-09-13T09:13:34.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6279999999999999e-06 2024-09-13T09:13:34.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030216+0000", 2024-09-13T09:13:34.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3240000000000002e-06 2024-09-13T09:13:34.092 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.092 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.092 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.092 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1453 3.c 3.d794f92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.093 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.030252+0000", 2024-09-13T09:13:34.093 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00085139500000000002, 2024-09-13T09:13:34.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001829109, 2024-09-13T09:13:34.093 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.093 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.093 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.093 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.093 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.093 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1453 2024-09-13T09:13:34.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.093 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030252+0000", 2024-09-13T09:13:34.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030252+0000", 2024-09-13T09:13:34.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030252+0000", 2024-09-13T09:13:34.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030254+0000", 2024-09-13T09:13:34.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2919999999999998e-06 2024-09-13T09:13:34.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030260+0000", 2024-09-13T09:13:34.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7579999999999998e-06 2024-09-13T09:13:34.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030264+0000", 2024-09-13T09:13:34.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4850000000000003e-06 2024-09-13T09:13:34.095 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.095 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.095 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.096 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1458 3.c 3.43bb89ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.096 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.030343+0000", 2024-09-13T09:13:34.096 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00076008800000000002, 2024-09-13T09:13:34.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0017839959999999999, 2024-09-13T09:13:34.096 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.096 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.096 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.096 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.096 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.096 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1458 2024-09-13T09:13:34.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.096 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030343+0000", 2024-09-13T09:13:34.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030343+0000", 2024-09-13T09:13:34.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030342+0000", 2024-09-13T09:13:34.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030346+0000", 2024-09-13T09:13:34.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1209999999999998e-06 2024-09-13T09:13:34.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030352+0000", 2024-09-13T09:13:34.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7089999999999996e-06 2024-09-13T09:13:34.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030357+0000", 2024-09-13T09:13:34.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.028e-06 2024-09-13T09:13:34.098 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.098 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.098 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.099 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1459 3.c 3.c954f9ac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.099 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.030362+0000", 2024-09-13T09:13:34.099 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00074103899999999998, 2024-09-13T09:13:34.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001791675, 2024-09-13T09:13:34.099 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.099 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.099 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.099 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.099 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.099 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1459 2024-09-13T09:13:34.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.099 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030362+0000", 2024-09-13T09:13:34.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030362+0000", 2024-09-13T09:13:34.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030362+0000", 2024-09-13T09:13:34.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030364+0000", 2024-09-13T09:13:34.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:13:34.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030370+0000", 2024-09-13T09:13:34.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9370000000000001e-06 2024-09-13T09:13:34.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030374+0000", 2024-09-13T09:13:34.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.211e-06 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5515.0:1462 3.c 3.d794f92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:34.030394+0000", 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00070888600000000004, 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0017864339999999999, 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5515", 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3350755321", 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1462 2024-09-13T09:13:34.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030394+0000", 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030394+0000", 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030394+0000", 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030396+0000", 2024-09-13T09:13:34.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1110000000000002e-06 2024-09-13T09:13:34.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030402+0000", 2024-09-13T09:13:34.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5079999999999996e-06 2024-09-13T09:13:34.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:34.030406+0000", 2024-09-13T09:13:34.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2259999999999999e-06 2024-09-13T09:13:34.104 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.105 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.105 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.105 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.105 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-13T09:13:34.105 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 21 2024-09-13T09:13:34.105 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:34.106 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:34.107 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:34.106+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:34.207 INFO:teuthology.orchestra.run.smithi157.stdout:4863 read 0x60a0e00 thru 0x60a2251 (0x1452 bytes) 2024-09-13T09:13:34.207 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:34.205+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:34.216 INFO:teuthology.orchestra.run.smithi157.stdout:4866 trunc from 0xa53fa00 to 0x3581000 2024-09-13T09:13:34.256 INFO:teuthology.orchestra.run.smithi157.stdout:4867 read 0x30c400 thru 0x310c4c (0x484d bytes) 2024-09-13T09:13:34.256 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:34.257 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:34.257 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:34.257 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:34.257 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:34.266 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:34.276 INFO:teuthology.orchestra.run.smithi157.stdout:4868 trunc from 0x3581000 to 0x9ad6200 2024-09-13T09:13:34.291 INFO:teuthology.orchestra.run.smithi157.stdout:4870 trunc from 0x9ad6200 to 0xceeee00 2024-09-13T09:13:34.305 INFO:teuthology.orchestra.run.smithi157.stdout:4871 read 0x1527400 thru 0x1527c27 (0x828 bytes) 2024-09-13T09:13:34.312 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:34.312+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:34.314 INFO:teuthology.orchestra.run.smithi157.stdout:4872 write 0x2e90a00 thru 0x2e917b8 (0xdb9 bytes) 2024-09-13T09:13:34.314 INFO:teuthology.orchestra.run.smithi157.stdout:4873 read 0x1a45800 thru 0x1a510e8 (0xb8e9 bytes) 2024-09-13T09:13:34.315 INFO:teuthology.orchestra.run.smithi157.stdout:4874 write 0x2a6e00 thru 0x2adb1d (0x6d1e bytes) 2024-09-13T09:13:34.316 INFO:teuthology.orchestra.run.smithi157.stdout:4877 read 0x171e200 thru 0x171edad (0xbae bytes) 2024-09-13T09:13:34.329 INFO:teuthology.orchestra.run.smithi157.stdout:4879 trunc from 0xceeee00 to 0x181b800 2024-09-13T09:13:34.357 INFO:teuthology.orchestra.run.smithi157.stdout:4882 read 0xd70c00 thru 0xd7b7c0 (0xabc1 bytes) 2024-09-13T09:13:34.365 INFO:teuthology.orchestra.run.smithi157.stdout:4883 read 0x2b7e00 thru 0x2bd413 (0x5614 bytes) 2024-09-13T09:13:34.365 INFO:teuthology.orchestra.run.smithi157.stdout:4884 read 0x1363c00 thru 0x13690ed (0x54ee bytes) 2024-09-13T09:13:34.366 INFO:teuthology.orchestra.run.smithi157.stdout:4886 write 0x934200 thru 0x939547 (0x5348 bytes) 2024-09-13T09:13:34.381 INFO:teuthology.orchestra.run.smithi157.stdout:4888 trunc from 0x181b800 to 0xaf5a800 2024-09-13T09:13:34.382 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:34.382 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:34.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:34.383 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:34.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.383 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9035 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2342912~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.383 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:34.383 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.633179855000002, 2024-09-13T09:13:34.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070293606999999994, 2024-09-13T09:13:34.383 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.383 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.384 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9035 2024-09-13T09:13:34.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.384 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:34.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:34.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:34.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745602+0000", 2024-09-13T09:13:34.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-06 2024-09-13T09:13:34.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745607+0000", 2024-09-13T09:13:34.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0770000000000001e-06 2024-09-13T09:13:34.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745609+0000", 2024-09-13T09:13:34.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3499999999999999e-06 2024-09-13T09:13:34.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815626+0000", 2024-09-13T09:13:34.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070017398999999994 2024-09-13T09:13:34.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815672+0000", 2024-09-13T09:13:34.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5275999999999998e-05 2024-09-13T09:13:34.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815895+0000", 2024-09-13T09:13:34.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022328300000000001 2024-09-13T09:13:34.387 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.387 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.387 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.387 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.387 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:34.388 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.633167311000001, 2024-09-13T09:13:34.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:13:34.388 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.388 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.388 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:13:34.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.388 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:34.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:34.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:34.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:13:34.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:34.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:13:34.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:13:34.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:13:34.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:13:34.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:13:34.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:13:34.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:13:34.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:13:34.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:13:34.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:13:34.392 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.392 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.392 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.392 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.392 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:34.392 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.633158633000001, 2024-09-13T09:13:34.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:13:34.392 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.392 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.392 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:13:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:34.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:13:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:13:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:13:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:13:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:13:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:13:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:13:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:13:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:13:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:13:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:13:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:13:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.396 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.633149647000003, 2024-09-13T09:13:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:13:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:13:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:13:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:13:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:13:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:13:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:13:34.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:13:34.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:13:34.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:13:34.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:13:34.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:13:34.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:13:34.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:13:34.401 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.401 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.401 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.401 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.401 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:34.401 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.633141674999997, 2024-09-13T09:13:34.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:13:34.401 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.401 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.402 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.402 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.402 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:13:34.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.402 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:34.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:34.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:34.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:34.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:13:34.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:13:34.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:13:34.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:13:34.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:13:34.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:13:34.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:13:34.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:13:34.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:13:34.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:13:34.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:13:34.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:13:34.405 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.405 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.405 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.406 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.406 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:34.406 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.631266840999999, 2024-09-13T09:13:34.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:13:34.406 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.406 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.406 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:13:34.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.406 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:34.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:34.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:34.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:13:34.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:13:34.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:13:34.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:13:34.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:13:34.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:13:34.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:13:34.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:13:34.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:13:34.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:13:34.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:13:34.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:13:34.410 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.410 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.410 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.410 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.410 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:34.410 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.631256180000001, 2024-09-13T09:13:34.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:13:34.410 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.410 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.411 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.411 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.411 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.411 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:13:34.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.411 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:34.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:34.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:34.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:34.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:13:34.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:13:34.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:13:34.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:13:34.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:13:34.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:13:34.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:13:34.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:13:34.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:13:34.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:13:34.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:13:34.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:13:34.414 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.414 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.414 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.415 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.415 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:34.415 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.631234204000002, 2024-09-13T09:13:34.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:13:34.415 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.415 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.415 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:13:34.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.415 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:34.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:34.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:34.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:13:34.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:13:34.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:13:34.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:13:34.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:13:34.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:13:34.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:13:34.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:13:34.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:13:34.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:13:34.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:13:34.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:13:34.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.419 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.419 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.419 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:34.419 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.631220702, 2024-09-13T09:13:34.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:13:34.419 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.419 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.420 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:13:34.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.420 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:34.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:34.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:34.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:13:34.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:13:34.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:13:34.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:13:34.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:13:34.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:13:34.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:13:34.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:13:34.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:13:34.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:13:34.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:13:34.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:13:34.423 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.423 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.423 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.423 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9056 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 7159808~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.423 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:34.424 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.629219872999997, 2024-09-13T09:13:34.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070283126000000001, 2024-09-13T09:13:34.424 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.424 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.424 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.424 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.424 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.424 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9056 2024-09-13T09:13:34.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.424 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:34.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:34.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:34.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:34.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749562+0000", 2024-09-13T09:13:34.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1340000000000001e-06 2024-09-13T09:13:34.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749565+0000", 2024-09-13T09:13:34.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8760000000000001e-06 2024-09-13T09:13:34.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749567+0000", 2024-09-13T09:13:34.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2340000000000001e-06 2024-09-13T09:13:34.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819750+0000", 2024-09-13T09:13:34.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070182537000000003 2024-09-13T09:13:34.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819792+0000", 2024-09-13T09:13:34.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2141999999999997e-05 2024-09-13T09:13:34.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819844+0000", 2024-09-13T09:13:34.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2428e-05 2024-09-13T09:13:34.427 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.428 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.428 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.428 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.428 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:34.428 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.629211603999998, 2024-09-13T09:13:34.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:13:34.428 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.428 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.428 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:13:34.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.429 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:34.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:34.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:34.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:13:34.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:13:34.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:13:34.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:13:34.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:13:34.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:13:34.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:13:34.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:13:34.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:13:34.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:13:34.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:13:34.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:13:34.432 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.432 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.432 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.432 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.432 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:34.433 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.629187281999997, 2024-09-13T09:13:34.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:13:34.433 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.433 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.433 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.433 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.433 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.433 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:13:34.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.433 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:34.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:34.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:34.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:13:34.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:13:34.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:13:34.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:13:34.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:13:34.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:13:34.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:13:34.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:13:34.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:13:34.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:13:34.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:13:34.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:13:34.437 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.437 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.437 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.437 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.437 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:34.437 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.629176307999998, 2024-09-13T09:13:34.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:13:34.437 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.437 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.438 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.438 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.438 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.438 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:13:34.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.438 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:34.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:34.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:13:34.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:13:34.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:13:34.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:13:34.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:13:34.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:13:34.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:13:34.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:13:34.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:13:34.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:13:34.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:13:34.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:13:34.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:13:34.441 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.441 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.441 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.442 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9063 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1589248~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.442 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:34.442 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.627272666000003, 2024-09-13T09:13:34.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070391924999999994, 2024-09-13T09:13:34.442 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.442 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.442 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.442 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.442 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.442 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9063 2024-09-13T09:13:34.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.442 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:34.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:34.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:34.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:34.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751509+0000", 2024-09-13T09:13:34.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.142e-06 2024-09-13T09:13:34.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751513+0000", 2024-09-13T09:13:34.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6669999999999998e-06 2024-09-13T09:13:34.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751520+0000", 2024-09-13T09:13:34.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5910000000000001e-06 2024-09-13T09:13:34.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821593+0000", 2024-09-13T09:13:34.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073519000000001 2024-09-13T09:13:34.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821648+0000", 2024-09-13T09:13:34.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:13:34.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821900+0000", 2024-09-13T09:13:34.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025263999999999999 2024-09-13T09:13:34.446 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.446 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.446 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.446 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.446 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:34.446 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.627258683000001, 2024-09-13T09:13:34.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:13:34.446 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.446 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.447 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:13:34.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.447 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:34.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:34.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:34.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:34.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:13:34.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:13:34.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:13:34.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:34.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:13:34.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:13:34.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:13:34.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:13:34.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:13:34.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:13:34.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:13:34.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:13:34.450 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.450 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.450 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.451 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.451 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:34.451 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.627249286000001, 2024-09-13T09:13:34.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:13:34.451 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.451 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.451 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.451 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.451 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.451 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:13:34.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.451 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:34.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:34.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:34.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:34.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:13:34.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:13:34.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:13:34.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:13:34.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:13:34.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:13:34.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:13:34.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:13:34.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:13:34.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:13:34.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:13:34.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:13:34.455 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.455 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.455 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.455 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9071 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:34.455 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:34.455 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.625266269000001, 2024-09-13T09:13:34.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070386377999999999, 2024-09-13T09:13:34.455 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.455 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.455 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.456 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:34.456 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:34.456 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9071 2024-09-13T09:13:34.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.456 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:34.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:34.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:34.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:34.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753516+0000", 2024-09-13T09:13:34.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1519999999999999e-06 2024-09-13T09:13:34.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753519+0000", 2024-09-13T09:13:34.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6559999999999998e-06 2024-09-13T09:13:34.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753522+0000", 2024-09-13T09:13:34.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:13:34.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823798+0000", 2024-09-13T09:13:34.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070275907999999998 2024-09-13T09:13:34.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823845+0000", 2024-09-13T09:13:34.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7593000000000003e-05 2024-09-13T09:13:34.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823901+0000", 2024-09-13T09:13:34.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5934999999999999e-05 2024-09-13T09:13:34.459 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.459 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.459 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.459 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3181 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T09:13:34.459 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:34.459 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.3568851100000003, 2024-09-13T09:13:34.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.480957603, 2024-09-13T09:13:34.460 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.460 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:34.460 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.460 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:13:34.460 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:13:34.460 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3181 2024-09-13T09:13:34.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.460 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:34.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:34.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021894+0000", 2024-09-13T09:13:34.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:13:34.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021903+0000", 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4109999999999996e-06 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021919+0000", 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6225000000000001e-05 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021926+0000", 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4120000000000004e-06 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021975+0000", 2024-09-13T09:13:34.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8526000000000003e-05 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022049+0000", 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4018999999999997e-05 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022230+0000", 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018085599999999999 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022796+0000", 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056638999999999995 2024-09-13T09:13:34.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499540+0000", 2024-09-13T09:13:34.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.47674342800000002 2024-09-13T09:13:34.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:34.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499547+0000", 2024-09-13T09:13:34.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:13:34.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502794+0000", 2024-09-13T09:13:34.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.003246818 2024-09-13T09:13:34.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502854+0000", 2024-09-13T09:13:34.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9925e-05 2024-09-13T09:13:34.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.465 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.465 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3182 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 139818970156544 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:13:34.465 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:34.465 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.883650555, 2024-09-13T09:13:34.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0089058390000001, 2024-09-13T09:13:34.466 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.466 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:13:34.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:13:34.466 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3182 2024-09-13T09:13:34.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.466 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:34.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:34.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495128+0000", 2024-09-13T09:13:34.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:13:34.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495139+0000", 2024-09-13T09:13:34.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0599e-05 2024-09-13T09:13:34.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495157+0000", 2024-09-13T09:13:34.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7830999999999999e-05 2024-09-13T09:13:34.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495167+0000", 2024-09-13T09:13:34.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0689999999999999e-05 2024-09-13T09:13:34.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495229+0000", 2024-09-13T09:13:34.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1898999999999998e-05 2024-09-13T09:13:34.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:13:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495243+0000", 2024-09-13T09:13:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3349e-05 2024-09-13T09:13:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499575+0000", 2024-09-13T09:13:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043324410000000002 2024-09-13T09:13:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499582+0000", 2024-09-13T09:13:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0849999999999999e-06 2024-09-13T09:13:34.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502872+0000", 2024-09-13T09:13:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0032895150000001 2024-09-13T09:13:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503016+0000", 2024-09-13T09:13:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014394200000000001 2024-09-13T09:13:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504036+0000", 2024-09-13T09:13:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010208680000000001 2024-09-13T09:13:34.470 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:19151 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 140383089882784 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:13:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.883639096, 2024-09-13T09:13:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.009071287, 2024-09-13T09:13:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:13:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:13:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19151 2024-09-13T09:13:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495140+0000", 2024-09-13T09:13:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:13:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495149+0000", 2024-09-13T09:13:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0759999999999992e-06 2024-09-13T09:13:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495165+0000", 2024-09-13T09:13:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5322e-05 2024-09-13T09:13:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495173+0000", 2024-09-13T09:13:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0979999999999994e-06 2024-09-13T09:13:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495318+0000", 2024-09-13T09:13:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014553 2024-09-13T09:13:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:13:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495325+0000", 2024-09-13T09:13:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:13:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499598+0000", 2024-09-13T09:13:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004273064 2024-09-13T09:13:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499602+0000", 2024-09-13T09:13:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8539999999999999e-06 2024-09-13T09:13:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503507+0000", 2024-09-13T09:13:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0039045520000001 2024-09-13T09:13:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503583+0000", 2024-09-13T09:13:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6283000000000001e-05 2024-09-13T09:13:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504213+0000", 2024-09-13T09:13:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00063044700000000002 2024-09-13T09:13:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:34.477 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:34.479 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:34.476+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:34.480 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:34.481 INFO:teuthology.orchestra.run.smithi157.stdout:4890 read 0x60ee000 thru 0x60f3a44 (0x5a45 bytes) 2024-09-13T09:13:34.481 INFO:teuthology.orchestra.run.smithi157.stdout:4891 read 0x70b5200 thru 0x70bf4a4 (0xa2a5 bytes) 2024-09-13T09:13:34.481 INFO:teuthology.orchestra.run.smithi157.stdout:4893 write 0x6570000 thru 0x657c0b7 (0xc0b8 bytes) 2024-09-13T09:13:34.481 INFO:teuthology.orchestra.run.smithi157.stdout:4895 trunc from 0xaf5a800 to 0x2619e00 2024-09-13T09:13:34.481 INFO:teuthology.orchestra.run.smithi157.stdout:4896 read 0x13a5200 thru 0x13b45fc (0xf3fd bytes) 2024-09-13T09:13:34.481 INFO:teuthology.orchestra.run.smithi157.stdout:4900 read 0x1d15e00 thru 0x1d1dfc1 (0x81c2 bytes) 2024-09-13T09:13:34.482 INFO:teuthology.orchestra.run.smithi157.stdout:4901 read 0x1847e00 thru 0x184d851 (0x5a52 bytes) 2024-09-13T09:13:34.482 INFO:teuthology.orchestra.run.smithi157.stdout:4902 write 0xb097400 thru 0xb09eb8e (0x778f bytes) 2024-09-13T09:13:34.482 INFO:teuthology.orchestra.run.smithi157.stdout:4905 read 0x5bacc00 thru 0x5bb1b8c (0x4f8d bytes) 2024-09-13T09:13:34.482 INFO:teuthology.orchestra.run.smithi157.stdout:4906 trunc from 0xb09ec00 to 0xbcbd000 2024-09-13T09:13:34.482 INFO:teuthology.orchestra.run.smithi157.stdout:4907 write 0x3a41600 thru 0x3a4707d (0x5a7e bytes) 2024-09-13T09:13:34.482 INFO:teuthology.orchestra.run.smithi157.stdout:4909 punch from 0xa84a600 to 0xa84d800, (0x3200 bytes) 2024-09-13T09:13:34.482 INFO:teuthology.orchestra.run.smithi157.stdout:4911 punch from 0x5723000 to 0x572da00, (0xaa00 bytes) 2024-09-13T09:13:34.482 INFO:teuthology.orchestra.run.smithi157.stdout:4914 write 0xbe3d400 thru 0xbe4bf4d (0xeb4e bytes) 2024-09-13T09:13:34.482 INFO:teuthology.orchestra.run.smithi157.stdout:4915 read 0x2b99000 thru 0x2b9dc35 (0x4c36 bytes) 2024-09-13T09:13:34.482 INFO:teuthology.orchestra.run.smithi157.stdout:4916 read 0x806c600 thru 0x8070454 (0x3e55 bytes) 2024-09-13T09:13:34.482 INFO:teuthology.orchestra.run.smithi157.stdout:4917 punch from 0x3a8000 to 0x3b0000, (0x8000 bytes) 2024-09-13T09:13:34.484 INFO:teuthology.orchestra.run.smithi157.stdout:4919 punch from 0x1976200 to 0x197f200, (0x9000 bytes) 2024-09-13T09:13:34.484 INFO:teuthology.orchestra.run.smithi157.stdout:4920 write 0x9b84200 thru 0x9b88b4b (0x494c bytes) 2024-09-13T09:13:34.484 INFO:teuthology.orchestra.run.smithi157.stdout:4922 write 0x34ccc00 thru 0x34d17bf (0x4bc0 bytes) 2024-09-13T09:13:34.484 INFO:teuthology.orchestra.run.smithi157.stdout:4923 read 0x5fc5a00 thru 0x5fcd882 (0x7e83 bytes) 2024-09-13T09:13:34.484 INFO:teuthology.orchestra.run.smithi157.stdout:4924 read 0x4a5c00 thru 0x4af602 (0x9a03 bytes) 2024-09-13T09:13:34.484 INFO:teuthology.orchestra.run.smithi157.stdout:4925 read 0x46db200 thru 0x46dfd86 (0x4b87 bytes) 2024-09-13T09:13:34.484 INFO:teuthology.orchestra.run.smithi157.stdout:4927 read 0x3532400 thru 0x353f453 (0xd054 bytes) 2024-09-13T09:13:34.484 INFO:teuthology.orchestra.run.smithi157.stdout:4929 write 0xa4be400 thru 0xa4c21be (0x3dbf bytes) 2024-09-13T09:13:34.499 INFO:teuthology.orchestra.run.smithi157.stdout:4931 write 0xbeb8e00 thru 0xbebff12 (0x7113 bytes) 2024-09-13T09:13:34.499 INFO:teuthology.orchestra.run.smithi157.stdout:4932 punch from 0x61f2c00 to 0x61fe600, (0xba00 bytes) 2024-09-13T09:13:34.504 INFO:teuthology.orchestra.run.smithi157.stdout:4934 write 0xa82cc00 thru 0xa830d55 (0x4156 bytes) 2024-09-13T09:13:34.505 INFO:teuthology.orchestra.run.smithi157.stdout:4935 read 0x129f200 thru 0x12a6dcc (0x7bcd bytes) 2024-09-13T09:13:34.506 INFO:teuthology.orchestra.run.smithi157.stdout:4936 punch from 0x397b400 to 0x397b800, (0x400 bytes) 2024-09-13T09:13:34.506 INFO:teuthology.orchestra.run.smithi157.stdout:4939 punch from 0x89c3a00 to 0x89c7c00, (0x4200 bytes) 2024-09-13T09:13:34.507 INFO:teuthology.orchestra.run.smithi157.stdout:4941 trunc from 0xbec0000 to 0xa8f8a00 2024-09-13T09:13:34.526 INFO:teuthology.orchestra.run.smithi044.stdout: marking collection for removal 2024-09-13T09:13:34.526 INFO:teuthology.orchestra.run.smithi044.stdout:setting '_remove' omap key 2024-09-13T09:13:34.527 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:13:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:finish_remove_pgs 3.18_head removing 3.18 2024-09-13T09:13:34.534 INFO:teuthology.orchestra.run.smithi044.stdout:Remove successful 2024-09-13T09:13:34.535 INFO:teuthology.orchestra.run.smithi157.stdout:4942 read 0xa6d400 thru 0xa72103 (0x4d04 bytes) 2024-09-13T09:13:34.535 INFO:teuthology.orchestra.run.smithi157.stdout:4943 write 0x162ba00 thru 0x162fea6 (0x44a7 bytes) 2024-09-13T09:13:34.536 INFO:teuthology.orchestra.run.smithi157.stdout:4944 trunc from 0xa8f8a00 to 0xa8f7200 2024-09-13T09:13:34.561 INFO:teuthology.orchestra.run.smithi157.stdout:4946 trunc from 0xa8f7200 to 0x8c4d600 2024-09-13T09:13:34.579 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:34.579+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:34.585 INFO:teuthology.orchestra.run.smithi157.stdout:4947 trunc from 0x8c4d600 to 0x6245800 2024-09-13T09:13:34.606 INFO:teuthology.orchestra.run.smithi157.stdout:4950 read 0x5bd9000 thru 0x5bdf89f (0x68a0 bytes) 2024-09-13T09:13:34.608 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:34.609 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:34.616 INFO:teuthology.orchestra.run.smithi157.stdout:4952 write 0x7103a00 thru 0x7104fcc (0x15cd bytes) 2024-09-13T09:13:34.618 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:34.621 INFO:teuthology.orchestra.run.smithi157.stdout:4953 punch from 0x23d6e00 to 0x23da000, (0x3200 bytes) 2024-09-13T09:13:34.628 INFO:teuthology.orchestra.run.smithi157.stdout:4954 trunc from 0x7105000 to 0xcb34a00 2024-09-13T09:13:34.642 INFO:teuthology.orchestra.run.smithi157.stdout:4955 punch from 0x9157e00 to 0x9162200, (0xa400 bytes) 2024-09-13T09:13:34.645 INFO:teuthology.orchestra.run.smithi157.stdout:4960 read 0x5fcc600 thru 0x5fda030 (0xda31 bytes) 2024-09-13T09:13:34.645 INFO:teuthology.orchestra.run.smithi157.stdout:4962 read 0xb94ca00 thru 0xb958b79 (0xc17a bytes) 2024-09-13T09:13:34.653 INFO:teuthology.orchestra.run.smithi157.stdout:4963 trunc from 0xcb34a00 to 0x6178200 2024-09-13T09:13:34.668 INFO:teuthology.orchestra.run.smithi157.stdout:4964 punch from 0x1c76800 to 0x1c7e200, (0x7a00 bytes) 2024-09-13T09:13:34.671 INFO:teuthology.orchestra.run.smithi157.stdout:4967 trunc from 0x6178200 to 0x54bf200 2024-09-13T09:13:34.680 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:34.679+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:34.701 INFO:teuthology.orchestra.run.smithi157.stdout:4968 write 0x8716800 thru 0x8718977 (0x2178 bytes) 2024-09-13T09:13:34.711 INFO:teuthology.orchestra.run.smithi157.stdout:4970 trunc from 0x8718a00 to 0xb94000 2024-09-13T09:13:34.755 INFO:teuthology.orchestra.run.smithi157.stdout:4973 trunc from 0xb94000 to 0xa2d9c00 2024-09-13T09:13:34.767 INFO:teuthology.orchestra.run.smithi157.stdout:4974 write 0xaec00 thru 0xb7557 (0x8958 bytes) 2024-09-13T09:13:34.769 INFO:teuthology.orchestra.run.smithi157.stdout:4975 write 0x496d600 thru 0x496e4c3 (0xec4 bytes) 2024-09-13T09:13:34.769 INFO:teuthology.orchestra.run.smithi157.stdout:4976 write 0x4b92600 thru 0x4b94be0 (0x25e1 bytes) 2024-09-13T09:13:34.769 INFO:teuthology.orchestra.run.smithi157.stdout:4978 write 0x5485000 thru 0x5485167 (0x168 bytes) 2024-09-13T09:13:34.769 INFO:teuthology.orchestra.run.smithi157.stdout:4980 read 0x8297200 thru 0x82a0dbe (0x9bbf bytes) 2024-09-13T09:13:34.781 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:34.778+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:34.785 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:34.785 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:34.791 INFO:teuthology.orchestra.run.smithi157.stdout:4983 write 0xd80f800 thru 0xd81118e (0x198f bytes) 2024-09-13T09:13:34.791 INFO:teuthology.orchestra.run.smithi157.stdout:4984 read 0x5f06600 thru 0x5f10cbc (0xa6bd bytes) 2024-09-13T09:13:34.792 INFO:teuthology.orchestra.run.smithi157.stdout:4986 punch from 0x74bbe00 to 0x74ca000, (0xe200 bytes) 2024-09-13T09:13:34.795 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:34.796 INFO:teuthology.orchestra.run.smithi157.stdout:4987 read 0x9475800 thru 0x9482fde (0xd7df bytes) 2024-09-13T09:13:34.796 INFO:teuthology.orchestra.run.smithi157.stdout:4988 write 0x8384c00 thru 0x838df8a (0x938b bytes) 2024-09-13T09:13:34.796 INFO:teuthology.orchestra.run.smithi157.stdout:4989 punch from 0x797b200 to 0x7982c00, (0x7a00 bytes) 2024-09-13T09:13:34.796 INFO:teuthology.orchestra.run.smithi157.stdout:4990 write 0x6a4bc00 thru 0x6a50779 (0x4b7a bytes) 2024-09-13T09:13:34.796 INFO:teuthology.orchestra.run.smithi157.stdout:4993 read 0xf88200 thru 0xf952be (0xd0bf bytes) 2024-09-13T09:13:34.804 INFO:teuthology.orchestra.run.smithi157.stdout:4996 trunc from 0xd811200 to 0x66e8600 2024-09-13T09:13:34.826 INFO:teuthology.orchestra.run.smithi157.stdout:4997 read 0x3ef9200 thru 0x3efddfd (0x4bfe bytes) 2024-09-13T09:13:34.882 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:34.882+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:34.943 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:34.943 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:34.952 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:34.996 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:34.994+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:35.043 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.18.2 2024-09-13T09:13:35.058 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:35.059 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:35.067 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:35.084 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:35.084+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:35.108 INFO:teuthology.orchestra.run.smithi157.stdout:4998 write 0xd765800 thru 0xd7680b8 (0x28b9 bytes) 2024-09-13T09:13:35.113 INFO:teuthology.orchestra.run.smithi157.stdout:4999 trunc from 0xd768200 to 0x9a16e00 2024-09-13T09:13:35.148 INFO:teuthology.orchestra.run.smithi157.stdout:5000 trunc from 0x9a16e00 to 0x1974200 2024-09-13T09:13:35.168 INFO:teuthology.orchestra.run.smithi157.stdout:5001 read 0x12f7e00 thru 0x12fa953 (0x2b54 bytes) 2024-09-13T09:13:35.168 INFO:teuthology.orchestra.run.smithi157.stdout:5003 read 0xac6600 thru 0xac8eaa (0x28ab bytes) 2024-09-13T09:13:35.168 INFO:teuthology.orchestra.run.smithi157.stdout:5006 punch from 0x10fd400 to 0x1107800, (0xa400 bytes) 2024-09-13T09:13:35.184 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:35.182+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:35.187 INFO:teuthology.orchestra.run.smithi157.stdout:5007 write 0x7d97400 thru 0x7da03f5 (0x8ff6 bytes) 2024-09-13T09:13:35.187 INFO:teuthology.orchestra.run.smithi157.stdout:5008 read 0x3524c00 thru 0x352c9cc (0x7dcd bytes) 2024-09-13T09:13:35.194 INFO:teuthology.orchestra.run.smithi157.stdout:5009 trunc from 0x7da0400 to 0x23f8400 2024-09-13T09:13:35.212 INFO:teuthology.orchestra.run.smithi157.stdout:5011 read 0x65ce00 thru 0x6674b6 (0xa6b7 bytes) 2024-09-13T09:13:35.213 INFO:teuthology.orchestra.run.smithi157.stdout:5013 read 0x1da8200 thru 0x1db1829 (0x962a bytes) 2024-09-13T09:13:35.213 INFO:teuthology.orchestra.run.smithi157.stdout:5015 read 0x146b400 thru 0x1479dda (0xe9db bytes) 2024-09-13T09:13:35.224 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:35.224 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:35.227 INFO:teuthology.orchestra.run.smithi157.stdout:5017 trunc from 0x23f8400 to 0xdcda400 2024-09-13T09:13:35.233 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:35.237 INFO:teuthology.orchestra.run.smithi157.stdout:5018 read 0x706f000 thru 0x707c0ad (0xd0ae bytes) 2024-09-13T09:13:35.237 INFO:teuthology.orchestra.run.smithi157.stdout:5019 write 0xc5ddc00 thru 0xc5ebec6 (0xe2c7 bytes) 2024-09-13T09:13:35.238 INFO:teuthology.orchestra.run.smithi157.stdout:5020 trunc from 0xdcda400 to 0xce29800 2024-09-13T09:13:35.253 INFO:teuthology.orchestra.run.smithi157.stdout:5022 write 0xc1c3a00 thru 0xc1cbfa6 (0x85a7 bytes) 2024-09-13T09:13:35.264 INFO:teuthology.orchestra.run.smithi157.stdout:5026 write 0xdf5fa00 thru 0xdf62eec (0x34ed bytes) 2024-09-13T09:13:35.268 INFO:teuthology.orchestra.run.smithi157.stdout:5027 write 0x7ed6600 thru 0x7ed89ee (0x23ef bytes) 2024-09-13T09:13:35.268 INFO:teuthology.orchestra.run.smithi157.stdout:5029 read 0x2819600 thru 0x281eb29 (0x552a bytes) 2024-09-13T09:13:35.268 INFO:teuthology.orchestra.run.smithi157.stdout:5032 write 0x17ff200 thru 0x180ef75 (0xfd76 bytes) 2024-09-13T09:13:35.268 INFO:teuthology.orchestra.run.smithi157.stdout:5034 write 0xdc19a00 thru 0xdc21bbd (0x81be bytes) 2024-09-13T09:13:35.268 INFO:teuthology.orchestra.run.smithi157.stdout:5035 read 0xd2aae00 thru 0xd2b4acc (0x9ccd bytes) 2024-09-13T09:13:35.268 INFO:teuthology.orchestra.run.smithi157.stdout:5036 write 0x8de0e00 thru 0x8debad0 (0xacd1 bytes) 2024-09-13T09:13:35.268 INFO:teuthology.orchestra.run.smithi157.stdout:5038 punch from 0x9f90000 to 0x9f94800, (0x4800 bytes) 2024-09-13T09:13:35.269 INFO:teuthology.orchestra.run.smithi157.stdout:5042 write 0xb657800 thru 0xb6595a9 (0x1daa bytes) 2024-09-13T09:13:35.269 INFO:teuthology.orchestra.run.smithi157.stdout:5044 write 0x988d800 thru 0x989a8db (0xd0dc bytes) 2024-09-13T09:13:35.269 INFO:teuthology.orchestra.run.smithi157.stdout:5047 punch from 0x1c7200 to 0x1c7800, (0x600 bytes) 2024-09-13T09:13:35.269 INFO:teuthology.orchestra.run.smithi157.stdout:5048 punch from 0xa628c00 to 0xa630800, (0x7c00 bytes) 2024-09-13T09:13:35.269 INFO:teuthology.orchestra.run.smithi157.stdout:5051 read 0x2c3e200 thru 0x2c4d445 (0xf246 bytes) 2024-09-13T09:13:35.283 INFO:teuthology.orchestra.run.smithi157.stdout:5052 trunc from 0xdf63000 to 0x2790c00 2024-09-13T09:13:35.306 INFO:teuthology.orchestra.run.smithi157.stdout:5053 read 0x1dd3600 thru 0x1dd88e4 (0x52e5 bytes) 2024-09-13T09:13:35.317 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:35.315+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:35.330 INFO:teuthology.orchestra.run.smithi157.stdout:5056 write 0x95afc00 thru 0x95b0049 (0x44a bytes) 2024-09-13T09:13:35.330 INFO:teuthology.orchestra.run.smithi157.stdout:5058 write 0x583be00 thru 0x583cd0d (0xf0e bytes) 2024-09-13T09:13:35.330 INFO:teuthology.orchestra.run.smithi157.stdout:5059 read 0xf6bc00 thru 0xf7aa2a (0xee2b bytes) 2024-09-13T09:13:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:13:35.354 INFO:teuthology.orchestra.run.smithi157.stdout:5060 write 0xe595200 thru 0xe59e4e0 (0x92e1 bytes) 2024-09-13T09:13:35.355 INFO:teuthology.orchestra.run.smithi157.stdout:5061 read 0x53d6e00 thru 0x53de4be (0x76bf bytes) 2024-09-13T09:13:35.355 INFO:teuthology.orchestra.run.smithi157.stdout:5065 read 0xac4200 thru 0xaca96e (0x676f bytes) 2024-09-13T09:13:35.355 INFO:teuthology.orchestra.run.smithi157.stdout:5066 write 0x72d2a00 thru 0x72df2d8 (0xc8d9 bytes) 2024-09-13T09:13:35.358 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:35.358 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:35.368 INFO:teuthology.orchestra.run.smithi157.stdout:5067 trunc from 0xe59e600 to 0x387c400 2024-09-13T09:13:35.375 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:35.376 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:35.386 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:35.384+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:35.397 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:35.397 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:35.398 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:35.398 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:35.401 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:13:35.402 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:13:35.408 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:35.487 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:35.486+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:35.512 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:35.512 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:35.521 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:35.533 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-09-13T09:13:35.589 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:35.586+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:35.661 INFO:teuthology.orchestra.run.smithi157.stdout:5069 write 0x4fe1a00 thru 0x4feae60 (0x9461 bytes) 2024-09-13T09:13:35.664 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:35.664 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:35.674 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:35.683 INFO:teuthology.orchestra.run.smithi157.stdout:5070 write 0x6d6da00 thru 0x6d76844 (0x8e45 bytes) 2024-09-13T09:13:35.683 INFO:teuthology.orchestra.run.smithi157.stdout:5071 punch from 0x2a05a00 to 0x2a08c00, (0x3200 bytes) 2024-09-13T09:13:35.687 INFO:teuthology.orchestra.run.smithi157.stdout:5073 trunc from 0x6d76a00 to 0x9899a00 2024-09-13T09:13:35.704 INFO:teuthology.orchestra.run.smithi157.stdout:5075 punch from 0x91eb400 to 0x91fa600, (0xf200 bytes) 2024-09-13T09:13:35.704 INFO:teuthology.orchestra.run.smithi157.stdout:5077 write 0x695aa00 thru 0x695c711 (0x1d12 bytes) 2024-09-13T09:13:35.717 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:35.717+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:35.719 INFO:teuthology.orchestra.run.smithi157.stdout:5078 write 0xbe29e00 thru 0xbe2d7bf (0x39c0 bytes) 2024-09-13T09:13:35.719 INFO:teuthology.orchestra.run.smithi157.stdout:5079 write 0x44dac00 thru 0x44e19c8 (0x6dc9 bytes) 2024-09-13T09:13:35.719 INFO:teuthology.orchestra.run.smithi157.stdout:5081 write 0x743a000 thru 0x744166e (0x766f bytes) 2024-09-13T09:13:35.719 INFO:teuthology.orchestra.run.smithi157.stdout:5082 punch from 0x64b000 to 0x65a200, (0xf200 bytes) 2024-09-13T09:13:35.720 INFO:teuthology.orchestra.run.smithi157.stdout:5084 trunc from 0xbe2d800 to 0xcaf5800 2024-09-13T09:13:35.733 INFO:teuthology.orchestra.run.smithi157.stdout:5085 write 0x25b1000 thru 0x25bf1ca (0xe1cb bytes) 2024-09-13T09:13:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:5086 read 0xb335a00 thru 0xb33f807 (0x9e08 bytes) 2024-09-13T09:13:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:5092 read 0x196d600 thru 0x196e994 (0x1395 bytes) 2024-09-13T09:13:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:5094 write 0xae1b400 thru 0xae2aae0 (0xf6e1 bytes) 2024-09-13T09:13:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:5095 read 0xa146600 thru 0xa14e7b1 (0x81b2 bytes) 2024-09-13T09:13:35.735 INFO:teuthology.orchestra.run.smithi157.stdout:5097 read 0xae0c00 thru 0xaee0c0 (0xd4c1 bytes) 2024-09-13T09:13:35.738 INFO:teuthology.orchestra.run.smithi157.stdout:5098 read 0x8a61600 thru 0x8a62d57 (0x1758 bytes) 2024-09-13T09:13:35.738 INFO:teuthology.orchestra.run.smithi157.stdout:5101 write 0xccf600 thru 0xcd409f (0x4aa0 bytes) 2024-09-13T09:13:35.739 INFO:teuthology.orchestra.run.smithi157.stdout:5105 read 0xb31a600 thru 0xb321fe0 (0x79e1 bytes) 2024-09-13T09:13:35.739 INFO:teuthology.orchestra.run.smithi157.stdout:5107 read 0x67ad400 thru 0x67b77d6 (0xa3d7 bytes) 2024-09-13T09:13:35.757 INFO:teuthology.orchestra.run.smithi157.stdout:5108 write 0xeaafc00 thru 0xeab5a3a (0x5e3b bytes) 2024-09-13T09:13:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:5111 read 0xcb4ec00 thru 0xcb54765 (0x5b66 bytes) 2024-09-13T09:13:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:5112 write 0xcf35000 thru 0xcf3ef6b (0x9f6c bytes) 2024-09-13T09:13:35.765 INFO:teuthology.orchestra.run.smithi157.stdout:5113 write 0xce1e00 thru 0xce763b (0x583c bytes) 2024-09-13T09:13:35.766 INFO:teuthology.orchestra.run.smithi157.stdout:5115 punch from 0x7815800 to 0x7824400, (0xec00 bytes) 2024-09-13T09:13:35.766 INFO:teuthology.orchestra.run.smithi157.stdout:5116 write 0x500200 thru 0x508ff9 (0x8dfa bytes) 2024-09-13T09:13:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:5117 write 0x7890000 thru 0x789123d (0x123e bytes) 2024-09-13T09:13:35.767 INFO:teuthology.orchestra.run.smithi157.stdout:5118 read 0x94a000 thru 0x94caa5 (0x2aa6 bytes) 2024-09-13T09:13:35.768 INFO:teuthology.orchestra.run.smithi157.stdout:5119 write 0x97ae600 thru 0x97aff75 (0x1976 bytes) 2024-09-13T09:13:35.786 INFO:teuthology.orchestra.run.smithi157.stdout:5122 write 0xeb3d000 thru 0xeb4c2a4 (0xf2a5 bytes) 2024-09-13T09:13:35.786 INFO:teuthology.orchestra.run.smithi157.stdout:5124 punch from 0x5163200 to 0x516f800, (0xc600 bytes) 2024-09-13T09:13:35.789 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:35.787+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:35.791 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:35.792 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:35.800 INFO:teuthology.orchestra.run.smithi157.stdout:5125 trunc from 0xeb4c400 to 0x3a1a000 2024-09-13T09:13:35.801 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:35.827 INFO:teuthology.orchestra.run.smithi157.stdout:5126 read 0x28b8e00 thru 0x28bc8fe (0x3aff bytes) 2024-09-13T09:13:35.871 INFO:teuthology.orchestra.run.smithi157.stdout:5127 punch from 0x2a86e00 to 0x2a96200, (0xf400 bytes) 2024-09-13T09:13:35.871 INFO:teuthology.orchestra.run.smithi157.stdout:5130 read 0xa0a600 thru 0xa13225 (0x8c26 bytes) 2024-09-13T09:13:35.871 INFO:teuthology.orchestra.run.smithi157.stdout:5131 read 0x2e8400 thru 0x2f14e8 (0x90e9 bytes) 2024-09-13T09:13:35.871 INFO:teuthology.orchestra.run.smithi157.stdout:5134 read 0x1866800 thru 0x186d53a (0x6d3b bytes) 2024-09-13T09:13:35.871 INFO:teuthology.orchestra.run.smithi157.stdout:5135 trunc from 0x3a1a000 to 0xb860400 2024-09-13T09:13:35.871 INFO:teuthology.orchestra.run.smithi157.stdout:5136 clone 30 order 23 su 1048576 sc 9 2024-09-13T09:13:35.890 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:35.888+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:35.971 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:35.971 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:35.979 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:35.991 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:35.989+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:36.081 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:36.082 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:36.090 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:36.092 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:36.090+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:36.161 INFO:teuthology.orchestra.run.smithi044.stdout:get_pg_num_history pg_num_history pg_num_history(e84 pg_nums {1={13=1},2={16=8},3={21=32}} deleted_pools ) 2024-09-13T09:13:36.161 INFO:teuthology.orchestra.run.smithi044.stdout:Importing pgid 3.18 2024-09-13T09:13:36.161 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:18750c32:::rbd_data.1123af309369.0000000000000038:1f# 2024-09-13T09:13:36.162 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:18750c32:::rbd_data.1123af309369.0000000000000038:head# 2024-09-13T09:13:36.162 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 1f=[]:{1f=[1f]} 2024-09-13T09:13:36.162 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:188b27a2:::rbd_data.1123ee68625c.0000000000000023:head# 2024-09-13T09:13:36.162 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 14=[]:{} 2024-09-13T09:13:36.163 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:18fa2bd0:::rbd_header.1123a0dcea4a:head# 2024-09-13T09:13:36.163 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 0=[]:{} 2024-09-13T09:13:36.163 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:190c77f9:::rbd_data.1123af309369.00000000000000b4:1f# 2024-09-13T09:13:36.164 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:190c77f9:::rbd_data.1123af309369.00000000000000b4:head# 2024-09-13T09:13:36.164 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 1f=[]:{1f=[1f]} 2024-09-13T09:13:36.164 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:193bbcf1:::rbd_id.image_client.0-clone20:head# 2024-09-13T09:13:36.164 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 0=[]:{} 2024-09-13T09:13:36.165 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:193f53c0:::rbd_data.11233da12af.000000000000000c:head# 2024-09-13T09:13:36.165 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 0=[]:{} 2024-09-13T09:13:36.167 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1958c3bf:::rbd_data.112326fb43d3.0000000000000038:11# 2024-09-13T09:13:36.167 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1958c3bf:::rbd_data.112326fb43d3.0000000000000038:head# 2024-09-13T09:13:36.167 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 11=[]:{11=[11]} 2024-09-13T09:13:36.168 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:196dfe53:::rbd_data.1123ee68625c.0000000000000020:head# 2024-09-13T09:13:36.168 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 14=[]:{} 2024-09-13T09:13:36.168 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1981c38e:::rbd_data.1123c35b2348.0000000000000039:head# 2024-09-13T09:13:36.168 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 0=[]:{} 2024-09-13T09:13:36.169 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:19fd4e96:::rbd_data.11233da12af.0000000000000023:head# 2024-09-13T09:13:36.170 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 0=[]:{} 2024-09-13T09:13:36.171 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1a35bbab:::rbd_data.1123af309369.0000000000000182:1f# 2024-09-13T09:13:36.172 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1a35bbab:::rbd_data.1123af309369.0000000000000182:head# 2024-09-13T09:13:36.172 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 1f=[]:{1f=[1f]} 2024-09-13T09:13:36.173 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1a99e484:::rbd_data.1123ee68625c.0000000000000025:head# 2024-09-13T09:13:36.173 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 14=[]:{} 2024-09-13T09:13:36.173 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1a9b4417:::rbd_data.1123af309369.0000000000000231:1f# 2024-09-13T09:13:36.173 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1a9b4417:::rbd_data.1123af309369.0000000000000231:head# 2024-09-13T09:13:36.173 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 1f=[]:{1f=[1f]} 2024-09-13T09:13:36.174 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1b0fc23a:::rbd_data.1123af309369.0000000000000035:1f# 2024-09-13T09:13:36.175 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1b0fc23a:::rbd_data.1123af309369.0000000000000035:head# 2024-09-13T09:13:36.175 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 1f=[]:{1f=[1f]} 2024-09-13T09:13:36.175 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1b8df09d:::rbd_data.1123b8b4a28a.000000000000003c:head# 2024-09-13T09:13:36.175 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 0=[]:{} 2024-09-13T09:13:36.177 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1bb498e5:::rbd_data.1123b428d19a.0000000000000019:16# 2024-09-13T09:13:36.178 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1bb498e5:::rbd_data.1123b428d19a.0000000000000019:head# 2024-09-13T09:13:36.178 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 16=[]:{16=[16]} 2024-09-13T09:13:36.178 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1be76995:::rbd_data.1123c35b2348.0000000000000007:head# 2024-09-13T09:13:36.178 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 0=[]:{} 2024-09-13T09:13:36.179 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1bf4adc9:::rbd_data.1123af309369.0000000000000158:1f# 2024-09-13T09:13:36.182 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1bf4adc9:::rbd_data.1123af309369.0000000000000158:head# 2024-09-13T09:13:36.182 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 1f=[]:{1f=[1f]} 2024-09-13T09:13:36.184 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1bfc759a:::rbd_data.1123276d9426.0000000000000011:19# 2024-09-13T09:13:36.185 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1bfc759a:::rbd_data.1123276d9426.0000000000000011:head# 2024-09-13T09:13:36.185 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 19=[]:{19=[19]} 2024-09-13T09:13:36.186 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1c152610:::rbd_data.11239759ab94.0000000000000085:1b# 2024-09-13T09:13:36.187 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1c152610:::rbd_data.11239759ab94.0000000000000085:head# 2024-09-13T09:13:36.187 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 1b=[]:{1b=[1b]} 2024-09-13T09:13:36.187 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1c248fb4:::rbd_data.1123ee68625c.000000000000002a:head# 2024-09-13T09:13:36.188 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 14=[]:{} 2024-09-13T09:13:36.188 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1c27f267:::rbd_data.11239759ab94.0000000000000014:1b# 2024-09-13T09:13:36.189 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1c27f267:::rbd_data.11239759ab94.0000000000000014:head# 2024-09-13T09:13:36.189 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 1b=[]:{1b=[1b]} 2024-09-13T09:13:36.189 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1c607c64:::rbd_data.1123276d9426.000000000000001a:19# 2024-09-13T09:13:36.192 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:36.192 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:36.192 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:36.192+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:36.197 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1c607c64:::rbd_data.1123276d9426.000000000000001a:head# 2024-09-13T09:13:36.197 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 19=[]:{19=[19]} 2024-09-13T09:13:36.197 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1c92b5b7:::rbd_data.1123b8d418eb.0000000000000008:b# 2024-09-13T09:13:36.199 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1c92b5b7:::rbd_data.1123b8d418eb.0000000000000008:head# 2024-09-13T09:13:36.199 INFO:teuthology.orchestra.run.smithi044.stdout:snapset b=[]:{b=[b]} 2024-09-13T09:13:36.200 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1e0409d3:::rbd_header.11239c07d29f:head# 2024-09-13T09:13:36.200 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 0=[]:{} 2024-09-13T09:13:36.201 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:36.202 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1e8b6a6d:::rbd_header.112360c1ab8:head# 2024-09-13T09:13:36.202 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 0=[]:{} 2024-09-13T09:13:36.202 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1f50bcb3:::rbd_data.1123b428d19a.0000000000000005:16# 2024-09-13T09:13:36.204 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1f50bcb3:::rbd_data.1123b428d19a.0000000000000005:head# 2024-09-13T09:13:36.204 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 16=[]:{16=[16]} 2024-09-13T09:13:36.205 INFO:teuthology.orchestra.run.smithi044.stdout:Write #3:1f517812:::rbd_data.1123ee68625c.000000000000002c:head# 2024-09-13T09:13:36.205 INFO:teuthology.orchestra.run.smithi044.stdout:snapset 14=[]:{} 2024-09-13T09:13:36.205 INFO:teuthology.orchestra.run.smithi044.stdout:write_pg epoch 83 info 3.18( v 83'380 (0'0,83'380] local-lis/les=21/22 n=42 ec=21/21 lis/c=21/21 les/c/f=22/22/0 sis=21) 2024-09-13T09:13:36.207 INFO:teuthology.orchestra.run.smithi044.stdout:Import successful 2024-09-13T09:13:36.293 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:36.291+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:36.344 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:36.344 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:36.352 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:36.396 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:36.396+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:36.456 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:36.456 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:36.464 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:36.506 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:36.505+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:36.595 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:36.594+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:36.614 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:36.614 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:36.624 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:36.632 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone29 from 0xb860400 (overlap 0xb94000) to 0x0 2024-09-13T09:13:36.696 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:36.695+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:36.737 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /var/log/ceph/exp.3.18.2 2024-09-13T09:13:36.745 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:36.745 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:36.755 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:36.808 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:36.805+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:36.897 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:36.897+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:36.914 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:36.914 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:36.923 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:36.962 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-13T09:13:36.999 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:36.998+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:37.018 INFO:teuthology.orchestra.run.smithi157.stdout:5137 read 0x8790800 thru 0x879e216 (0xda17 bytes) 2024-09-13T09:13:37.019 INFO:teuthology.orchestra.run.smithi157.stdout:5138 punch from 0x1939000 to 0x1941400, (0x8400 bytes) 2024-09-13T09:13:37.030 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:37.030 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:37.033 INFO:teuthology.orchestra.run.smithi157.stdout:5139 read 0x798200 thru 0x79b226 (0x3027 bytes) 2024-09-13T09:13:37.034 INFO:teuthology.orchestra.run.smithi157.stdout:5141 read 0x22ce200 thru 0x22d8d76 (0xab77 bytes) 2024-09-13T09:13:37.034 INFO:teuthology.orchestra.run.smithi157.stdout:5143 read 0x9359200 thru 0x9365af2 (0xc8f3 bytes) 2024-09-13T09:13:37.039 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:37.045 INFO:teuthology.orchestra.run.smithi157.stdout:5144 write 0xc4ea400 thru 0xc4eb125 (0xd26 bytes) 2024-09-13T09:13:37.046 INFO:teuthology.orchestra.run.smithi157.stdout:5146 write 0x4b3dc00 thru 0x4b4a80b (0xcc0c bytes) 2024-09-13T09:13:37.046 INFO:teuthology.orchestra.run.smithi157.stdout:5148 read 0xa979000 thru 0xa982d6f (0x9d70 bytes) 2024-09-13T09:13:37.046 INFO:teuthology.orchestra.run.smithi157.stdout:5149 read 0x3e77a00 thru 0x3e794d8 (0x1ad9 bytes) 2024-09-13T09:13:37.046 INFO:teuthology.orchestra.run.smithi157.stdout:5150 write 0x2dd1a00 thru 0x2dd9f0b (0x850c bytes) 2024-09-13T09:13:37.046 INFO:teuthology.orchestra.run.smithi157.stdout:5151 punch from 0x762800 to 0x767c00, (0x5400 bytes) 2024-09-13T09:13:37.050 INFO:teuthology.orchestra.run.smithi157.stdout:5153 punch from 0x2891400 to 0x289c200, (0xae00 bytes) 2024-09-13T09:13:37.050 INFO:teuthology.orchestra.run.smithi157.stdout:5155 punch from 0x14c7c00 to 0x14d1000, (0x9400 bytes) 2024-09-13T09:13:37.050 INFO:teuthology.orchestra.run.smithi157.stdout:5156 read 0x351b800 thru 0x3526a03 (0xb204 bytes) 2024-09-13T09:13:37.063 INFO:teuthology.orchestra.run.smithi157.stdout:5157 trunc from 0xc4eb200 to 0x21bfa00 2024-09-13T09:13:37.087 INFO:teuthology.orchestra.run.smithi157.stdout:5160 punch from 0x1a6e200 to 0x1a70600, (0x2400 bytes) 2024-09-13T09:13:37.087 INFO:teuthology.orchestra.run.smithi157.stdout:5161 write 0x2127e00 thru 0x212823e (0x43f bytes) 2024-09-13T09:13:37.100 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:37.099+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:37.111 INFO:teuthology.orchestra.run.smithi157.stdout:5162 write 0xdb73c00 thru 0xdb7d8bb (0x9cbc bytes) 2024-09-13T09:13:37.126 INFO:teuthology.orchestra.run.smithi157.stdout:5166 write 0xee13c00 thru 0xee22fa6 (0xf3a7 bytes) 2024-09-13T09:13:37.127 INFO:teuthology.orchestra.run.smithi157.stdout:5169 read 0x8401600 thru 0x8407daf (0x67b0 bytes) 2024-09-13T09:13:37.127 INFO:teuthology.orchestra.run.smithi157.stdout:5171 write 0xb5aa800 thru 0xb5ad9ed (0x31ee bytes) 2024-09-13T09:13:37.127 INFO:teuthology.orchestra.run.smithi157.stdout:5172 read 0xcdc9a00 thru 0xcdd5304 (0xb905 bytes) 2024-09-13T09:13:37.127 INFO:teuthology.orchestra.run.smithi157.stdout:5173 punch from 0x2a08e00 to 0x2a14a00, (0xbc00 bytes) 2024-09-13T09:13:37.128 INFO:teuthology.orchestra.run.smithi157.stdout:5175 write 0x27d7400 thru 0x27d7a80 (0x681 bytes) 2024-09-13T09:13:37.128 INFO:teuthology.orchestra.run.smithi157.stdout:5177 punch from 0x56f2400 to 0x56f5600, (0x3200 bytes) 2024-09-13T09:13:37.133 INFO:teuthology.orchestra.run.smithi157.stdout:5180 trunc from 0xee23000 to 0xb0cc600 2024-09-13T09:13:37.152 INFO:teuthology.orchestra.run.smithi157.stdout:5182 trunc from 0xb0cc600 to 0x9cf4c00 2024-09-13T09:13:37.153 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:37.153 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:37.162 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:37.166 INFO:teuthology.orchestra.run.smithi157.stdout:5183 read 0x3df6600 thru 0x3e004e4 (0x9ee5 bytes) 2024-09-13T09:13:37.166 INFO:teuthology.orchestra.run.smithi157.stdout:5185 read 0x329dc00 thru 0x32ac67f (0xea80 bytes) 2024-09-13T09:13:37.167 INFO:teuthology.orchestra.run.smithi157.stdout:5191 read 0x3fbfa00 thru 0x3fcdf19 (0xe51a bytes) 2024-09-13T09:13:37.167 INFO:teuthology.orchestra.run.smithi157.stdout:5192 write 0x2511c00 thru 0x2514c85 (0x3086 bytes) 2024-09-13T09:13:37.168 INFO:teuthology.orchestra.run.smithi157.stdout:5197 trunc from 0x9cf4c00 to 0xa1e5c00 2024-09-13T09:13:37.178 INFO:teuthology.orchestra.run.smithi157.stdout:5199 punch from 0x5b4da00 to 0x5b5d600, (0xfc00 bytes) 2024-09-13T09:13:37.179 INFO:teuthology.orchestra.run.smithi157.stdout:5200 write 0x8116600 thru 0x81173ac (0xdad bytes) 2024-09-13T09:13:37.182 INFO:teuthology.orchestra.run.smithi157.stdout:5203 trunc from 0xa1e5c00 to 0xd32fc00 2024-09-13T09:13:37.193 INFO:teuthology.orchestra.run.smithi157.stdout:5204 write 0x2abac00 thru 0x2abae83 (0x284 bytes) 2024-09-13T09:13:37.200 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:37.199+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:37.207 INFO:teuthology.orchestra.run.smithi157.stdout:5206 trunc from 0xd32fc00 to 0x219ce00 2024-09-13T09:13:37.244 INFO:teuthology.orchestra.run.smithi157.stdout:5208 write 0x70de600 thru 0x70e56f4 (0x70f5 bytes) 2024-09-13T09:13:37.247 INFO:teuthology.orchestra.run.smithi157.stdout:5209 trunc from 0x70e5800 to 0x9bb7c00 2024-09-13T09:13:37.264 INFO:teuthology.orchestra.run.smithi157.stdout:skipping zero length punch hole 2024-09-13T09:13:37.264 INFO:teuthology.orchestra.run.smithi157.stdout:5213 read 0x6856a00 thru 0x6862f40 (0xc541 bytes) 2024-09-13T09:13:37.264 INFO:teuthology.orchestra.run.smithi157.stdout:5215 punch from 0x1770000 to 0x1777000, (0x7000 bytes) 2024-09-13T09:13:37.264 INFO:teuthology.orchestra.run.smithi157.stdout:5216 read 0x724a800 thru 0x7251ee2 (0x76e3 bytes) 2024-09-13T09:13:37.264 INFO:teuthology.orchestra.run.smithi157.stdout:5217 read 0x2bb0800 thru 0x2bbc2cf (0xbad0 bytes) 2024-09-13T09:13:37.265 INFO:teuthology.orchestra.run.smithi157.stdout:5218 read 0x290d200 thru 0x29112f7 (0x40f8 bytes) 2024-09-13T09:13:37.266 INFO:teuthology.orchestra.run.smithi157.stdout:5220 read 0x7351c00 thru 0x735717b (0x557c bytes) 2024-09-13T09:13:37.266 INFO:teuthology.orchestra.run.smithi157.stdout:5221 read 0x28f3e00 thru 0x2901be8 (0xdde9 bytes) 2024-09-13T09:13:37.266 INFO:teuthology.orchestra.run.smithi157.stdout:5228 punch from 0x954dc00 to 0x9551200, (0x3600 bytes) 2024-09-13T09:13:37.266 INFO:teuthology.orchestra.run.smithi157.stdout:5230 read 0x58b9a00 thru 0x58c6758 (0xcd59 bytes) 2024-09-13T09:13:37.271 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:37.271 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:37.271 INFO:teuthology.orchestra.run.smithi157.stdout:5234 trunc from 0x9bb7c00 to 0x5ad9000 2024-09-13T09:13:37.280 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:37.289 INFO:teuthology.orchestra.run.smithi157.stdout:5237 read 0x399e600 thru 0x39a3347 (0x4d48 bytes) 2024-09-13T09:13:37.289 INFO:teuthology.orchestra.run.smithi157.stdout:5238 punch from 0x41a9000 to 0x41aa000, (0x1000 bytes) 2024-09-13T09:13:37.294 INFO:teuthology.orchestra.run.smithi157.stdout:5239 trunc from 0x5ad9000 to 0x1ca0600 2024-09-13T09:13:37.301 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:37.301+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:37.334 INFO:teuthology.orchestra.run.smithi157.stdout:5241 write 0x6e54a00 thru 0x6e5a8a6 (0x5ea7 bytes) 2024-09-13T09:13:37.334 INFO:teuthology.orchestra.run.smithi157.stdout:5242 read 0x479c00 thru 0x479da0 (0x1a1 bytes) 2024-09-13T09:13:37.334 INFO:teuthology.orchestra.run.smithi157.stdout:5244 read 0x5a6bc00 thru 0x5a72d7e (0x717f bytes) 2024-09-13T09:13:37.334 INFO:teuthology.orchestra.run.smithi157.stdout:5245 punch from 0x4d33400 to 0x4d35400, (0x2000 bytes) 2024-09-13T09:13:37.334 INFO:teuthology.orchestra.run.smithi157.stdout:5247 write 0x324bc00 thru 0x3255309 (0x970a bytes) 2024-09-13T09:13:37.334 INFO:teuthology.orchestra.run.smithi157.stdout:5248 read 0xb7be00 thru 0xb83d80 (0x7f81 bytes) 2024-09-13T09:13:37.337 INFO:teuthology.orchestra.run.smithi157.stdout:5254 trunc from 0x6e5aa00 to 0x84b9c00 2024-09-13T09:13:37.351 INFO:teuthology.orchestra.run.smithi157.stdout:5257 read 0x2fed600 thru 0x2ff1f2d (0x492e bytes) 2024-09-13T09:13:37.373 INFO:teuthology.orchestra.run.smithi157.stdout:5260 write 0xe8ef800 thru 0xe8f5588 (0x5d89 bytes) 2024-09-13T09:13:37.388 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:37.388 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:37.389 INFO:teuthology.orchestra.run.smithi157.stdout:5262 trunc from 0xe8f5600 to 0x24e8600 2024-09-13T09:13:37.397 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:37.401 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:37.399+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:37.403 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:13:37.414 INFO:teuthology.orchestra.run.smithi157.stdout:5264 read 0x1ae9800 thru 0x1af6018 (0xc819 bytes) 2024-09-13T09:13:37.418 INFO:teuthology.orchestra.run.smithi157.stdout:5265 trunc from 0x24e8600 to 0x509c200 2024-09-13T09:13:37.456 INFO:teuthology.orchestra.run.smithi157.stdout:5266 write 0xd8e5c00 thru 0xd8f4572 (0xe973 bytes) 2024-09-13T09:13:37.456 INFO:teuthology.orchestra.run.smithi157.stdout:5267 read 0x869da00 thru 0x86a087d (0x2e7e bytes) 2024-09-13T09:13:37.456 INFO:teuthology.orchestra.run.smithi157.stdout:5268 write 0x3fbd000 thru 0x3fca0a1 (0xd0a2 bytes) 2024-09-13T09:13:37.472 INFO:teuthology.orchestra.run.smithi157.stdout:5269 trunc from 0xd8f4600 to 0xfa7c00 2024-09-13T09:13:37.499 INFO:teuthology.orchestra.run.smithi157.stdout:5271 trunc from 0xfa7c00 to 0x2b72600 2024-09-13T09:13:37.502 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:37.502+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:37.511 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:37.511 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:37.513 INFO:teuthology.orchestra.run.smithi157.stdout:5272 punch from 0x1ccdc00 to 0x1ccee00, (0x1200 bytes) 2024-09-13T09:13:37.521 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:37.526 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:37.526 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:37.541 INFO:teuthology.orchestra.run.smithi157.stdout:5273 write 0xdff1600 thru 0xdff57fa (0x41fb bytes) 2024-09-13T09:13:37.542 INFO:teuthology.orchestra.run.smithi157.stdout:5274 write 0x1d8ea00 thru 0x1d8f268 (0x869 bytes) 2024-09-13T09:13:37.542 INFO:teuthology.orchestra.run.smithi157.stdout:5277 read 0xc76f800 thru 0xc77905c (0x985d bytes) 2024-09-13T09:13:37.542 INFO:teuthology.orchestra.run.smithi157.stdout:5278 read 0x3669600 thru 0x366f8f1 (0x62f2 bytes) 2024-09-13T09:13:37.543 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:37.543 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:37.544 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:13:37.547 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:13:37.551 INFO:teuthology.orchestra.run.smithi157.stdout:5279 trunc from 0xdff5800 to 0x7351800 2024-09-13T09:13:37.562 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:37.562 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:37.574 INFO:teuthology.orchestra.run.smithi157.stdout:5284 punch from 0x2cc5600 to 0x2cc9e00, (0x4800 bytes) 2024-09-13T09:13:37.580 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:37.580 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:37.596 INFO:teuthology.orchestra.run.smithi157.stdout:5285 write 0xc67b600 thru 0xc6835b4 (0x7fb5 bytes) 2024-09-13T09:13:37.597 INFO:teuthology.orchestra.run.smithi157.stdout:5286 write 0x9965c00 thru 0x9974da1 (0xf1a2 bytes) 2024-09-13T09:13:37.597 INFO:teuthology.orchestra.run.smithi157.stdout:5287 punch from 0x999fe00 to 0x99aa600, (0xa800 bytes) 2024-09-13T09:13:37.598 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:37.598 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:37.598 INFO:teuthology.orchestra.run.smithi157.stdout:5288 write 0xb722600 thru 0xb72d6f8 (0xb0f9 bytes) 2024-09-13T09:13:37.599 INFO:teuthology.orchestra.run.smithi157.stdout:5294 write 0x2015a00 thru 0x20203dd (0xa9de bytes) 2024-09-13T09:13:37.599 INFO:teuthology.orchestra.run.smithi157.stdout:5299 write 0xc055c00 thru 0xc05fd38 (0xa139 bytes) 2024-09-13T09:13:37.599 INFO:teuthology.orchestra.run.smithi157.stdout:5300 punch from 0x429aa00 to 0x42a8600, (0xdc00 bytes) 2024-09-13T09:13:37.599 INFO:teuthology.orchestra.run.smithi157.stdout:5302 punch from 0x4e93400 to 0x4e9b800, (0x8400 bytes) 2024-09-13T09:13:37.599 INFO:teuthology.orchestra.run.smithi157.stdout:5305 read 0x694aa00 thru 0x694ce46 (0x2447 bytes) 2024-09-13T09:13:37.599 INFO:teuthology.orchestra.run.smithi157.stdout:5311 punch from 0x1a3e200 to 0x1a4be00, (0xdc00 bytes) 2024-09-13T09:13:37.601 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:13:37.601 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:13:37.606 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:37.606+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:37.621 INFO:teuthology.orchestra.run.smithi157.stdout:5312 write 0xdcaa600 thru 0xdcab277 (0xc78 bytes) 2024-09-13T09:13:37.621 INFO:teuthology.orchestra.run.smithi157.stdout:5313 read 0xae27a00 thru 0xae32361 (0xa962 bytes) 2024-09-13T09:13:37.636 INFO:teuthology.orchestra.run.smithi157.stdout:5314 trunc from 0xdcab400 to 0x1bea400 2024-09-13T09:13:37.677 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:37.677 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:37.677 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:37.677 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:37.678 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:37.682 INFO:teuthology.orchestra.run.smithi157.stdout:5315 write 0x50f6c00 thru 0x5102ba1 (0xbfa2 bytes) 2024-09-13T09:13:37.682 INFO:teuthology.orchestra.run.smithi157.stdout:5318 read 0x193ee00 thru 0x194c28d (0xd48e bytes) 2024-09-13T09:13:37.683 INFO:teuthology.orchestra.run.smithi157.stdout:5319 read 0x44c3200 thru 0x44c7d41 (0x4b42 bytes) 2024-09-13T09:13:37.683 INFO:teuthology.orchestra.run.smithi157.stdout:5323 read 0x497a400 thru 0x498273f (0x8340 bytes) 2024-09-13T09:13:37.683 INFO:teuthology.orchestra.run.smithi157.stdout:5325 read 0x1a54c00 thru 0x1a5af7b (0x637c bytes) 2024-09-13T09:13:37.684 INFO:teuthology.orchestra.run.smithi157.stdout:5328 write 0x122b000 thru 0x1238589 (0xd58a bytes) 2024-09-13T09:13:37.684 INFO:teuthology.orchestra.run.smithi157.stdout:5329 write 0x30a7000 thru 0x30b48df (0xd8e0 bytes) 2024-09-13T09:13:37.684 INFO:teuthology.orchestra.run.smithi157.stdout:5330 punch from 0x2bd3200 to 0x2bda600, (0x7400 bytes) 2024-09-13T09:13:37.685 INFO:teuthology.orchestra.run.smithi157.stdout:5331 read 0x4b73600 thru 0x4b7d2b1 (0x9cb2 bytes) 2024-09-13T09:13:37.685 INFO:teuthology.orchestra.run.smithi157.stdout:5332 read 0x3750600 thru 0x375bb39 (0xb53a bytes) 2024-09-13T09:13:37.688 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:37.704 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:37.702+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:37.707 INFO:teuthology.orchestra.run.smithi157.stdout:5333 write 0xc5adc00 thru 0xc5b252a (0x492b bytes) 2024-09-13T09:13:37.708 INFO:teuthology.orchestra.run.smithi157.stdout:5334 write 0x9f5000 thru 0x9ff43a (0xa43b bytes) 2024-09-13T09:13:37.708 INFO:teuthology.orchestra.run.smithi157.stdout:5335 read 0x5816c00 thru 0x5825888 (0xec89 bytes) 2024-09-13T09:13:37.716 INFO:teuthology.orchestra.run.smithi157.stdout:5337 trunc from 0xc5b2600 to 0x5509800 2024-09-13T09:13:37.762 INFO:teuthology.orchestra.run.smithi157.stdout:5339 write 0xce6a000 thru 0xce7547e (0xb47f bytes) 2024-09-13T09:13:37.762 INFO:teuthology.orchestra.run.smithi157.stdout:5340 write 0xbf14a00 thru 0xbf1d2e6 (0x88e7 bytes) 2024-09-13T09:13:37.762 INFO:teuthology.orchestra.run.smithi157.stdout:5341 read 0xc44ca00 thru 0xc44f9f6 (0x2ff7 bytes) 2024-09-13T09:13:37.765 INFO:teuthology.orchestra.run.smithi157.stdout:5342 trunc from 0xce75600 to 0xa856200 2024-09-13T09:13:37.783 INFO:teuthology.orchestra.run.smithi157.stdout:5346 write 0x705f600 thru 0x706038d (0xd8e bytes) 2024-09-13T09:13:37.783 INFO:teuthology.orchestra.run.smithi157.stdout:5347 read 0x31c1400 thru 0x31c1aa3 (0x6a4 bytes) 2024-09-13T09:13:37.784 INFO:teuthology.orchestra.run.smithi157.stdout:5348 punch from 0x3c06800 to 0x3c0a800, (0x4000 bytes) 2024-09-13T09:13:37.784 INFO:teuthology.orchestra.run.smithi157.stdout:5349 punch from 0x6183800 to 0x6192a00, (0xf200 bytes) 2024-09-13T09:13:37.784 INFO:teuthology.orchestra.run.smithi157.stdout:5350 read 0x95b4800 thru 0x95c224a (0xda4b bytes) 2024-09-13T09:13:37.788 INFO:teuthology.orchestra.run.smithi157.stdout:5353 trunc from 0xa856200 to 0x6fc4200 2024-09-13T09:13:37.805 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:37.803+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:37.807 INFO:teuthology.orchestra.run.smithi157.stdout:5354 read 0x3aa2000 thru 0x3aace4b (0xae4c bytes) 2024-09-13T09:13:37.807 INFO:teuthology.orchestra.run.smithi157.stdout:5356 read 0x50e6200 thru 0x50f07f8 (0xa5f9 bytes) 2024-09-13T09:13:37.830 INFO:teuthology.orchestra.run.smithi157.stdout:5357 write 0xcdbc400 thru 0xcdc1781 (0x5382 bytes) 2024-09-13T09:13:37.830 INFO:teuthology.orchestra.run.smithi157.stdout:5358 read 0x8aee400 thru 0x8afdad8 (0xf6d9 bytes) 2024-09-13T09:13:37.830 INFO:teuthology.orchestra.run.smithi157.stdout:5359 read 0x564ec00 thru 0x565a159 (0xb55a bytes) 2024-09-13T09:13:37.831 INFO:teuthology.orchestra.run.smithi157.stdout:5360 clone 31 order 24 su 65536 sc 3 2024-09-13T09:13:37.843 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:37.843 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:37.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:37.843 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:37.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.843 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.843 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:37.843 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.320121106000002, 2024-09-13T09:13:37.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:13:37.844 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.844 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.844 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:13:37.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.844 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:37.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:37.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:37.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:37.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:13:37.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:13:37.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:13:37.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:13:37.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:13:37.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:13:37.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:13:37.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:13:37.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:13:37.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:13:37.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:13:37.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:13:37.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:13:37.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:13:37.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.848 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.849 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.849 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:37.849 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.32007205, 2024-09-13T09:13:37.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:13:37.849 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.849 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.849 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:13:37.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.849 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:37.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:37.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:37.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:37.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:13:37.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:13:37.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:13:37.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:13:37.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:13:37.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:13:37.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:13:37.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:13:37.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:13:37.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:13:37.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:13:37.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:13:37.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:13:37.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:13:37.854 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.854 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.854 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.854 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.854 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:37.855 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.320031974999999, 2024-09-13T09:13:37.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:13:37.855 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.855 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.855 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.855 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.855 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.855 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:13:37.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.855 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:37.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:37.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:37.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:37.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:13:37.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:13:37.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:13:37.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:13:37.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:13:37.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:13:37.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:13:37.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:13:37.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:13:37.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:13:37.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:13:37.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:13:37.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:13:37.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:13:37.859 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.859 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.859 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.860 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.860 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:37.860 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.320002160000001, 2024-09-13T09:13:37.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:13:37.860 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.860 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.860 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:13:37.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.860 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:37.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:37.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:37.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:37.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:13:37.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:13:37.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:13:37.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:37.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:13:37.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:13:37.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:13:37.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:13:37.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:13:37.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:13:37.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:13:37.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:13:37.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:13:37.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:13:37.864 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.864 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.864 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.865 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.865 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:37.865 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.319972499000002, 2024-09-13T09:13:37.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:13:37.865 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.865 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.865 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:13:37.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.865 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:37.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:37.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:37.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:37.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:13:37.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:13:37.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:13:37.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:13:37.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:13:37.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:13:37.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:13:37.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:13:37.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:13:37.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:13:37.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:13:37.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:13:37.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:13:37.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:13:37.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.869 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.870 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.870 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:37.870 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.319943291999998, 2024-09-13T09:13:37.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:13:37.870 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.870 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.870 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:13:37.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.870 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:37.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:37.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:37.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:37.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:13:37.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:13:37.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:13:37.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:13:37.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:13:37.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:13:37.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:13:37.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:13:37.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:13:37.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:13:37.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:13:37.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:13:37.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:13:37.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:13:37.874 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.874 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.874 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.874 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.874 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:37.875 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.319887111, 2024-09-13T09:13:37.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:13:37.875 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.875 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.875 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.875 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.875 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.875 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:13:37.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.875 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:37.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:37.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:37.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:37.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:13:37.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:13:37.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:13:37.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:13:37.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:13:37.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:13:37.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:13:37.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:13:37.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:13:37.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:13:37.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:13:37.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:13:37.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:13:37.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:13:37.879 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.879 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.879 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.879 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.879 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:37.879 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.31983365, 2024-09-13T09:13:37.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:13:37.880 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.880 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.880 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.880 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.880 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.880 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:13:37.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.880 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:37.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:37.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:13:37.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:37.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:13:37.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:13:37.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:13:37.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:13:37.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:13:37.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:13:37.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:13:37.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:13:37.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:13:37.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:13:37.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:13:37.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:13:37.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:13:37.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:13:37.884 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.884 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.884 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.885 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.885 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:37.885 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.319776820000001, 2024-09-13T09:13:37.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:13:37.885 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.885 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.885 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.885 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.885 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.885 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:13:37.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.885 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:37.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:37.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:13:37.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:37.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:13:37.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:13:37.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:13:37.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:13:37.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:13:37.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:13:37.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:13:37.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:13:37.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:13:37.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:13:37.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:13:37.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:13:37.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:13:37.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:13:37.889 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.889 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.890 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.890 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.890 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:37.890 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.319725793000003, 2024-09-13T09:13:37.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:13:37.890 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.890 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.890 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.890 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.890 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.890 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:13:37.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.891 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:37.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:37.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:13:37.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:37.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:13:37.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:13:37.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:13:37.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:13:37.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:13:37.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:13:37.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:13:37.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:13:37.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:13:37.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:13:37.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:13:37.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:13:37.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:13:37.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:13:37.895 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.895 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.895 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.895 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.895 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:37.895 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.319684602999999, 2024-09-13T09:13:37.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:13:37.895 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.895 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.896 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:13:37.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.896 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:37.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:37.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:37.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:37.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:13:37.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:13:37.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:13:37.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:13:37.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:13:37.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:13:37.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:13:37.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:13:37.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:13:37.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:13:37.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:13:37.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:13:37.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:13:37.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:13:37.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.900 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.900 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.900 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:37.900 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.319642487000003, 2024-09-13T09:13:37.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:13:37.900 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.900 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.900 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.900 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.901 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:13:37.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.901 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:37.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:37.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:37.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:37.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:13:37.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:13:37.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:13:37.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:13:37.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:13:37.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:13:37.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:13:37.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:13:37.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:13:37.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:13:37.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:13:37.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:13:37.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:13:37.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:13:37.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.904 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.905 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.905 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.905 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:37.905 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.319598608, 2024-09-13T09:13:37.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:13:37.905 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.905 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.905 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:13:37.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.906 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:37.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:37.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:37.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:37.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:13:37.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:13:37.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:13:37.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:13:37.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:13:37.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:13:37.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:13:37.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:13:37.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:13:37.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:13:37.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:13:37.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:13:37.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:13:37.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:13:37.909 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.909 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.909 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.910 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.910 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:37.910 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.319552690999998, 2024-09-13T09:13:37.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:13:37.910 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.910 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.910 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:13:37.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.910 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:37.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:37.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:13:37.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:37.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:13:37.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:13:37.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:13:37.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:13:37.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:13:37.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:13:37.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:13:37.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:13:37.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:13:37.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:13:37.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:13:37.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:13:37.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:13:37.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:13:37.914 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.914 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.914 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.915 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.915 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:37.915 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.319502919999998, 2024-09-13T09:13:37.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:13:37.915 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.915 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.915 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.915 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.915 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.915 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:13:37.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.915 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:37.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:37.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:37.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:37.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:13:37.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:13:37.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:13:37.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:13:37.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:13:37.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:13:37.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:13:37.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:13:37.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:13:37.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:13:37.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:13:37.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:13:37.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:13:37.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:13:37.919 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.919 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.919 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.919 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.920 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:37.920 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.319457516, 2024-09-13T09:13:37.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:13:37.920 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.920 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.920 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.920 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.920 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.920 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:13:37.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.920 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:37.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:37.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:37.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:37.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:13:37.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:13:37.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:13:37.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:13:37.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:13:37.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:13:37.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:13:37.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:13:37.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:13:37.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:13:37.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:13:37.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:13:37.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:13:37.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:13:37.924 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.924 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.924 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.924 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.924 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:37.925 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.319406878000002, 2024-09-13T09:13:37.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:13:37.925 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.925 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.925 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:13:37.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.925 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:37.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:37.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:13:37.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:37.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:13:37.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:13:37.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:13:37.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:13:37.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:13:37.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:13:37.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:13:37.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:13:37.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:13:37.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:13:37.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:13:37.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:13:37.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:13:37.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:13:37.929 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.929 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.929 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.930 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.930 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:37.930 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.319351034999997, 2024-09-13T09:13:37.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:13:37.930 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.930 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.930 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.930 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.930 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.930 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:13:37.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.931 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:37.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:37.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:37.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:37.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:13:37.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:13:37.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:13:37.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:13:37.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:13:37.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:13:37.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:13:37.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:13:37.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:13:37.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:13:37.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:13:37.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:13:37.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:13:37.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:13:37.934 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.934 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.934 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.935 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.935 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:37.935 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.319294112999998, 2024-09-13T09:13:37.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:13:37.935 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.935 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.935 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:13:37.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.935 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:37.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:37.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:37.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:37.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:13:37.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:13:37.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:13:37.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:13:37.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:13:37.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:13:37.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:13:37.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:13:37.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:13:37.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:13:37.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:13:37.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:13:37.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:13:37.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:13:37.939 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.939 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.939 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.940 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:37.940 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:37.940 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.319240550000004, 2024-09-13T09:13:37.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:13:37.940 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:37.940 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:37.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:37.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:37.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:37.940 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:13:37.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.940 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:37.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:37.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:37.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:37.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:37.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:37.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:37.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:37.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:37.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:37.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:13:37.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:13:37.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:37.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:13:37.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:13:37.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:37.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:13:37.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:13:37.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:13:37.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:13:37.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:37.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:13:37.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:13:37.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:37.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:13:37.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:13:37.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:37.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:37.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:37.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:13:37.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:13:37.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.944 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:37.944 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:37.944 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:37.946 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:37.946 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:37.945+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:38.047 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:38.046+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:38.087 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:38.088 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:38.088 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:38.088 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:38.096 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:38.147 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:38.147+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:38.197 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:38.197 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:38.198 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:38.198 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:38.198 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:38.206 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:38.248 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:38.247+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:38.349 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:38.348+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:38.350 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:38.350 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:38.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:38.350 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:38.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.350 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2185 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.350 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:38.350 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822632938000002, 2024-09-13T09:13:38.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063348994000000006, 2024-09-13T09:13:38.351 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.351 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.351 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.351 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.351 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.351 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2185 2024-09-13T09:13:38.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.351 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:38.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:38.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:38.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525765+0000", 2024-09-13T09:13:38.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2390000000000001e-06 2024-09-13T09:13:38.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525772+0000", 2024-09-13T09:13:38.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:13:38.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525777+0000", 2024-09-13T09:13:38.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0799999999999996e-06 2024-09-13T09:13:38.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589020+0000", 2024-09-13T09:13:38.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063242501000000007 2024-09-13T09:13:38.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589063+0000", 2024-09-13T09:13:38.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2673999999999998e-05 2024-09-13T09:13:38.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589112+0000", 2024-09-13T09:13:38.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9286000000000002e-05 2024-09-13T09:13:38.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.354 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.355 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2187 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.355 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:38.355 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822613391000001, 2024-09-13T09:13:38.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063500102000000003, 2024-09-13T09:13:38.355 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.355 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.355 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2187 2024-09-13T09:13:38.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.355 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:38.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:38.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:38.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525784+0000", 2024-09-13T09:13:38.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1629999999999999e-06 2024-09-13T09:13:38.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525792+0000", 2024-09-13T09:13:38.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:38.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525796+0000", 2024-09-13T09:13:38.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7389999999999999e-06 2024-09-13T09:13:38.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589132+0000", 2024-09-13T09:13:38.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063336119999999996 2024-09-13T09:13:38.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589187+0000", 2024-09-13T09:13:38.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4126000000000003e-05 2024-09-13T09:13:38.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589283+0000", 2024-09-13T09:13:38.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6032999999999993e-05 2024-09-13T09:13:38.359 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.359 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.359 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.359 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2188 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.359 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:38.359 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822595345, 2024-09-13T09:13:38.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063724617999999997, 2024-09-13T09:13:38.359 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.360 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.360 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.360 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.360 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.360 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2188 2024-09-13T09:13:38.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.360 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:38.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:38.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:38.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525802+0000", 2024-09-13T09:13:38.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0439999999999998e-06 2024-09-13T09:13:38.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525809+0000", 2024-09-13T09:13:38.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.238e-06 2024-09-13T09:13:38.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525814+0000", 2024-09-13T09:13:38.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-13T09:13:38.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589327+0000", 2024-09-13T09:13:38.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063513257000000004 2024-09-13T09:13:38.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589418+0000", 2024-09-13T09:13:38.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0796000000000003e-05 2024-09-13T09:13:38.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589525+0000", 2024-09-13T09:13:38.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010693500000000001 2024-09-13T09:13:38.363 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.363 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.364 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.364 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2190 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.364 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:38.364 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822577258999999, 2024-09-13T09:13:38.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063863981, 2024-09-13T09:13:38.364 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.364 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.364 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.364 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.364 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.364 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2190 2024-09-13T09:13:38.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.365 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:38.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:38.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525818+0000", 2024-09-13T09:13:38.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525820+0000", 2024-09-13T09:13:38.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:13:38.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525827+0000", 2024-09-13T09:13:38.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0589999999999999e-06 2024-09-13T09:13:38.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525831+0000", 2024-09-13T09:13:38.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7870000000000001e-06 2024-09-13T09:13:38.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589562+0000", 2024-09-13T09:13:38.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063731038000000004 2024-09-13T09:13:38.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589612+0000", 2024-09-13T09:13:38.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0087999999999999e-05 2024-09-13T09:13:38.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589683+0000", 2024-09-13T09:13:38.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0121000000000002e-05 2024-09-13T09:13:38.368 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.368 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.368 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.368 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2192 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.368 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:38.368 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822560298999999, 2024-09-13T09:13:38.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064050299000000005, 2024-09-13T09:13:38.369 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.369 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.369 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.369 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.369 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.369 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2192 2024-09-13T09:13:38.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.369 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:38.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:38.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525835+0000", 2024-09-13T09:13:38.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525837+0000", 2024-09-13T09:13:38.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:13:38.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525845+0000", 2024-09-13T09:13:38.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3660000000000003e-06 2024-09-13T09:13:38.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525850+0000", 2024-09-13T09:13:38.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0030000000000002e-06 2024-09-13T09:13:38.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589720+0000", 2024-09-13T09:13:38.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063870176000000001 2024-09-13T09:13:38.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589801+0000", 2024-09-13T09:13:38.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0889000000000004e-05 2024-09-13T09:13:38.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589886+0000", 2024-09-13T09:13:38.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5183000000000001e-05 2024-09-13T09:13:38.372 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.373 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.373 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.373 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.373 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:38.373 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822542196000001, 2024-09-13T09:13:38.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:13:38.373 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.373 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.373 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:13:38.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.374 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:38.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:38.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:13:38.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:13:38.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:13:38.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:13:38.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:13:38.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:13:38.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:13:38.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:13:38.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:13:38.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:13:38.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:13:38.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:13:38.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:13:38.377 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.377 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.377 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.377 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.377 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:38.377 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822524498, 2024-09-13T09:13:38.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:13:38.377 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:13:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:38.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:38.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:13:38.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:13:38.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:13:38.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:13:38.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:13:38.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:13:38.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:13:38.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:13:38.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:13:38.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:13:38.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:13:38.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:13:38.381 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.381 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.381 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.382 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.382 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:38.382 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822503883, 2024-09-13T09:13:38.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:13:38.382 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.382 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.382 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:13:38.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.382 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:38.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:38.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:38.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:13:38.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:13:38.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:13:38.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:13:38.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:13:38.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:13:38.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:13:38.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:13:38.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:13:38.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:13:38.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:13:38.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:13:38.386 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.386 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.386 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.386 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.386 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:38.386 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822484927000001, 2024-09-13T09:13:38.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:13:38.386 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.387 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.387 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:13:38.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.387 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:38.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:38.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:38.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:13:38.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:13:38.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:13:38.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:13:38.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:13:38.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:13:38.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:13:38.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:13:38.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:13:38.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:13:38.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:13:38.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:13:38.390 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.390 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.390 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.391 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.391 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:38.391 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822467669000002, 2024-09-13T09:13:38.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:13:38.391 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.391 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.391 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.391 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.391 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.391 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:13:38.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.391 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:38.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:38.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:38.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:13:38.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:13:38.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:13:38.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:13:38.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:13:38.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:13:38.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:13:38.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:13:38.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:13:38.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:13:38.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:13:38.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:13:38.395 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.395 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.395 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.395 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.395 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:38.395 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822447655000001, 2024-09-13T09:13:38.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:13:38.395 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.396 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.396 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.396 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.396 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.396 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:13:38.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.396 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:38.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:38.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:38.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:13:38.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:13:38.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:13:38.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:13:38.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:13:38.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:13:38.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:13:38.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:13:38.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:13:38.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:13:38.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:13:38.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:13:38.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.399 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.400 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.400 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:38.400 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822422072999998, 2024-09-13T09:13:38.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:13:38.400 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.400 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.400 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:13:38.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.401 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:38.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:38.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:13:38.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:13:38.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:13:38.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:13:38.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:13:38.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:13:38.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:13:38.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:13:38.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.404 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.404 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.404 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:38.404 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822402789000002, 2024-09-13T09:13:38.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:13:38.404 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.404 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.405 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:13:38.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.405 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:38.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:38.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:38.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:13:38.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:13:38.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:13:38.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:13:38.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:13:38.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:13:38.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:13:38.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:13:38.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:13:38.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:13:38.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:13:38.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:13:38.408 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.408 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.408 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.409 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.409 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:38.409 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822385532999999, 2024-09-13T09:13:38.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:13:38.409 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.409 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.409 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:13:38.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.409 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:38.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:38.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:38.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:13:38.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:13:38.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:13:38.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:13:38.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:13:38.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:13:38.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:13:38.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:13:38.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:13:38.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:13:38.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:13:38.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:13:38.413 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.413 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.413 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.413 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.413 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:38.413 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822366053, 2024-09-13T09:13:38.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:13:38.414 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.414 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.414 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:13:38.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.414 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:38.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:38.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:13:38.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:13:38.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:13:38.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:13:38.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:13:38.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:13:38.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:13:38.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:13:38.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:13:38.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:13:38.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:13:38.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:13:38.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:13:38.417 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.417 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.417 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.418 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.418 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:38.418 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822349123999999, 2024-09-13T09:13:38.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:13:38.418 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.418 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.418 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.418 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.418 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.418 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:13:38.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.418 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:38.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:38.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:13:38.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:13:38.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:13:38.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:13:38.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:13:38.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:13:38.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:13:38.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:13:38.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:13:38.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:13:38.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:13:38.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:13:38.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:13:38.422 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.422 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.422 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.422 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.422 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:38.422 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822329974999999, 2024-09-13T09:13:38.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:13:38.422 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.422 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.423 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.423 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:13:38.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.423 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:38.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:38.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:38.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:13:38.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:13:38.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:13:38.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:13:38.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:13:38.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:13:38.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:13:38.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:13:38.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:13:38.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:13:38.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:13:38.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:13:38.426 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.426 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.426 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.426 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.426 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.822311836000001, 2024-09-13T09:13:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:13:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:13:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:13:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:13:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:13:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:13:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:13:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:13:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:13:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:13:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:13:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:13:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:13:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:13:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.821901341, 2024-09-13T09:13:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:13:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:13:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:13:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:13:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:13:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:13:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:13:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:13:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:13:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:13:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:13:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:13:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:13:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:13:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 22.821878439999999, 2024-09-13T09:13:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:13:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:13:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:13:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:13:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:13:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:13:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:13:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:13:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:13:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:13:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:13:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:13:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:13:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:13:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.440 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:38.441 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:38.450 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:38.448+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:38.546 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-09-13T09:13:38.550 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:38.549+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:38.588 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:38.588 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:38.588 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:38.588 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:38.597 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:38.648 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone30 from 0xcdc1800 (overlap 0xfa7c00) to 0x4b2400 2024-09-13T09:13:38.650 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:38.649+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:38.745 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:38.745 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:38.746 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:38.746 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:38.746 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:38.751 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:38.750+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:38.754 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:38.852 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:38.849+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:38.902 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:38.902 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:38.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:38.902 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:38.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.903 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9035 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2342912~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.903 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:38.903 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.154665473000001, 2024-09-13T09:13:38.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070293606999999994, 2024-09-13T09:13:38.903 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.903 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.903 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.903 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.903 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.903 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9035 2024-09-13T09:13:38.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.903 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:38.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:38.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:38.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745602+0000", 2024-09-13T09:13:38.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-06 2024-09-13T09:13:38.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745607+0000", 2024-09-13T09:13:38.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0770000000000001e-06 2024-09-13T09:13:38.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745609+0000", 2024-09-13T09:13:38.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3499999999999999e-06 2024-09-13T09:13:38.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815626+0000", 2024-09-13T09:13:38.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070017398999999994 2024-09-13T09:13:38.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815672+0000", 2024-09-13T09:13:38.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5275999999999998e-05 2024-09-13T09:13:38.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815895+0000", 2024-09-13T09:13:38.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022328300000000001 2024-09-13T09:13:38.907 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.907 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.907 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.907 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.907 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:38.907 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.154652929000001, 2024-09-13T09:13:38.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:13:38.908 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.908 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.908 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:13:38.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.908 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:38.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:38.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:38.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:13:38.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:38.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:13:38.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:13:38.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:13:38.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:13:38.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:13:38.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:13:38.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:13:38.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:13:38.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:13:38.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:13:38.912 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.912 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.912 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.912 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.912 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:38.912 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.154644251000001, 2024-09-13T09:13:38.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:13:38.912 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.912 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.912 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.912 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.913 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:13:38.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.913 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:38.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:38.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:38.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:38.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:13:38.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:13:38.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:13:38.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:13:38.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:13:38.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:13:38.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:13:38.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:13:38.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:13:38.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:13:38.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:13:38.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:13:38.916 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.916 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.916 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.916 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.154635265000003, 2024-09-13T09:13:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:13:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:13:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:38.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:38.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:38.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:13:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:13:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:13:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:13:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:13:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:13:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:13:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:13:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:13:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:13:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:13:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:13:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.154627292999997, 2024-09-13T09:13:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:13:38.921 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.922 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.922 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.922 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.922 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.922 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:13:38.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.922 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:38.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:38.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:38.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:38.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:13:38.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:13:38.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:13:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:13:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:13:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:13:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:13:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:13:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:13:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:13:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:13:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:13:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.152752458999998, 2024-09-13T09:13:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:13:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:13:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:13:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:13:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:13:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:13:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:13:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:13:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:13:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:13:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:13:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:13:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:13:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:13:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.152741798000001, 2024-09-13T09:13:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:13:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:13:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:13:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:13:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:13:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:13:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:13:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:13:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:13:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:13:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:13:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:13:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:13:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:13:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.152719822000002, 2024-09-13T09:13:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:13:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:13:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:13:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:13:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:13:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:13:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:13:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:13:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:13:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:13:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:13:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:13:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:13:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:13:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.15270632, 2024-09-13T09:13:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:13:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:13:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:13:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:13:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:13:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:13:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:13:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:13:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:13:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:13:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:13:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:13:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:13:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:13:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9056 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 7159808~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.150705490999997, 2024-09-13T09:13:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070283126000000001, 2024-09-13T09:13:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9056 2024-09-13T09:13:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749562+0000", 2024-09-13T09:13:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1340000000000001e-06 2024-09-13T09:13:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749565+0000", 2024-09-13T09:13:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8760000000000001e-06 2024-09-13T09:13:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749567+0000", 2024-09-13T09:13:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2340000000000001e-06 2024-09-13T09:13:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819750+0000", 2024-09-13T09:13:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070182537000000003 2024-09-13T09:13:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819792+0000", 2024-09-13T09:13:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2141999999999997e-05 2024-09-13T09:13:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819844+0000", 2024-09-13T09:13:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2428e-05 2024-09-13T09:13:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.150697221999998, 2024-09-13T09:13:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:13:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:13:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:13:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:13:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:13:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:13:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:13:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:13:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:13:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:13:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:13:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:13:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:13:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:13:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.150672900000004, 2024-09-13T09:13:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:13:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:13:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:13:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:13:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:13:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:13:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:13:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:13:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:13:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:13:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:13:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:13:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:13:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:13:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.150661925999998, 2024-09-13T09:13:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:13:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:13:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:13:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:13:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:13:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:13:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:13:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:13:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:13:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:13:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:13:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:13:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:13:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:13:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:13:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9063 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1589248~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.148758284000003, 2024-09-13T09:13:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070391924999999994, 2024-09-13T09:13:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9063 2024-09-13T09:13:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751509+0000", 2024-09-13T09:13:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.142e-06 2024-09-13T09:13:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751513+0000", 2024-09-13T09:13:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6669999999999998e-06 2024-09-13T09:13:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751520+0000", 2024-09-13T09:13:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5910000000000001e-06 2024-09-13T09:13:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821593+0000", 2024-09-13T09:13:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073519000000001 2024-09-13T09:13:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821648+0000", 2024-09-13T09:13:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:13:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821900+0000", 2024-09-13T09:13:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025263999999999999 2024-09-13T09:13:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.148744301000001, 2024-09-13T09:13:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:13:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:13:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:13:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:13:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:13:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:13:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:13:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:13:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:13:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:13:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:13:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:13:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:13:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.148734904000001, 2024-09-13T09:13:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:13:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:13:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:13:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:13:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:13:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:13:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:13:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:13:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:13:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:13:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:13:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:13:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:13:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:13:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9071 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.146751887000001, 2024-09-13T09:13:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070386377999999999, 2024-09-13T09:13:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9071 2024-09-13T09:13:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753516+0000", 2024-09-13T09:13:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1519999999999999e-06 2024-09-13T09:13:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753519+0000", 2024-09-13T09:13:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6559999999999998e-06 2024-09-13T09:13:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753522+0000", 2024-09-13T09:13:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:13:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823798+0000", 2024-09-13T09:13:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070275907999999998 2024-09-13T09:13:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823845+0000", 2024-09-13T09:13:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7593000000000003e-05 2024-09-13T09:13:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823901+0000", 2024-09-13T09:13:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5934999999999999e-05 2024-09-13T09:13:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3181 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T09:13:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.8783707280000002, 2024-09-13T09:13:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.480957603, 2024-09-13T09:13:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:13:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:13:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3181 2024-09-13T09:13:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021894+0000", 2024-09-13T09:13:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:13:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021903+0000", 2024-09-13T09:13:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4109999999999996e-06 2024-09-13T09:13:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021919+0000", 2024-09-13T09:13:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6225000000000001e-05 2024-09-13T09:13:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021926+0000", 2024-09-13T09:13:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4120000000000004e-06 2024-09-13T09:13:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021975+0000", 2024-09-13T09:13:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8526000000000003e-05 2024-09-13T09:13:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022049+0000", 2024-09-13T09:13:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4018999999999997e-05 2024-09-13T09:13:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:13:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022230+0000", 2024-09-13T09:13:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018085599999999999 2024-09-13T09:13:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:13:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022796+0000", 2024-09-13T09:13:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056638999999999995 2024-09-13T09:13:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499540+0000", 2024-09-13T09:13:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.47674342800000002 2024-09-13T09:13:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499547+0000", 2024-09-13T09:13:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:13:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502794+0000", 2024-09-13T09:13:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.003246818 2024-09-13T09:13:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502854+0000", 2024-09-13T09:13:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9925e-05 2024-09-13T09:13:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3182 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 139818970156544 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:13:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.4051361730000007, 2024-09-13T09:13:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0089058390000001, 2024-09-13T09:13:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:13:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:13:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3182 2024-09-13T09:13:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495128+0000", 2024-09-13T09:13:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:13:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495139+0000", 2024-09-13T09:13:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0599e-05 2024-09-13T09:13:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495157+0000", 2024-09-13T09:13:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7830999999999999e-05 2024-09-13T09:13:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495167+0000", 2024-09-13T09:13:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0689999999999999e-05 2024-09-13T09:13:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495229+0000", 2024-09-13T09:13:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1898999999999998e-05 2024-09-13T09:13:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:13:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495243+0000", 2024-09-13T09:13:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3349e-05 2024-09-13T09:13:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499575+0000", 2024-09-13T09:13:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043324410000000002 2024-09-13T09:13:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499582+0000", 2024-09-13T09:13:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0849999999999999e-06 2024-09-13T09:13:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502872+0000", 2024-09-13T09:13:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0032895150000001 2024-09-13T09:13:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503016+0000", 2024-09-13T09:13:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014394200000000001 2024-09-13T09:13:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504036+0000", 2024-09-13T09:13:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010208680000000001 2024-09-13T09:13:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:19151 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 140383089882784 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:13:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.4051247139999994, 2024-09-13T09:13:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.009071287, 2024-09-13T09:13:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:13:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:13:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19151 2024-09-13T09:13:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495140+0000", 2024-09-13T09:13:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:13:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495149+0000", 2024-09-13T09:13:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0759999999999992e-06 2024-09-13T09:13:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495165+0000", 2024-09-13T09:13:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5322e-05 2024-09-13T09:13:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495173+0000", 2024-09-13T09:13:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0979999999999994e-06 2024-09-13T09:13:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495318+0000", 2024-09-13T09:13:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014553 2024-09-13T09:13:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:13:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495325+0000", 2024-09-13T09:13:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:13:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499598+0000", 2024-09-13T09:13:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004273064 2024-09-13T09:13:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499602+0000", 2024-09-13T09:13:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8539999999999999e-06 2024-09-13T09:13:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503507+0000", 2024-09-13T09:13:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0039045520000001 2024-09-13T09:13:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503583+0000", 2024-09-13T09:13:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6283000000000001e-05 2024-09-13T09:13:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504213+0000", 2024-09-13T09:13:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00063044700000000002 2024-09-13T09:13:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:38.998 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:38.999 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:38.998+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:38.999 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:39.010 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-13T09:13:39.069 INFO:teuthology.orchestra.run.smithi157.stdout:5361 write 0x807d600 thru 0x808bf19 (0xe91a bytes) 2024-09-13T09:13:39.099 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:39.097+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:39.103 INFO:teuthology.orchestra.run.smithi157.stdout:5362 write 0xe1e9a00 thru 0xe1ec07a (0x267b bytes) 2024-09-13T09:13:39.104 INFO:teuthology.orchestra.run.smithi157.stdout:5363 write 0x9c2a200 thru 0x9c37cf7 (0xdaf8 bytes) 2024-09-13T09:13:39.104 INFO:teuthology.orchestra.run.smithi157.stdout:5365 read 0x8e0200 thru 0x8e51a6 (0x4fa7 bytes) 2024-09-13T09:13:39.110 INFO:teuthology.orchestra.run.smithi157.stdout:5367 trunc from 0xe1ec200 to 0x99f4800 2024-09-13T09:13:39.117 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:39.118 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:39.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.118 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:616 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.118 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111625+0000", 2024-09-13T09:13:39.118 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.004468255, 2024-09-13T09:13:39.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044894710000000001, 2024-09-13T09:13:39.118 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.118 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.118 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.118 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.118 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.118 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 616 2024-09-13T09:13:39.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.119 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111625+0000", 2024-09-13T09:13:39.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111625+0000", 2024-09-13T09:13:39.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111624+0000", 2024-09-13T09:13:39.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111627+0000", 2024-09-13T09:13:39.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3149999999999999e-06 2024-09-13T09:13:39.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111633+0000", 2024-09-13T09:13:39.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9619999999999998e-06 2024-09-13T09:13:39.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111638+0000", 2024-09-13T09:13:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3190000000000002e-06 2024-09-13T09:13:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:617 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111642+0000", 2024-09-13T09:13:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0044507269999999998, 2024-09-13T09:13:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045384509999999998, 2024-09-13T09:13:39.121 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 617 2024-09-13T09:13:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111642+0000", 2024-09-13T09:13:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111642+0000", 2024-09-13T09:13:39.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111642+0000", 2024-09-13T09:13:39.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111644+0000", 2024-09-13T09:13:39.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3819999999999998e-06 2024-09-13T09:13:39.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111650+0000", 2024-09-13T09:13:39.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.02e-06 2024-09-13T09:13:39.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.124 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.124 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111656+0000", 2024-09-13T09:13:39.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7780000000000004e-06 2024-09-13T09:13:39.124 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.124 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.124 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.124 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:618 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.124 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111660+0000", 2024-09-13T09:13:39.124 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0044325509999999999, 2024-09-13T09:13:39.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045774850000000001, 2024-09-13T09:13:39.125 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.125 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.125 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.125 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.125 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.125 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 618 2024-09-13T09:13:39.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.125 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111660+0000", 2024-09-13T09:13:39.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111660+0000", 2024-09-13T09:13:39.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111660+0000", 2024-09-13T09:13:39.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111662+0000", 2024-09-13T09:13:39.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2290000000000002e-06 2024-09-13T09:13:39.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111672+0000", 2024-09-13T09:13:39.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2320000000000007e-06 2024-09-13T09:13:39.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111676+0000", 2024-09-13T09:13:39.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9280000000000001e-06 2024-09-13T09:13:39.127 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.127 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.127 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.128 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:619 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.128 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111681+0000", 2024-09-13T09:13:39.128 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0044121450000000001, 2024-09-13T09:13:39.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046172690000000002, 2024-09-13T09:13:39.128 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.128 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.128 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.128 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.128 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.128 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 619 2024-09-13T09:13:39.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.128 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111681+0000", 2024-09-13T09:13:39.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111681+0000", 2024-09-13T09:13:39.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111681+0000", 2024-09-13T09:13:39.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.130 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111683+0000", 2024-09-13T09:13:39.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2350000000000002e-06 2024-09-13T09:13:39.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.130 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111693+0000", 2024-09-13T09:13:39.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0141e-05 2024-09-13T09:13:39.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111698+0000", 2024-09-13T09:13:39.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0309999999999998e-06 2024-09-13T09:13:39.131 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.131 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.131 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.131 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:620 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.131 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111721+0000", 2024-09-13T09:13:39.131 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0043722919999999998, 2024-09-13T09:13:39.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046316660000000004, 2024-09-13T09:13:39.131 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.131 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.131 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.132 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.132 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.132 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 620 2024-09-13T09:13:39.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.132 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111721+0000", 2024-09-13T09:13:39.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111721+0000", 2024-09-13T09:13:39.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111720+0000", 2024-09-13T09:13:39.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111723+0000", 2024-09-13T09:13:39.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9390000000000002e-06 2024-09-13T09:13:39.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111731+0000", 2024-09-13T09:13:39.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3900000000000004e-06 2024-09-13T09:13:39.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.134 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111736+0000", 2024-09-13T09:13:39.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4349999999999996e-06 2024-09-13T09:13:39.134 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.134 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.134 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.134 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:621 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.134 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111740+0000", 2024-09-13T09:13:39.134 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0043528259999999997, 2024-09-13T09:13:39.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046636589999999997, 2024-09-13T09:13:39.135 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.135 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.135 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.135 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.135 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.135 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 621 2024-09-13T09:13:39.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.135 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.135 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111740+0000", 2024-09-13T09:13:39.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111740+0000", 2024-09-13T09:13:39.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111740+0000", 2024-09-13T09:13:39.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111742+0000", 2024-09-13T09:13:39.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:13:39.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111748+0000", 2024-09-13T09:13:39.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.66e-06 2024-09-13T09:13:39.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111752+0000", 2024-09-13T09:13:39.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:13:39.137 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.137 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.137 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.138 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:622 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.138 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111756+0000", 2024-09-13T09:13:39.138 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.004337339, 2024-09-13T09:13:39.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046983839999999999, 2024-09-13T09:13:39.138 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.138 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.138 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.138 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.138 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.138 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 622 2024-09-13T09:13:39.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.138 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111756+0000", 2024-09-13T09:13:39.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111756+0000", 2024-09-13T09:13:39.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111755+0000", 2024-09-13T09:13:39.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111761+0000", 2024-09-13T09:13:39.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0939999999999996e-06 2024-09-13T09:13:39.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111767+0000", 2024-09-13T09:13:39.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8030000000000002e-06 2024-09-13T09:13:39.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111772+0000", 2024-09-13T09:13:39.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:13:39.141 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.141 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.141 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.141 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:623 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.141 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111777+0000", 2024-09-13T09:13:39.141 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0043164509999999998, 2024-09-13T09:13:39.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047319759999999997, 2024-09-13T09:13:39.141 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.141 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.141 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.141 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.142 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.142 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 623 2024-09-13T09:13:39.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.142 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111777+0000", 2024-09-13T09:13:39.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111777+0000", 2024-09-13T09:13:39.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111776+0000", 2024-09-13T09:13:39.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111778+0000", 2024-09-13T09:13:39.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0679999999999999e-06 2024-09-13T09:13:39.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111784+0000", 2024-09-13T09:13:39.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.049e-06 2024-09-13T09:13:39.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111790+0000", 2024-09-13T09:13:39.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.259e-06 2024-09-13T09:13:39.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.144 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.144 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:624 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.144 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111794+0000", 2024-09-13T09:13:39.144 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.004299209, 2024-09-13T09:13:39.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047716529999999998, 2024-09-13T09:13:39.145 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.145 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.145 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.145 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.145 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.145 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 624 2024-09-13T09:13:39.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.145 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111794+0000", 2024-09-13T09:13:39.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111794+0000", 2024-09-13T09:13:39.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111793+0000", 2024-09-13T09:13:39.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111796+0000", 2024-09-13T09:13:39.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:13:39.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.147 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.147 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111802+0000", 2024-09-13T09:13:39.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8220000000000003e-06 2024-09-13T09:13:39.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.147 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.147 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111807+0000", 2024-09-13T09:13:39.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9799999999999998e-06 2024-09-13T09:13:39.147 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.147 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.147 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.148 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:625 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.148 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111818+0000", 2024-09-13T09:13:39.148 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0042745830000000002, 2024-09-13T09:13:39.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048054509999999996, 2024-09-13T09:13:39.148 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.148 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.148 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.148 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.148 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.148 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 625 2024-09-13T09:13:39.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.148 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111818+0000", 2024-09-13T09:13:39.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111818+0000", 2024-09-13T09:13:39.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111818+0000", 2024-09-13T09:13:39.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111821+0000", 2024-09-13T09:13:39.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8890000000000002e-06 2024-09-13T09:13:39.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111830+0000", 2024-09-13T09:13:39.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1889999999999996e-06 2024-09-13T09:13:39.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111836+0000", 2024-09-13T09:13:39.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:13:39.151 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.151 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.151 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.151 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:607 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.151 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111470+0000", 2024-09-13T09:13:39.151 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0046232859999999999, 2024-09-13T09:13:39.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052102190000000003, 2024-09-13T09:13:39.151 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.151 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.151 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.152 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.152 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.152 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 607 2024-09-13T09:13:39.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.152 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111470+0000", 2024-09-13T09:13:39.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111470+0000", 2024-09-13T09:13:39.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111469+0000", 2024-09-13T09:13:39.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111471+0000", 2024-09-13T09:13:39.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0370000000000001e-06 2024-09-13T09:13:39.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111477+0000", 2024-09-13T09:13:39.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7400000000000001e-06 2024-09-13T09:13:39.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111482+0000", 2024-09-13T09:13:39.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5580000000000002e-06 2024-09-13T09:13:39.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.154 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.154 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:608 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.154 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111486+0000", 2024-09-13T09:13:39.155 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0046073420000000004, 2024-09-13T09:13:39.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052408910000000001, 2024-09-13T09:13:39.155 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.155 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.155 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.155 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.155 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.155 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 608 2024-09-13T09:13:39.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.155 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111486+0000", 2024-09-13T09:13:39.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111486+0000", 2024-09-13T09:13:39.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111485+0000", 2024-09-13T09:13:39.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111487+0000", 2024-09-13T09:13:39.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2249999999999999e-06 2024-09-13T09:13:39.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111500+0000", 2024-09-13T09:13:39.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2864000000000001e-05 2024-09-13T09:13:39.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111505+0000", 2024-09-13T09:13:39.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9400000000000001e-06 2024-09-13T09:13:39.157 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.157 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.157 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.158 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:609 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.158 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111514+0000", 2024-09-13T09:13:39.158 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0045790730000000003, 2024-09-13T09:13:39.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052406290000000001, 2024-09-13T09:13:39.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.158 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.158 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.158 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.158 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.158 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 609 2024-09-13T09:13:39.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.158 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111514+0000", 2024-09-13T09:13:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111514+0000", 2024-09-13T09:13:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111514+0000", 2024-09-13T09:13:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111516+0000", 2024-09-13T09:13:39.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6299999999999998e-06 2024-09-13T09:13:39.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111522+0000", 2024-09-13T09:13:39.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9839999999999998e-06 2024-09-13T09:13:39.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111527+0000", 2024-09-13T09:13:39.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6689999999999997e-06 2024-09-13T09:13:39.161 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.161 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.161 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.161 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:611 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.161 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111531+0000", 2024-09-13T09:13:39.161 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.004561987, 2024-09-13T09:13:39.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052508040000000004, 2024-09-13T09:13:39.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.161 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.162 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.162 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.162 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.162 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 611 2024-09-13T09:13:39.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.162 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111531+0000", 2024-09-13T09:13:39.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111531+0000", 2024-09-13T09:13:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111531+0000", 2024-09-13T09:13:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111533+0000", 2024-09-13T09:13:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0099999999999998e-06 2024-09-13T09:13:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111541+0000", 2024-09-13T09:13:39.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6500000000000002e-06 2024-09-13T09:13:39.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111546+0000", 2024-09-13T09:13:39.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0509999999999996e-06 2024-09-13T09:13:39.164 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.164 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.164 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.164 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:612 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.164 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111551+0000", 2024-09-13T09:13:39.164 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0045420729999999998, 2024-09-13T09:13:39.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052659990000000004, 2024-09-13T09:13:39.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.165 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.165 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.165 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.165 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.165 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 612 2024-09-13T09:13:39.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.165 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111551+0000", 2024-09-13T09:13:39.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111551+0000", 2024-09-13T09:13:39.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111551+0000", 2024-09-13T09:13:39.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111553+0000", 2024-09-13T09:13:39.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2970000000000002e-06 2024-09-13T09:13:39.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111559+0000", 2024-09-13T09:13:39.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0769999999999996e-06 2024-09-13T09:13:39.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111564+0000", 2024-09-13T09:13:39.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6439999999999999e-06 2024-09-13T09:13:39.167 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.167 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.167 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.168 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:613 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.168 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111568+0000", 2024-09-13T09:13:39.168 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0045250380000000003, 2024-09-13T09:13:39.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052778909999999998, 2024-09-13T09:13:39.168 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.168 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.168 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.168 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.168 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.168 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 613 2024-09-13T09:13:39.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.168 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111568+0000", 2024-09-13T09:13:39.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111568+0000", 2024-09-13T09:13:39.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111568+0000", 2024-09-13T09:13:39.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111570+0000", 2024-09-13T09:13:39.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3039999999999999e-06 2024-09-13T09:13:39.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111577+0000", 2024-09-13T09:13:39.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.109e-06 2024-09-13T09:13:39.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111583+0000", 2024-09-13T09:13:39.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5809999999999996e-06 2024-09-13T09:13:39.171 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.171 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.171 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.171 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:614 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.171 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111589+0000", 2024-09-13T09:13:39.171 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0045043260000000003, 2024-09-13T09:13:39.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052838219999999997, 2024-09-13T09:13:39.171 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.171 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.172 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.172 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.172 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.172 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 614 2024-09-13T09:13:39.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.172 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111589+0000", 2024-09-13T09:13:39.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111589+0000", 2024-09-13T09:13:39.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111588+0000", 2024-09-13T09:13:39.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111591+0000", 2024-09-13T09:13:39.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4229999999999999e-06 2024-09-13T09:13:39.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111596+0000", 2024-09-13T09:13:39.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8340000000000004e-06 2024-09-13T09:13:39.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111602+0000", 2024-09-13T09:13:39.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1460000000000004e-06 2024-09-13T09:13:39.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.174 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.174 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5589.0:615 3.1 3.7aea7b21 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T09:13:39.174 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:39.111607+0000", 2024-09-13T09:13:39.174 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0044864240000000001, 2024-09-13T09:13:39.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052932630000000003, 2024-09-13T09:13:39.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.175 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:13:39.175 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.175 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5589", 2024-09-13T09:13:39.175 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/1984925464", 2024-09-13T09:13:39.175 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 615 2024-09-13T09:13:39.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.175 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111607+0000", 2024-09-13T09:13:39.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111607+0000", 2024-09-13T09:13:39.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111606+0000", 2024-09-13T09:13:39.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111609+0000", 2024-09-13T09:13:39.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3829999999999999e-06 2024-09-13T09:13:39.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111615+0000", 2024-09-13T09:13:39.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.212e-06 2024-09-13T09:13:39.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:39.111620+0000", 2024-09-13T09:13:39.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0320000000000003e-06 2024-09-13T09:13:39.177 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.177 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.178 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.178 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.178 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:13:39.178 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 18 2024-09-13T09:13:39.178 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:39.178 INFO:teuthology.orchestra.run.smithi157.stdout:5369 trunc from 0x99f4800 to 0x7585c00 2024-09-13T09:13:39.179 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:39.190 INFO:teuthology.orchestra.run.smithi157.stdout:5371 trunc from 0x7585c00 to 0x37a2400 2024-09-13T09:13:39.200 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:39.200+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:39.248 INFO:teuthology.orchestra.run.smithi157.stdout:5372 write 0x4f11c00 thru 0x4f1d7c8 (0xbbc9 bytes) 2024-09-13T09:13:39.249 INFO:teuthology.orchestra.run.smithi157.stdout:5373 punch from 0x51e00 to 0x61800, (0xfa00 bytes) 2024-09-13T09:13:39.253 INFO:teuthology.orchestra.run.smithi157.stdout:5374 read 0xa2f600 thru 0xa323d2 (0x2dd3 bytes) 2024-09-13T09:13:39.272 INFO:teuthology.orchestra.run.smithi157.stdout:5375 trunc from 0x4f1d800 to 0xc99bc00 2024-09-13T09:13:39.302 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:39.301+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:39.304 INFO:teuthology.orchestra.run.smithi157.stdout:5376 write 0xcae3800 thru 0xcaea652 (0x6e53 bytes) 2024-09-13T09:13:39.305 INFO:teuthology.orchestra.run.smithi157.stdout:5377 punch from 0x7586200 to 0x7590c00, (0xaa00 bytes) 2024-09-13T09:13:39.305 INFO:teuthology.orchestra.run.smithi157.stdout:5378 read 0x4119200 thru 0x4121aa1 (0x88a2 bytes) 2024-09-13T09:13:39.306 INFO:teuthology.orchestra.run.smithi157.stdout:5383 trunc from 0xcaea800 to 0xc533e00 2024-09-13T09:13:39.322 INFO:teuthology.orchestra.run.smithi157.stdout:5384 write 0xa635200 thru 0xa638525 (0x3326 bytes) 2024-09-13T09:13:39.322 INFO:teuthology.orchestra.run.smithi157.stdout:5385 write 0x7a74000 thru 0x7a783a7 (0x43a8 bytes) 2024-09-13T09:13:39.323 INFO:teuthology.orchestra.run.smithi157.stdout:5387 read 0xc4c4a00 thru 0xc4c8e30 (0x4431 bytes) 2024-09-13T09:13:39.323 INFO:teuthology.orchestra.run.smithi157.stdout:5389 read 0x9e00200 thru 0x9e0552e (0x532f bytes) 2024-09-13T09:13:39.323 INFO:teuthology.orchestra.run.smithi157.stdout:5391 write 0xbf7b800 thru 0xbf8b11e (0xf91f bytes) 2024-09-13T09:13:39.323 INFO:teuthology.orchestra.run.smithi157.stdout:5392 punch from 0x11bfc00 to 0x11c9200, (0x9600 bytes) 2024-09-13T09:13:39.341 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:39.341 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:39.342 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:39.342 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:39.342 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:39.351 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:39.358 INFO:teuthology.orchestra.run.smithi157.stdout:5395 write 0xe87ac00 thru 0xe8847f9 (0x9bfa bytes) 2024-09-13T09:13:39.363 INFO:teuthology.orchestra.run.smithi157.stdout:5397 read 0x75be00 thru 0x76a18c (0xe38d bytes) 2024-09-13T09:13:39.364 INFO:teuthology.orchestra.run.smithi157.stdout:5398 read 0x5ffbc00 thru 0x600b6c1 (0xfac2 bytes) 2024-09-13T09:13:39.365 INFO:teuthology.orchestra.run.smithi157.stdout:5399 read 0x6834c00 thru 0x683f5ed (0xa9ee bytes) 2024-09-13T09:13:39.365 INFO:teuthology.orchestra.run.smithi157.stdout:5400 write 0x69e7400 thru 0x69ea6c2 (0x32c3 bytes) 2024-09-13T09:13:39.366 INFO:teuthology.orchestra.run.smithi157.stdout:5403 read 0xb4f5c00 thru 0xb4ff5b2 (0x99b3 bytes) 2024-09-13T09:13:39.366 INFO:teuthology.orchestra.run.smithi157.stdout:5404 write 0x120600 thru 0x1212f6 (0xcf7 bytes) 2024-09-13T09:13:39.378 INFO:teuthology.orchestra.run.smithi157.stdout:5406 trunc from 0xe884800 to 0x5018800 2024-09-13T09:13:39.402 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:39.402+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:39.404 INFO:teuthology.orchestra.run.smithi157.stdout:5412 trunc from 0x5018800 to 0xda2600 2024-09-13T09:13:39.421 INFO:teuthology.orchestra.run.smithi157.stdout:5413 punch from 0x984800 to 0x985a00, (0x1200 bytes) 2024-09-13T09:13:39.421 INFO:teuthology.orchestra.run.smithi157.stdout:5414 read 0x833800 thru 0x83677d (0x2f7e bytes) 2024-09-13T09:13:39.427 INFO:teuthology.orchestra.run.smithi157.stdout:5415 trunc from 0xda2600 to 0x3db0e00 2024-09-13T09:13:39.435 INFO:teuthology.orchestra.run.smithi157.stdout:5418 punch from 0x13af000 to 0x13b6e00, (0x7e00 bytes) 2024-09-13T09:13:39.436 INFO:teuthology.orchestra.run.smithi157.stdout:5419 read 0x1268600 thru 0x12704e3 (0x7ee4 bytes) 2024-09-13T09:13:39.438 INFO:teuthology.orchestra.run.smithi157.stdout:5420 read 0x183aa00 thru 0x18464f5 (0xbaf6 bytes) 2024-09-13T09:13:39.438 INFO:teuthology.orchestra.run.smithi157.stdout:5423 punch from 0x1f98400 to 0x1f9ce00, (0x4a00 bytes) 2024-09-13T09:13:39.440 INFO:teuthology.orchestra.run.smithi157.stdout:5424 trunc from 0x3db0e00 to 0x52c4800 2024-09-13T09:13:39.451 INFO:teuthology.orchestra.run.smithi157.stdout:5425 read 0x1fd9a00 thru 0x1fdc35c (0x295d bytes) 2024-09-13T09:13:39.451 INFO:teuthology.orchestra.run.smithi157.stdout:5426 read 0x441fc00 thru 0x44241b0 (0x45b1 bytes) 2024-09-13T09:13:39.466 INFO:teuthology.orchestra.run.smithi157.stdout:5427 write 0x8bfa600 thru 0x8bfebd2 (0x45d3 bytes) 2024-09-13T09:13:39.467 INFO:teuthology.orchestra.run.smithi157.stdout:5429 write 0x1a51c00 thru 0x1a56c6e (0x506f bytes) 2024-09-13T09:13:39.468 INFO:teuthology.orchestra.run.smithi157.stdout:5430 read 0x511ac00 thru 0x5127f27 (0xd328 bytes) 2024-09-13T09:13:39.468 INFO:teuthology.orchestra.run.smithi157.stdout:5432 read 0x91f000 thru 0x92cd84 (0xdd85 bytes) 2024-09-13T09:13:39.468 INFO:teuthology.orchestra.run.smithi157.stdout:5433 trunc from 0x8bfec00 to 0x8de3e00 2024-09-13T09:13:39.477 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:39.477 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:39.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:39.477 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:39.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.477 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.478 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:39.478 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.94560104, 2024-09-13T09:13:39.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:13:39.478 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.478 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.478 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.478 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.478 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.478 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:13:39.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.478 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.478 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:39.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:39.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:13:39.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:13:39.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:13:39.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:13:39.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:13:39.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:13:39.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:13:39.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:13:39.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:13:39.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:13:39.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:13:39.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:13:39.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:13:39.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:13:39.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:13:39.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:13:39.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:13:39.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:13:39.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:13:39.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:13:39.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:13:39.483 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.483 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.483 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.483 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.484 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:39.484 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.945583005000003, 2024-09-13T09:13:39.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:13:39.484 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.484 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.484 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.484 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.484 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.484 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:13:39.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.484 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:39.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:39.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:39.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:13:39.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:13:39.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:13:39.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:13:39.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:13:39.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:13:39.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:13:39.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:13:39.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:13:39.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:13:39.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:13:39.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:13:39.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:13:39.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:13:39.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:13:39.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:13:39.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:13:39.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:13:39.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:13:39.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:13:39.489 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.489 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.489 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.489 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.489 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:39.489 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.945566335000002, 2024-09-13T09:13:39.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:13:39.490 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.490 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.490 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.490 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.490 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.490 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:13:39.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.490 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:39.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:39.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:13:39.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:13:39.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:39.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:13:39.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:13:39.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:13:39.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:13:39.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:13:39.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:13:39.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:13:39.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:13:39.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:13:39.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:13:39.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:13:39.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:13:39.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:13:39.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:13:39.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:13:39.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:13:39.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:13:39.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:13:39.495 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.495 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.495 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.495 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.495 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:39.496 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.945548703, 2024-09-13T09:13:39.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:13:39.496 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.496 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.496 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.496 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.496 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.496 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:13:39.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.496 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:39.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:39.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:39.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:13:39.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:13:39.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:13:39.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:13:39.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:13:39.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:13:39.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:13:39.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:13:39.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:13:39.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:13:39.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:13:39.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:13:39.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:13:39.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:13:39.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:13:39.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:13:39.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:13:39.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:13:39.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:13:39.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:13:39.501 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.501 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.501 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.501 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.501 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:39.502 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.945502398000002, 2024-09-13T09:13:39.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:13:39.502 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.502 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.502 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.502 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.502 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.502 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:13:39.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.502 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:39.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:39.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:13:39.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:13:39.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:13:39.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:13:39.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:13:39.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:13:39.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:13:39.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:13:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:13:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:13:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:13:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:13:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:13:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:13:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:13:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:13:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:13:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:13:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:13:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:13:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:13:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:39.507 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.945477097000001, 2024-09-13T09:13:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:13:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:13:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:39.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:13:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:13:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:13:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:13:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:13:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:13:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:13:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:13:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:13:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:13:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:13:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:13:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:13:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:13:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:13:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:13:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:13:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:13:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:13:39.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:13:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:13:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.945458868000003, 2024-09-13T09:13:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:13:39.513 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:13:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.514 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:13:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:13:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:13:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:13:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:13:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:13:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:13:39.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:13:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:13:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:13:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:13:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:13:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:13:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:13:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:13:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:13:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:13:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:13:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:13:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:13:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.945411548999999, 2024-09-13T09:13:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:13:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.519 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:13:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:13:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:13:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:13:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:13:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:13:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:13:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:13:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:13:39.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:13:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:13:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:13:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:13:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:13:39.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:13:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:13:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:13:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:13:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:13:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:13:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:13:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.944897310999998, 2024-09-13T09:13:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:13:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.525 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:13:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:39.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:13:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:13:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:13:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:13:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:13:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:13:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:13:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:13:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:13:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:13:39.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:13:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:13:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:13:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:13:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:13:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:13:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:13:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:13:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:13:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:13:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:13:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:13:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.944862780999998, 2024-09-13T09:13:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:13:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.531 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:13:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:39.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:13:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:13:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:13:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:13:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:13:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:13:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:13:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:13:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:13:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:13:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:13:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:13:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:13:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:13:39.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:13:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:13:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:13:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:13:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:13:39.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:13:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.944844873999998, 2024-09-13T09:13:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:13:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.537 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.538 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.538 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:13:39.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.538 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:39.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:39.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:39.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:13:39.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:13:39.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:13:39.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:13:39.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:13:39.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:13:39.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:13:39.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:13:39.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:13:39.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:13:39.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.541 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:13:39.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:13:39.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.541 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:13:39.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:13:39.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:13:39.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:13:39.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:13:39.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:13:39.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:13:39.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:13:39.543 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.543 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.543 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.543 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.543 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:39.543 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.935718026000004, 2024-09-13T09:13:39.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:13:39.543 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.543 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.543 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.543 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.543 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.544 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:13:39.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.544 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:39.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:39.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:39.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:13:39.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:13:39.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:13:39.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:13:39.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:13:39.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:13:39.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:13:39.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:13:39.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:13:39.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:13:39.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:13:39.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:13:39.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:13:39.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:13:39.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:13:39.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:13:39.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:13:39.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:13:39.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:13:39.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:13:39.549 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.549 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.549 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.549 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.549 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:39.549 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.935666445999999, 2024-09-13T09:13:39.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:13:39.549 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.549 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.549 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.550 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.550 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.550 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:13:39.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.550 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:39.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:39.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:13:39.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:13:39.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:13:39.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:13:39.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:13:39.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:13:39.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:13:39.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:13:39.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:13:39.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:13:39.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:13:39.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:13:39.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:13:39.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:13:39.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:13:39.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:13:39.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:13:39.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:13:39.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:13:39.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:13:39.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:13:39.555 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.555 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.555 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.555 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.555 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:39.555 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.935650170000002, 2024-09-13T09:13:39.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:13:39.556 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.556 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.556 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.556 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.556 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.556 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:13:39.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.556 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:39.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:39.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:13:39.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:13:39.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:13:39.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:13:39.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:13:39.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:13:39.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:13:39.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:13:39.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:13:39.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:13:39.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:13:39.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:13:39.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:13:39.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:13:39.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:13:39.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:13:39.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:13:39.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:13:39.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:13:39.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:13:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:13:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.935634395999998, 2024-09-13T09:13:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:13:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.561 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:13:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:13:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:13:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:13:39.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:13:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:13:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:13:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:13:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:13:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:13:39.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:13:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:13:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:13:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:13:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:13:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:13:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:13:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:13:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:13:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:13:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:13:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:13:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.935619521, 2024-09-13T09:13:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:13:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.567 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:13:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:13:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:13:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:13:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:13:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:13:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:13:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:13:39.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:13:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:13:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:13:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:13:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:13:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:13:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:13:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:13:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:13:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:13:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:13:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:13:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:13:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.935602058999997, 2024-09-13T09:13:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:13:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.573 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:13:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:13:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:13:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:13:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:13:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:13:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:13:39.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:13:39.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:13:39.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:13:39.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:13:39.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:13:39.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:13:39.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:13:39.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:13:39.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:13:39.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:13:39.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:13:39.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:13:39.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:13:39.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:13:39.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:13:39.578 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.578 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.578 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.579 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.579 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:39.579 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.934001426999998, 2024-09-13T09:13:39.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:13:39.579 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.579 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.579 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.579 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.579 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.579 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:13:39.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.579 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:39.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:39.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:13:39.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:13:39.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:13:39.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:13:39.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:13:39.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:13:39.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:13:39.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:13:39.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:13:39.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:13:39.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:13:39.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:13:39.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:13:39.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:13:39.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:13:39.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:13:39.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:13:39.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:13:39.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:13:39.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:13:39.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:13:39.584 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.584 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.584 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.585 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.585 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:39.585 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.933961375999999, 2024-09-13T09:13:39.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:13:39.585 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.585 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.585 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.585 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.585 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.585 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:13:39.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.585 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:39.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:39.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:13:39.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:13:39.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:13:39.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:13:39.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:13:39.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:13:39.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:13:39.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:13:39.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:13:39.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:13:39.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:13:39.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:13:39.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:13:39.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:13:39.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:13:39.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:13:39.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:13:39.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:13:39.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:13:39.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:13:39.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:13:39.590 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.590 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.590 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.590 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:39.591 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:39.591 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.933917612999998, 2024-09-13T09:13:39.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:13:39.591 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:39.591 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:39.591 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:39.591 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:39.591 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:39.591 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:13:39.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.591 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:39.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:39.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:39.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:39.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:39.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:39.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:39.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:39.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:39.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:39.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:13:39.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:13:39.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:39.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:13:39.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:13:39.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:39.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:13:39.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:13:39.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:13:39.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:13:39.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:13:39.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:13:39.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:13:39.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:13:39.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:39.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:13:39.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:13:39.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:39.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:13:39.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:13:39.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:39.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:13:39.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:13:39.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:39.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:39.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:39.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:13:39.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:13:39.596 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.596 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.596 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.597 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:39.597 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:39.597 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:39.599 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:39.599 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:39.597+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:39.600 INFO:teuthology.orchestra.run.smithi157.stdout:5437 trunc from 0x8de3e00 to 0xe198800 2024-09-13T09:13:39.600 INFO:teuthology.orchestra.run.smithi157.stdout:5439 punch from 0xaf8800 to 0xaf8c00, (0x400 bytes) 2024-09-13T09:13:39.600 INFO:teuthology.orchestra.run.smithi157.stdout:5441 read 0x95f6600 thru 0x9605d8d (0xf78e bytes) 2024-09-13T09:13:39.600 INFO:teuthology.orchestra.run.smithi157.stdout:5442 write 0xea7c00 thru 0xeb0399 (0x879a bytes) 2024-09-13T09:13:39.600 INFO:teuthology.orchestra.run.smithi157.stdout:5444 trunc from 0xe198800 to 0x2589400 2024-09-13T09:13:39.600 INFO:teuthology.orchestra.run.smithi157.stdout:5445 trunc from 0x2589400 to 0xbbfba00 2024-09-13T09:13:39.600 INFO:teuthology.orchestra.run.smithi157.stdout:5446 write 0x4254800 thru 0x425bfe0 (0x77e1 bytes) 2024-09-13T09:13:39.600 INFO:teuthology.orchestra.run.smithi157.stdout:5447 read 0xabd800 thru 0xac92bc (0xbabd bytes) 2024-09-13T09:13:39.600 INFO:teuthology.orchestra.run.smithi157.stdout:5449 clone 32 order 20 su 65536 sc 2 2024-09-13T09:13:39.603 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:13:39.698 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:39.698+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:39.749 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:39.749 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:39.749 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:39.749 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:39.758 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:39.763 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:39.763 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:39.780 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:39.781 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:39.781 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:13:39.798 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:39.798 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:39.799 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:39.798+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:39.814 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:39.814 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:39.831 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:39.831 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:39.834 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:13:39.835 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:13:39.865 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:39.865 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:39.874 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:39.900 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:39.899+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:40.001 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:40.000+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:40.020 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:40.021 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:40.029 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:40.102 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:40.101+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:40.136 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:40.136 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:40.145 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:40.203 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:40.202+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:40.247 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:40.247 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:40.255 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:40.339 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:40.339+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:40.399 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:40.399 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:40.405 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:40.404+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:40.408 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:40.505 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:40.504+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:40.551 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:40.551 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:40.560 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:40.564 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:13:40.606 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:40.606+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:40.665 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:40.665 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:40.669 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone31 from 0xbbfba00 (overlap 0xda2600) to 0x15d000 2024-09-13T09:13:40.674 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:40.716 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:40.715+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:40.809 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:40.808+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:40.822 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:40.822 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:40.831 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:40.915 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:40.915+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:40.978 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:40.979 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:40.987 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:41.029 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:41.028+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:41.107 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-13T09:13:41.110 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:41.109+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:41.137 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:41.137 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:41.146 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:41.185 INFO:teuthology.orchestra.run.smithi157.stdout:5450 write 0x7ed4c00 thru 0x7ee0638 (0xba39 bytes) 2024-09-13T09:13:41.187 INFO:teuthology.orchestra.run.smithi157.stdout:5451 read 0x39ef400 thru 0x39fcd23 (0xd924 bytes) 2024-09-13T09:13:41.187 INFO:teuthology.orchestra.run.smithi157.stdout:5452 write 0xba98e00 thru 0xbaa344b (0xa64c bytes) 2024-09-13T09:13:41.188 INFO:teuthology.orchestra.run.smithi157.stdout:5453 write 0xa71aa00 thru 0xa721f96 (0x7597 bytes) 2024-09-13T09:13:41.197 INFO:teuthology.orchestra.run.smithi157.stdout:5454 trunc from 0xbbfba00 to 0x44cc800 2024-09-13T09:13:41.211 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:41.210+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:41.227 INFO:teuthology.orchestra.run.smithi157.stdout:5455 read 0x3d6f200 thru 0x3d7b438 (0xc239 bytes) 2024-09-13T09:13:41.239 INFO:teuthology.orchestra.run.smithi157.stdout:5456 trunc from 0x44cc800 to 0xd984e00 2024-09-13T09:13:41.252 INFO:teuthology.orchestra.run.smithi157.stdout:5458 read 0x1d63e00 thru 0x1d659c5 (0x1bc6 bytes) 2024-09-13T09:13:41.260 INFO:teuthology.orchestra.run.smithi157.stdout:5463 write 0xdbd3c00 thru 0xdbd8e3f (0x5240 bytes) 2024-09-13T09:13:41.262 INFO:teuthology.orchestra.run.smithi157.stdout:5464 read 0xa5c0a00 thru 0xa5cd9e1 (0xcfe2 bytes) 2024-09-13T09:13:41.262 INFO:teuthology.orchestra.run.smithi157.stdout:5466 write 0x4bf0400 thru 0x4bffe3d (0xfa3e bytes) 2024-09-13T09:13:41.262 INFO:teuthology.orchestra.run.smithi157.stdout:5468 punch from 0x725da00 to 0x7266000, (0x8600 bytes) 2024-09-13T09:13:41.262 INFO:teuthology.orchestra.run.smithi157.stdout:5471 punch from 0xa26ae00 to 0xa271000, (0x6200 bytes) 2024-09-13T09:13:41.263 INFO:teuthology.orchestra.run.smithi157.stdout:5476 read 0xb628c00 thru 0xb63811f (0xf520 bytes) 2024-09-13T09:13:41.275 INFO:teuthology.orchestra.run.smithi157.stdout:5478 trunc from 0xdbd9000 to 0x370ce00 2024-09-13T09:13:41.295 INFO:teuthology.orchestra.run.smithi157.stdout:5479 read 0x2b2be00 thru 0x2b377a3 (0xb9a4 bytes) 2024-09-13T09:13:41.296 INFO:teuthology.orchestra.run.smithi157.stdout:5482 punch from 0x295600 to 0x29b200, (0x5c00 bytes) 2024-09-13T09:13:41.300 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:41.300 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:41.309 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:41.311 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:41.311+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:41.318 INFO:teuthology.orchestra.run.smithi157.stdout:5484 write 0xe274a00 thru 0xe283552 (0xeb53 bytes) 2024-09-13T09:13:41.325 INFO:teuthology.orchestra.run.smithi157.stdout:5485 write 0x92bb600 thru 0x92c6127 (0xab28 bytes) 2024-09-13T09:13:41.331 INFO:teuthology.orchestra.run.smithi157.stdout:5486 trunc from 0xe283600 to 0x8da8a00 2024-09-13T09:13:41.348 INFO:teuthology.orchestra.run.smithi157.stdout:5487 punch from 0x2e7ca00 to 0x2e82a00, (0x6000 bytes) 2024-09-13T09:13:41.349 INFO:teuthology.orchestra.run.smithi157.stdout:5489 read 0x6016800 thru 0x60172bd (0xabe bytes) 2024-09-13T09:13:41.352 INFO:teuthology.orchestra.run.smithi157.stdout:5491 trunc from 0x8da8a00 to 0x6388c00 2024-09-13T09:13:41.377 INFO:teuthology.orchestra.run.smithi157.stdout:5494 trunc from 0x6388c00 to 0x19a3400 2024-09-13T09:13:41.401 INFO:teuthology.orchestra.run.smithi157.stdout:5495 punch from 0xaa5000 to 0xaaf200, (0xa200 bytes) 2024-09-13T09:13:41.411 INFO:teuthology.orchestra.run.smithi157.stdout:5496 trunc from 0x19a3400 to 0x9e0f200 2024-09-13T09:13:41.412 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:41.411+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:41.420 INFO:teuthology.orchestra.run.smithi157.stdout:5499 write 0x8318400 thru 0x831b3ca (0x2fcb bytes) 2024-09-13T09:13:41.420 INFO:teuthology.orchestra.run.smithi157.stdout:5500 punch from 0x4033c00 to 0x403ee00, (0xb200 bytes) 2024-09-13T09:13:41.421 INFO:teuthology.orchestra.run.smithi157.stdout:5504 punch from 0x74e3e00 to 0x74ece00, (0x9000 bytes) 2024-09-13T09:13:41.421 INFO:teuthology.orchestra.run.smithi157.stdout:5505 read 0x5cb7200 thru 0x5cc3e82 (0xcc83 bytes) 2024-09-13T09:13:41.422 INFO:teuthology.orchestra.run.smithi157.stdout:5506 trunc from 0x9e0f200 to 0x88c9e00 2024-09-13T09:13:41.431 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:41.431 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:41.436 INFO:teuthology.orchestra.run.smithi157.stdout:5507 write 0x6a40200 thru 0x6a4739b (0x719c bytes) 2024-09-13T09:13:41.436 INFO:teuthology.orchestra.run.smithi157.stdout:5508 read 0x76dc000 thru 0x76e021a (0x421b bytes) 2024-09-13T09:13:41.436 INFO:teuthology.orchestra.run.smithi157.stdout:5509 read 0x6dc2200 thru 0x6dcb561 (0x9362 bytes) 2024-09-13T09:13:41.436 INFO:teuthology.orchestra.run.smithi157.stdout:5510 clone 33 order 21 su 65536 sc 7 2024-09-13T09:13:41.441 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:41.513 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:41.512+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-09-13T09:13:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:41.598 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:41.607 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:41.648 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:41.647+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:41.683 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone32 from 0x88c9e00 (overlap 0x19a3400) to 0xcd1a00 2024-09-13T09:13:41.715 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:41.714+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:41.764 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:41.764 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:41.766 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5] 2024-09-13T09:13:41.766 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:13:41.766 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T09:13:41.766 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 1.000000 2024-09-13T09:13:41.766 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-13T09:13:41.773 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:41.816 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:41.815+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:41.836 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:13:41.916 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:41.915+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:41.928 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:41.928 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:41.939 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:41.985 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:42.004 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:42.004 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:42.005 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:13:42.017 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:42.016+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:42.023 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:42.023 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:42.041 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:42.042 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:42.062 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:42.063 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:42.066 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:13:42.066 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:13:42.093 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:42.093 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:42.093 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:42.093 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:42.094 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:42.102 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:42.117 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:42.117+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:42.184 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-13T09:13:42.219 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:42.217+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:42.252 INFO:teuthology.orchestra.run.smithi157.stdout:5512 write 0x40e0a00 thru 0x40efd39 (0xf33a bytes) 2024-09-13T09:13:42.255 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:42.255 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:42.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:42.255 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:42.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.255 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.255 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:42.255 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.749300812999998, 2024-09-13T09:13:42.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:13:42.255 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.255 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.256 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.256 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.256 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.256 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:13:42.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.256 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:42.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:42.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:13:42.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:13:42.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:13:42.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:13:42.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:13:42.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:13:42.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:13:42.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:13:42.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:13:42.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:13:42.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:13:42.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:13:42.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:13:42.259 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.259 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.259 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.260 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.260 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:42.260 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.749273129000002, 2024-09-13T09:13:42.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:13:42.260 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.260 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.260 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.260 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.260 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.260 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:13:42.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.261 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:42.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:42.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:42.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:13:42.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:13:42.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:13:42.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:42.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:13:42.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:13:42.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:13:42.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:13:42.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:13:42.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:13:42.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:13:42.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:13:42.264 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.264 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.264 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.264 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.264 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:42.264 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.749248788000003, 2024-09-13T09:13:42.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:13:42.265 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.265 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.265 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.265 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.265 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.265 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:13:42.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.265 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:42.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:42.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:13:42.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:13:42.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:13:42.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:13:42.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:13:42.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:13:42.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:13:42.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:13:42.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:13:42.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.268 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:13:42.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:13:42.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.268 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:13:42.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:13:42.268 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.268 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.269 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.269 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.269 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:42.269 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.749232468000002, 2024-09-13T09:13:42.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:13:42.269 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.269 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.269 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.269 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.269 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.269 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:13:42.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.270 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:42.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:42.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:42.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:13:42.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:42.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:13:42.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:13:42.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:13:42.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:13:42.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:13:42.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:13:42.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:13:42.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:13:42.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.273 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:13:42.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:13:42.273 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.273 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.273 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.273 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.273 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:42.273 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.749216052999998, 2024-09-13T09:13:42.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:13:42.274 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.274 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.274 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.274 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.274 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.274 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:13:42.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.274 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.274 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.274 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:42.274 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:42.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:42.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:13:42.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:13:42.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:13:42.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:13:42.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:13:42.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:13:42.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:13:42.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:13:42.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:13:42.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:13:42.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:13:42.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:13:42.277 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.278 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.278 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.278 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.278 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:42.278 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.749199945000001, 2024-09-13T09:13:42.278 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:13:42.278 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.278 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.278 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.278 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.278 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.278 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:13:42.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.279 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.279 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.279 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:42.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.279 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.279 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:42.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.279 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.280 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:42.280 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.280 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.280 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:13:42.280 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:13:42.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.280 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.280 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:13:42.280 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:13:42.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:13:42.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:13:42.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:13:42.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:13:42.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:13:42.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:13:42.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.282 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.282 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:13:42.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:13:42.282 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.282 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.282 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.282 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.282 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:42.282 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.749184384000003, 2024-09-13T09:13:42.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:13:42.283 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.283 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.283 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.283 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.283 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.283 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:13:42.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.283 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:42.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:42.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:42.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:13:42.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:13:42.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:13:42.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:13:42.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:13:42.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:13:42.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:13:42.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:13:42.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.286 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.286 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:13:42.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:13:42.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.286 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.286 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:13:42.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:13:42.286 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.286 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.286 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.287 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.287 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:42.287 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.749167546000002, 2024-09-13T09:13:42.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:13:42.287 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.287 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.287 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.287 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.287 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.287 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:13:42.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.287 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:42.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:42.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:13:42.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:13:42.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:13:42.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:13:42.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:13:42.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:13:42.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:13:42.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:13:42.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:13:42.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:13:42.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:13:42.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.291 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:13:42.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:13:42.291 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.291 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.291 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.291 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.291 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:42.291 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.749151554000001, 2024-09-13T09:13:42.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:13:42.291 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.292 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.292 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:13:42.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.292 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.292 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.292 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:42.292 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.292 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:42.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:13:42.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:13:42.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:13:42.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:13:42.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:13:42.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:13:42.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:13:42.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:13:42.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:13:42.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:13:42.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:13:42.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:13:42.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:13:42.295 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.295 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.295 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.296 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.296 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:42.296 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.749124055999999, 2024-09-13T09:13:42.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:13:42.296 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.296 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.296 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.296 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.296 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.296 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:13:42.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.297 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.297 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.297 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:42.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.297 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.297 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:42.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.297 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.297 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:42.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:13:42.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:13:42.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:13:42.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:13:42.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:13:42.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:13:42.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:13:42.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:13:42.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:13:42.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:13:42.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:13:42.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:13:42.300 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.300 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.300 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.300 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.300 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:42.300 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.749098197000002, 2024-09-13T09:13:42.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:13:42.300 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.301 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.301 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.301 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.301 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.301 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:13:42.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.301 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:42.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.302 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:42.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.302 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.302 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:42.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.302 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.302 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:13:42.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:13:42.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:13:42.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:13:42.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:13:42.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:13:42.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:13:42.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:13:42.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:13:42.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:13:42.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:13:42.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:13:42.305 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.305 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.305 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.305 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.305 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:42.305 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.749056707000001, 2024-09-13T09:13:42.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:13:42.305 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.305 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.305 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.306 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.306 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.306 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:13:42.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.306 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.306 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:42.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.306 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:42.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:13:42.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:13:42.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:13:42.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:13:42.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:13:42.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:13:42.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:13:42.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:13:42.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:13:42.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:13:42.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:13:42.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:13:42.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:13:42.309 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.309 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.309 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.310 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.310 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:42.310 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.749035468999999, 2024-09-13T09:13:42.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:13:42.310 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.310 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.310 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.310 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.310 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.310 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:13:42.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.310 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:42.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.311 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.311 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:42.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.311 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.311 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:42.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.311 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:13:42.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:13:42.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:13:42.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:13:42.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:13:42.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:13:42.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:13:42.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:13:42.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:13:42.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:13:42.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:13:42.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:13:42.314 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.314 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.314 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.314 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.314 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:42.314 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.749012669000003, 2024-09-13T09:13:42.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:13:42.314 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.314 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.314 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.314 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.315 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.315 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:13:42.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.315 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:42.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:42.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:13:42.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:13:42.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:13:42.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:13:42.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:42.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:13:42.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:13:42.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:13:42.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:13:42.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:13:42.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:13:42.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:13:42.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:13:42.318 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.318 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.318 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.319 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.319 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:42.319 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.748996212999998, 2024-09-13T09:13:42.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:13:42.319 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.319 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.319 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.319 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.319 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.319 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:13:42.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.319 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:42.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:42.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:42.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:13:42.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:13:42.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:13:42.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:13:42.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:13:42.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:13:42.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:13:42.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:13:42.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:13:42.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:13:42.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:13:42.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:13:42.323 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.323 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.323 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.323 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.323 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:42.323 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.748751075999998, 2024-09-13T09:13:42.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:13:42.324 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.324 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.324 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.324 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.324 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.324 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:13:42.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.324 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.324 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.324 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:42.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:42.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:13:42.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:13:42.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:13:42.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:13:42.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:13:42.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:13:42.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:13:42.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:42.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:13:42.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:13:42.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:13:42.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:13:42.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:13:42.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:13:42.327 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.327 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.328 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.328 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.328 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:42.328 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.746865517000003, 2024-09-13T09:13:42.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:13:42.328 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.328 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.328 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.328 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.328 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.328 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:13:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:13:42.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:13:42.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:13:42.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:13:42.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:13:42.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:13:42.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:13:42.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:13:42.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:13:42.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:13:42.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:13:42.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:13:42.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:13:42.332 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.332 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.332 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.332 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.332 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:42.332 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.746844150999998, 2024-09-13T09:13:42.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:13:42.332 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.333 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.333 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.333 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.333 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.333 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:13:42.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.333 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:42.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:42.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:42.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:13:42.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:13:42.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:13:42.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:13:42.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:13:42.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:13:42.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:13:42.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:13:42.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:13:42.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:13:42.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:13:42.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:13:42.336 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.336 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.336 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.337 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:42.337 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:42.337 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.746638783000002, 2024-09-13T09:13:42.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:13:42.337 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.337 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:42.337 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.337 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:42.337 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.337 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:13:42.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.338 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:42.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:42.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:13:42.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:13:42.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:13:42.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:13:42.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:13:42.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:13:42.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:13:42.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:13:42.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:13:42.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:13:42.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:13:42.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:13:42.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:13:42.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:13:42.341 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.341 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.341 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.341 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5272.0:1775 3.0 3:02a42fca:::rbd_data.11235782ee8d.0000000000000030:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2244608~40448] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:13:42.341 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:42.341 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.005096942000002, 2024-09-13T09:13:42.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.666843732, 2024-09-13T09:13:42.341 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:42.341 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:42.342 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:42.342 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5272", 2024-09-13T09:13:42.342 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4089902097", 2024-09-13T09:13:42.342 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1775 2024-09-13T09:13:42.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.342 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:42.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:42.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:42.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:42.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:42.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:42.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:42.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247780+0000", 2024-09-13T09:13:42.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:13:42.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:42.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247865+0000", 2024-09-13T09:13:42.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4359999999999999e-05 2024-09-13T09:13:42.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:42.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247876+0000", 2024-09-13T09:13:42.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1456e-05 2024-09-13T09:13:42.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247883+0000", 2024-09-13T09:13:42.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:13:42.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247926+0000", 2024-09-13T09:13:42.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3203e-05 2024-09-13T09:13:42.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247932+0000", 2024-09-13T09:13:42.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:13:42.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909639+0000", 2024-09-13T09:13:42.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6617069099999999 2024-09-13T09:13:42.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:13:42.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909828+0000", 2024-09-13T09:13:42.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000189075 2024-09-13T09:13:42.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:42.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911676+0000", 2024-09-13T09:13:42.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001847235 2024-09-13T09:13:42.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:42.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911765+0000", 2024-09-13T09:13:42.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9530000000000005e-05 2024-09-13T09:13:42.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:13:42.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911901+0000", 2024-09-13T09:13:42.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013603899999999999 2024-09-13T09:13:42.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:13:42.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.912889+0000", 2024-09-13T09:13:42.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098807400000000003 2024-09-13T09:13:42.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:42.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914438+0000", 2024-09-13T09:13:42.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015491070000000001 2024-09-13T09:13:42.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:13:42.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914505+0000", 2024-09-13T09:13:42.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:13:42.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:42.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:42.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:42.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914625+0000", 2024-09-13T09:13:42.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120108 2024-09-13T09:13:42.348 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.348 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.348 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.348 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:42.348 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:42.348 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:42.350 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:42.352 INFO:teuthology.orchestra.run.smithi157.stdout:5516 write 0xc178e00 thru 0xc17dacd (0x4cce bytes) 2024-09-13T09:13:42.352 INFO:teuthology.orchestra.run.smithi157.stdout:5517 write 0x7411c00 thru 0x74158f4 (0x3cf5 bytes) 2024-09-13T09:13:42.353 INFO:teuthology.orchestra.run.smithi157.stdout:5518 trunc from 0xc17dc00 to 0xa2e8c00 2024-09-13T09:13:42.353 INFO:teuthology.orchestra.run.smithi157.stdout:5519 write 0x38e3400 thru 0x38ef032 (0xbc33 bytes) 2024-09-13T09:13:42.353 INFO:teuthology.orchestra.run.smithi157.stdout:5522 punch from 0x1814400 to 0x1819c00, (0x5800 bytes) 2024-09-13T09:13:42.353 INFO:teuthology.orchestra.run.smithi157.stdout:5524 read 0x2351600 thru 0x2358497 (0x6e98 bytes) 2024-09-13T09:13:42.353 INFO:teuthology.orchestra.run.smithi157.stdout:5525 write 0xc1ab600 thru 0xc1ae601 (0x3002 bytes) 2024-09-13T09:13:42.353 INFO:teuthology.orchestra.run.smithi157.stdout:5527 read 0x456fe00 thru 0x4576e1e (0x701f bytes) 2024-09-13T09:13:42.353 INFO:teuthology.orchestra.run.smithi157.stdout:5529 read 0x248e000 thru 0x249c345 (0xe346 bytes) 2024-09-13T09:13:42.353 INFO:teuthology.orchestra.run.smithi157.stdout:5530 trunc from 0xc1ae800 to 0xe672e00 2024-09-13T09:13:42.353 INFO:teuthology.orchestra.run.smithi157.stdout:5532 write 0xd1ff600 thru 0xd20eca3 (0xf6a4 bytes) 2024-09-13T09:13:42.353 INFO:teuthology.orchestra.run.smithi157.stdout:5533 read 0x13b6800 thru 0x13be676 (0x7e77 bytes) 2024-09-13T09:13:42.353 INFO:teuthology.orchestra.run.smithi157.stdout:5535 punch from 0xd777c00 to 0xd783e00, (0xc200 bytes) 2024-09-13T09:13:42.353 INFO:teuthology.orchestra.run.smithi157.stdout:5536 read 0xe2bea00 thru 0xe2c2453 (0x3a54 bytes) 2024-09-13T09:13:42.353 INFO:teuthology.orchestra.run.smithi157.stdout:5538 punch from 0x7566200 to 0x7575400, (0xf200 bytes) 2024-09-13T09:13:42.354 INFO:teuthology.orchestra.run.smithi157.stdout:5540 write 0x9e0fc00 thru 0x9e180b2 (0x84b3 bytes) 2024-09-13T09:13:42.354 INFO:teuthology.orchestra.run.smithi157.stdout:5543 read 0x9735000 thru 0x97373bb (0x23bc bytes) 2024-09-13T09:13:42.354 INFO:teuthology.orchestra.run.smithi157.stdout:5544 punch from 0x901ea00 to 0x9026800, (0x7e00 bytes) 2024-09-13T09:13:42.354 INFO:teuthology.orchestra.run.smithi157.stdout:5547 punch from 0xdf00400 to 0xdf0f800, (0xf400 bytes) 2024-09-13T09:13:42.354 INFO:teuthology.orchestra.run.smithi157.stdout:5548 read 0x6076600 thru 0x6079ca8 (0x36a9 bytes) 2024-09-13T09:13:42.354 INFO:teuthology.orchestra.run.smithi157.stdout:5549 write 0x5952200 thru 0x59595c1 (0x73c2 bytes) 2024-09-13T09:13:42.354 INFO:teuthology.orchestra.run.smithi157.stdout:5550 read 0x2bc2a00 thru 0x2bcec34 (0xc235 bytes) 2024-09-13T09:13:42.354 INFO:teuthology.orchestra.run.smithi157.stdout:5551 write 0x4b83e00 thru 0x4b8d4ef (0x96f0 bytes) 2024-09-13T09:13:42.354 INFO:teuthology.orchestra.run.smithi157.stdout:5553 read 0xd0e6a00 thru 0xd0f0658 (0x9c59 bytes) 2024-09-13T09:13:42.355 INFO:teuthology.orchestra.run.smithi157.stdout:5556 trunc from 0xe672e00 to 0xfdec00 2024-09-13T09:13:42.355 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:42.348+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:42.382 INFO:teuthology.orchestra.run.smithi157.stdout:5557 punch from 0xc9f600 to 0xca7800, (0x8200 bytes) 2024-09-13T09:13:42.390 INFO:teuthology.orchestra.run.smithi157.stdout:5558 trunc from 0xfdec00 to 0x713de00 2024-09-13T09:13:42.410 INFO:teuthology.orchestra.run.smithi157.stdout:5559 write 0x4fd4400 thru 0x4fe228b (0xde8c bytes) 2024-09-13T09:13:42.438 INFO:teuthology.orchestra.run.smithi157.stdout:5561 write 0xd7f1a00 thru 0xd7fa6ae (0x8caf bytes) 2024-09-13T09:13:42.438 INFO:teuthology.orchestra.run.smithi157.stdout:5565 write 0x385be00 thru 0x3863f13 (0x8114 bytes) 2024-09-13T09:13:42.438 INFO:teuthology.orchestra.run.smithi157.stdout:5567 write 0xcf08c00 thru 0xcf0c06e (0x346f bytes) 2024-09-13T09:13:42.438 INFO:teuthology.orchestra.run.smithi157.stdout:5569 write 0x52fa400 thru 0x5303b87 (0x9788 bytes) 2024-09-13T09:13:42.439 INFO:teuthology.orchestra.run.smithi157.stdout:5570 read 0x9160800 thru 0x91622ab (0x1aac bytes) 2024-09-13T09:13:42.439 INFO:teuthology.orchestra.run.smithi157.stdout:5571 trunc from 0xd7fa800 to 0xc96f000 2024-09-13T09:13:42.451 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:42.449+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:42.462 INFO:teuthology.orchestra.run.smithi157.stdout:5573 clone 34 order 23 su 131072 sc 9 2024-09-13T09:13:42.467 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:42.467 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:42.467 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:42.468 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:42.477 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:42.561 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:42.559+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:42.579 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-13T09:13:42.621 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:42.622 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:42.622 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:42.622 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:42.622 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:42.631 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:42.672 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:42.670+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:42.688 INFO:teuthology.orchestra.run.smithi157.stdout:flattening image image_client.0-clone33 2024-09-13T09:13:42.753 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:42.751+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:42.782 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:42.782 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:42.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:42.783 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:42.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.783 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.783 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:42.783 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.259514058000001, 2024-09-13T09:13:42.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:13:42.783 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.783 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.783 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.783 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.783 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.783 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:13:42.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.784 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:42.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:42.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:42.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:13:42.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:13:42.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:13:42.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:13:42.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:13:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:13:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:13:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:13:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:13:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:13:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:13:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:13:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:13:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:13:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.788 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.788 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:42.788 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.259465001999999, 2024-09-13T09:13:42.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:13:42.788 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.788 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.788 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:13:42.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.788 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:42.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:42.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:42.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:13:42.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:13:42.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:13:42.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:13:42.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:13:42.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:13:42.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:13:42.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:13:42.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:13:42.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:13:42.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:13:42.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:13:42.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:13:42.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:13:42.792 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.792 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.792 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.792 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.792 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:42.792 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.259424926999998, 2024-09-13T09:13:42.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:13:42.793 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.793 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.793 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.793 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.793 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.793 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:13:42.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.793 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:42.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:42.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:42.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:13:42.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:13:42.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:13:42.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:13:42.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:13:42.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:13:42.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:13:42.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:13:42.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:13:42.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:13:42.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:13:42.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:13:42.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:13:42.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:13:42.797 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.797 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.797 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.797 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.797 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:42.797 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.259395112, 2024-09-13T09:13:42.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:13:42.797 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.797 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.797 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.798 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.798 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.798 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:13:42.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.798 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:42.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:42.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:42.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:13:42.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:13:42.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:13:42.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:42.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:13:42.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:13:42.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:13:42.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:13:42.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:13:42.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:13:42.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:13:42.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:13:42.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:13:42.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:13:42.801 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.802 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.802 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.802 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.802 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:42.802 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.259365451000001, 2024-09-13T09:13:42.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:13:42.802 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.802 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.802 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.802 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.802 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.802 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:13:42.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.803 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:42.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:42.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:42.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:13:42.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:13:42.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:13:42.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:13:42.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:13:42.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:13:42.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:13:42.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:13:42.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:13:42.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:13:42.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:13:42.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:13:42.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:13:42.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:13:42.806 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.806 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.806 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.807 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.807 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:42.807 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.259336243999996, 2024-09-13T09:13:42.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:13:42.807 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.807 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.807 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.807 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.807 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.807 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:13:42.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.807 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:42.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:42.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:42.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:13:42.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:13:42.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:13:42.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:13:42.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:13:42.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:13:42.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:13:42.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:13:42.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:13:42.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:13:42.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:13:42.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:13:42.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:13:42.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:13:42.811 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.811 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.811 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.812 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.812 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:42.812 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.259280062999999, 2024-09-13T09:13:42.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:13:42.812 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.812 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.812 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.812 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.812 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.812 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:13:42.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.812 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:42.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:42.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:42.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:13:42.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:13:42.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:13:42.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:13:42.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:13:42.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:13:42.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:13:42.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:13:42.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:13:42.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:13:42.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:13:42.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:13:42.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:13:42.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:13:42.817 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.817 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.817 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.817 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.818 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:42.818 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.259226601999998, 2024-09-13T09:13:42.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:13:42.818 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.818 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.818 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.818 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.818 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.818 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:13:42.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.818 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:42.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:42.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:13:42.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:13:42.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:13:42.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:13:42.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:13:42.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:13:42.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:13:42.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:13:42.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:13:42.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:13:42.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:13:42.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:13:42.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:13:42.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:13:42.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:13:42.822 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.822 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.822 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.822 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.822 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:42.822 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.259169772, 2024-09-13T09:13:42.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:13:42.822 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.823 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.823 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.823 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.823 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.823 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:13:42.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.823 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:42.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:42.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:13:42.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:13:42.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:13:42.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:13:42.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:13:42.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:13:42.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:13:42.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:13:42.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:13:42.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:13:42.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:13:42.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:13:42.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:13:42.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:13:42.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:13:42.827 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.827 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.827 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.827 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.827 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:42.827 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.259118745000002, 2024-09-13T09:13:42.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:13:42.828 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.828 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.828 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.828 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.828 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.828 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:13:42.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.828 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:42.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:42.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:13:42.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:13:42.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:13:42.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:13:42.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:13:42.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:13:42.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:13:42.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:13:42.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:13:42.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:13:42.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:13:42.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:13:42.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:13:42.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:13:42.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:13:42.832 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.832 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.832 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.832 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.832 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:42.832 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.259077554999998, 2024-09-13T09:13:42.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:13:42.832 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.832 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.833 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.833 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.833 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.833 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:13:42.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.833 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:42.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:13:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:13:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:13:42.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:13:42.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:13:42.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:13:42.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:13:42.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:13:42.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:13:42.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:13:42.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:13:42.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:13:42.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:13:42.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:13:42.836 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.837 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.837 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.837 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.837 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:42.837 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.259035439000002, 2024-09-13T09:13:42.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:13:42.837 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.837 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.837 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.837 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.837 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.838 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:13:42.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.838 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:42.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:42.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:42.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:42.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:13:42.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:13:42.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:13:42.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:13:42.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:13:42.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:13:42.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:13:42.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:13:42.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:13:42.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:13:42.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:13:42.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:13:42.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:13:42.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:13:42.841 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.841 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.841 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.842 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.842 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:42.842 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.258991559999998, 2024-09-13T09:13:42.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:13:42.842 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.842 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.842 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.842 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.842 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.842 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:13:42.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.842 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:42.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:42.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:42.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:42.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:13:42.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:13:42.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:13:42.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:13:42.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:13:42.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:13:42.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:13:42.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:13:42.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:13:42.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:13:42.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:13:42.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:13:42.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:13:42.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:13:42.846 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.846 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.846 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.846 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.846 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:42.847 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.258945642999997, 2024-09-13T09:13:42.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:13:42.847 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.847 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.847 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:13:42.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.847 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:42.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:42.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:13:42.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:13:42.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:13:42.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:13:42.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:13:42.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:13:42.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:13:42.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:13:42.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:13:42.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:13:42.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:13:42.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:13:42.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:13:42.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:13:42.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:13:42.851 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.851 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.851 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.851 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.851 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:42.851 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.258895871999997, 2024-09-13T09:13:42.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:13:42.851 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.851 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.852 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:13:42.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.852 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:42.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:42.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:42.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:42.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:13:42.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:13:42.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:13:42.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:13:42.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:13:42.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:13:42.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:13:42.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:13:42.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:13:42.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:13:42.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:13:42.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:13:42.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:13:42.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:13:42.855 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.856 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.856 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.856 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:42.856 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.258850467999999, 2024-09-13T09:13:42.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:13:42.856 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.856 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.856 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.856 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.856 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.856 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:13:42.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.857 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:42.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:42.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:42.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:42.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:13:42.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:13:42.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:13:42.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:13:42.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:13:42.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:13:42.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:13:42.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:13:42.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:13:42.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:13:42.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:13:42.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:13:42.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:13:42.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:13:42.860 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.860 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.860 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.861 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.861 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:42.861 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.258799830000001, 2024-09-13T09:13:42.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:13:42.861 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.861 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.861 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.861 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.861 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.861 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:13:42.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.861 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:42.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:42.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:13:42.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:13:42.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:13:42.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:13:42.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:13:42.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:13:42.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:13:42.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:13:42.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:13:42.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:13:42.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:13:42.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:13:42.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:13:42.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:13:42.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:13:42.865 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.865 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.865 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.865 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.866 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:42.866 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.258743987000003, 2024-09-13T09:13:42.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:13:42.866 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.866 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.866 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.866 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.866 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.866 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:13:42.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.866 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:42.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:42.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:42.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:13:42.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:13:42.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:13:42.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:13:42.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:13:42.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:13:42.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:13:42.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:13:42.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:13:42.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:13:42.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:13:42.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:13:42.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:13:42.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:13:42.870 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.870 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.870 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.870 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.870 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:42.870 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.258687064999997, 2024-09-13T09:13:42.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:13:42.871 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.871 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.871 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.871 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.871 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.871 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:13:42.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.871 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:42.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:42.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:42.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:42.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:13:42.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:13:42.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:13:42.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:13:42.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:13:42.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:13:42.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:13:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:13:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:13:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:13:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:13:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:13:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:13:42.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:13:42.875 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.875 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.875 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.875 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:42.875 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:42.875 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 60.258633502000002, 2024-09-13T09:13:42.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:13:42.875 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:42.875 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:42.875 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:42.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:42.876 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:42.876 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:13:42.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.876 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:42.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:42.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:42.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:42.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:42.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:42.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:42.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:42.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:42.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:42.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:13:42.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:13:42.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:42.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:13:42.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:13:42.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:42.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:13:42.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:13:42.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:13:42.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:13:42.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:42.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:13:42.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:13:42.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:42.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:13:42.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:13:42.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:42.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:42.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:42.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:13:42.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:13:42.879 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.879 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.879 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.880 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:42.880 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:42.880 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:42.881 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:42.882 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:42.879+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:42.982 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:42.980+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:43.031 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:43.032 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:43.032 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:43.032 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:43.041 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:43.082 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:43.080+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:43.183 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:43.182+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:43.184 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-13T09:13:43.188 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:43.188 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:43.188 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:43.188 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:43.188 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:43.198 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:43.283 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:43.282+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:43.305 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:43.305 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:43.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:43.305 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:43.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.305 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2185 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.306 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:43.306 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777575366000001, 2024-09-13T09:13:43.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063348994000000006, 2024-09-13T09:13:43.306 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.306 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.306 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.306 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.306 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.306 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2185 2024-09-13T09:13:43.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.306 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:43.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:43.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:43.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525765+0000", 2024-09-13T09:13:43.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2390000000000001e-06 2024-09-13T09:13:43.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525772+0000", 2024-09-13T09:13:43.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:13:43.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525777+0000", 2024-09-13T09:13:43.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0799999999999996e-06 2024-09-13T09:13:43.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589020+0000", 2024-09-13T09:13:43.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063242501000000007 2024-09-13T09:13:43.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589063+0000", 2024-09-13T09:13:43.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2673999999999998e-05 2024-09-13T09:13:43.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589112+0000", 2024-09-13T09:13:43.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9286000000000002e-05 2024-09-13T09:13:43.310 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.310 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.310 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.311 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2187 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.311 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:43.311 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777555819, 2024-09-13T09:13:43.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063500102000000003, 2024-09-13T09:13:43.311 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.311 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.311 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.311 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.311 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.311 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2187 2024-09-13T09:13:43.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.312 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:43.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:43.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:43.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525784+0000", 2024-09-13T09:13:43.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1629999999999999e-06 2024-09-13T09:13:43.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525792+0000", 2024-09-13T09:13:43.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:43.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525796+0000", 2024-09-13T09:13:43.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7389999999999999e-06 2024-09-13T09:13:43.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589132+0000", 2024-09-13T09:13:43.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063336119999999996 2024-09-13T09:13:43.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589187+0000", 2024-09-13T09:13:43.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4126000000000003e-05 2024-09-13T09:13:43.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589283+0000", 2024-09-13T09:13:43.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6032999999999993e-05 2024-09-13T09:13:43.315 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.315 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.315 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.315 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2188 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.315 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:43.315 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777537772999999, 2024-09-13T09:13:43.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063724617999999997, 2024-09-13T09:13:43.316 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.316 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.316 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.316 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.316 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.316 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2188 2024-09-13T09:13:43.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.316 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:43.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:43.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:43.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525802+0000", 2024-09-13T09:13:43.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0439999999999998e-06 2024-09-13T09:13:43.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525809+0000", 2024-09-13T09:13:43.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.238e-06 2024-09-13T09:13:43.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525814+0000", 2024-09-13T09:13:43.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-13T09:13:43.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589327+0000", 2024-09-13T09:13:43.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063513257000000004 2024-09-13T09:13:43.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589418+0000", 2024-09-13T09:13:43.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0796000000000003e-05 2024-09-13T09:13:43.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589525+0000", 2024-09-13T09:13:43.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010693500000000001 2024-09-13T09:13:43.319 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.319 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.319 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.320 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2190 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.320 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:43.320 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777519687000002, 2024-09-13T09:13:43.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063863981, 2024-09-13T09:13:43.320 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.320 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.320 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2190 2024-09-13T09:13:43.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.321 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:43.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:43.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525818+0000", 2024-09-13T09:13:43.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525820+0000", 2024-09-13T09:13:43.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:13:43.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525827+0000", 2024-09-13T09:13:43.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0589999999999999e-06 2024-09-13T09:13:43.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525831+0000", 2024-09-13T09:13:43.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7870000000000001e-06 2024-09-13T09:13:43.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589562+0000", 2024-09-13T09:13:43.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063731038000000004 2024-09-13T09:13:43.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589612+0000", 2024-09-13T09:13:43.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0087999999999999e-05 2024-09-13T09:13:43.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589683+0000", 2024-09-13T09:13:43.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0121000000000002e-05 2024-09-13T09:13:43.324 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.324 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.324 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.324 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2192 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.325 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:43.325 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777502727000002, 2024-09-13T09:13:43.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064050299000000005, 2024-09-13T09:13:43.325 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.325 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.325 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.325 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.325 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.325 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2192 2024-09-13T09:13:43.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.325 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:43.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:43.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525835+0000", 2024-09-13T09:13:43.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525837+0000", 2024-09-13T09:13:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:13:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525845+0000", 2024-09-13T09:13:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3660000000000003e-06 2024-09-13T09:13:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525850+0000", 2024-09-13T09:13:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0030000000000002e-06 2024-09-13T09:13:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589720+0000", 2024-09-13T09:13:43.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063870176000000001 2024-09-13T09:13:43.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589801+0000", 2024-09-13T09:13:43.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0889000000000004e-05 2024-09-13T09:13:43.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589886+0000", 2024-09-13T09:13:43.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5183000000000001e-05 2024-09-13T09:13:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777484624, 2024-09-13T09:13:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:13:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.330 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:13:43.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.330 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:43.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:43.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:13:43.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:13:43.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:13:43.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:13:43.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:13:43.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:13:43.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:13:43.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:13:43.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:13:43.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:13:43.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:13:43.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:13:43.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:13:43.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.333 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.333 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.333 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:43.334 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777466925999999, 2024-09-13T09:13:43.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:13:43.334 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.334 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.334 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.334 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.334 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.334 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:13:43.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.334 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:43.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:43.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:43.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:13:43.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:13:43.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:13:43.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:13:43.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:13:43.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:13:43.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:13:43.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:13:43.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:13:43.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:13:43.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:13:43.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:13:43.338 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.338 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.338 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.338 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.338 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:43.338 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777446310999999, 2024-09-13T09:13:43.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:13:43.338 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.338 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.338 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.339 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.339 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.339 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:13:43.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.339 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:43.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:43.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:43.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:13:43.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:13:43.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:13:43.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:13:43.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:13:43.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:13:43.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:13:43.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:13:43.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:13:43.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:13:43.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:13:43.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:13:43.342 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.342 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.342 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.343 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.343 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:43.343 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777427355, 2024-09-13T09:13:43.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:13:43.343 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.343 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.343 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.343 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.343 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.343 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:13:43.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.343 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:43.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:43.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:43.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:13:43.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:13:43.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:13:43.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:13:43.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:13:43.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:13:43.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:13:43.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:13:43.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:13:43.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:13:43.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:13:43.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:13:43.347 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.347 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.347 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.347 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.347 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:43.347 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777410097000001, 2024-09-13T09:13:43.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:13:43.347 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.347 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.347 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.347 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.348 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.348 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:13:43.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.348 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:43.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:43.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:43.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:13:43.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:13:43.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:13:43.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:13:43.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:13:43.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:13:43.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:13:43.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:13:43.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:13:43.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:13:43.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:13:43.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:13:43.351 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.351 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.352 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.352 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.352 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:43.352 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777390083, 2024-09-13T09:13:43.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:13:43.352 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.352 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.352 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.352 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.352 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.352 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:13:43.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.353 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:43.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:43.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:43.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:13:43.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:13:43.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:13:43.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:13:43.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:13:43.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:13:43.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:13:43.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:13:43.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:13:43.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:13:43.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:13:43.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:13:43.356 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.356 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.357 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.357 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.357 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:43.357 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777364501000001, 2024-09-13T09:13:43.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:13:43.357 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.357 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.357 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.357 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.357 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.357 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:13:43.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.358 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:43.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:43.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:13:43.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:13:43.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:13:43.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:13:43.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:13:43.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:13:43.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:13:43.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:13:43.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:13:43.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:13:43.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:13:43.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:13:43.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:13:43.361 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.361 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.361 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.361 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.361 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:43.361 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777345217000001, 2024-09-13T09:13:43.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:13:43.362 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.362 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.362 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:13:43.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.362 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:43.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:43.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:43.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:13:43.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:13:43.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:13:43.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:13:43.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:13:43.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:13:43.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:13:43.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:13:43.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:13:43.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:13:43.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:13:43.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:13:43.366 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.366 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.366 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.366 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.366 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:43.366 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777327961000001, 2024-09-13T09:13:43.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:13:43.366 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.366 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.366 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.366 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.366 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.367 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:13:43.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.367 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:43.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:43.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:43.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:13:43.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:13:43.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:13:43.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:13:43.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:13:43.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:13:43.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:13:43.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:13:43.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:13:43.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:13:43.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:13:43.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:13:43.370 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.370 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.370 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.371 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.371 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:43.371 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777308480999999, 2024-09-13T09:13:43.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:13:43.371 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.371 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.371 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:13:43.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.372 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:43.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:43.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:13:43.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:13:43.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:13:43.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:13:43.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:13:43.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:13:43.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:13:43.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:13:43.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:13:43.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:13:43.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:13:43.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:13:43.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:13:43.375 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.375 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.375 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.375 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.375 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:43.376 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777291552000001, 2024-09-13T09:13:43.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:13:43.376 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.376 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.376 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:13:43.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.376 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:43.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:43.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:13:43.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:13:43.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:13:43.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:13:43.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:13:43.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:13:43.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:13:43.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:13:43.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:13:43.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:13:43.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:13:43.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:13:43.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:13:43.380 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.380 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.380 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.380 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.380 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:43.380 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777272403000001, 2024-09-13T09:13:43.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:13:43.381 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.381 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.381 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.381 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.381 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.381 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:13:43.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.381 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:43.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:43.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:43.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:13:43.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:13:43.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:13:43.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:13:43.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:13:43.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:13:43.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:13:43.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:13:43.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:13:43.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:13:43.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:13:43.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:13:43.385 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.385 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.385 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.385 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.385 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:43.385 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.777254264, 2024-09-13T09:13:43.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:13:43.385 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.385 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.385 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.385 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.385 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.386 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:13:43.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.386 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:43.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:43.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:43.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:13:43.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:13:43.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:13:43.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:13:43.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:13:43.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:13:43.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:13:43.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:13:43.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:13:43.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:13:43.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:13:43.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:13:43.389 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.389 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.389 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.389 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.389 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:43.389 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.776843768999999, 2024-09-13T09:13:43.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:13:43.390 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.390 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.390 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.390 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.390 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.390 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:13:43.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.390 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:43.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:43.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:43.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:13:43.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:13:43.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:13:43.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:13:43.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:13:43.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:13:43.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:13:43.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:13:43.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:13:43.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:13:43.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:13:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:13:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.776820868000001, 2024-09-13T09:13:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:13:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:13:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:13:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:13:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:13:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:13:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:13:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:13:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:13:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:13:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:13:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:13:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:13:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:13:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.399 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:43.400 INFO:teuthology.orchestra.run.smithi157.stdout:5575 trunc from 0xc96f000 to 0x432ca00 2024-09-13T09:13:43.400 INFO:teuthology.orchestra.run.smithi157.stdout:5576 write 0x3117e00 thru 0x31235a0 (0xb7a1 bytes) 2024-09-13T09:13:43.400 INFO:teuthology.orchestra.run.smithi157.stdout:5578 trunc from 0x432ca00 to 0x126f000 2024-09-13T09:13:43.401 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:43.401 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:43.398+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:43.434 INFO:teuthology.orchestra.run.smithi157.stdout:5580 write 0xeb12200 thru 0xeb20b24 (0xe925 bytes) 2024-09-13T09:13:43.435 INFO:teuthology.orchestra.run.smithi157.stdout:5583 write 0x8210a00 thru 0x821defa (0xd4fb bytes) 2024-09-13T09:13:43.435 INFO:teuthology.orchestra.run.smithi157.stdout:5586 read 0x1524a00 thru 0x152c2a8 (0x78a9 bytes) 2024-09-13T09:13:43.436 INFO:teuthology.orchestra.run.smithi157.stdout:5587 write 0x8e7fa00 thru 0x8e8c943 (0xcf44 bytes) 2024-09-13T09:13:43.436 INFO:teuthology.orchestra.run.smithi157.stdout:5594 read 0x9f74000 thru 0x9f833aa (0xf3ab bytes) 2024-09-13T09:13:43.436 INFO:teuthology.orchestra.run.smithi157.stdout:5597 write 0x6a11c00 thru 0x6a16403 (0x4804 bytes) 2024-09-13T09:13:43.436 INFO:teuthology.orchestra.run.smithi157.stdout:5599 trunc from 0xeb20c00 to 0xebf5200 2024-09-13T09:13:43.457 INFO:teuthology.orchestra.run.smithi157.stdout:5600 write 0x7f0000 thru 0x7fed20 (0xed21 bytes) 2024-09-13T09:13:43.457 INFO:teuthology.orchestra.run.smithi157.stdout:5601 punch from 0x83d2e00 to 0x83d8200, (0x5400 bytes) 2024-09-13T09:13:43.458 INFO:teuthology.orchestra.run.smithi157.stdout:5602 read 0x54fcc00 thru 0x5500cdf (0x40e0 bytes) 2024-09-13T09:13:43.461 INFO:teuthology.orchestra.run.smithi157.stdout:5604 trunc from 0xebf5200 to 0xc2ec800 2024-09-13T09:13:43.480 INFO:teuthology.orchestra.run.smithi157.stdout:5607 punch from 0x85f200 to 0x864e00, (0x5c00 bytes) 2024-09-13T09:13:43.480 INFO:teuthology.orchestra.run.smithi157.stdout:5608 read 0xb786400 thru 0xb78eda8 (0x89a9 bytes) 2024-09-13T09:13:43.481 INFO:teuthology.orchestra.run.smithi157.stdout:5609 trunc from 0xc2ec800 to 0xca74e00 2024-09-13T09:13:43.496 INFO:teuthology.orchestra.run.smithi157.stdout:5610 read 0x3bc0200 thru 0x3bcc383 (0xc184 bytes) 2024-09-13T09:13:43.496 INFO:teuthology.orchestra.run.smithi157.stdout:5612 punch from 0x31f7200 to 0x31f9a00, (0x2800 bytes) 2024-09-13T09:13:43.500 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:43.499+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:43.505 INFO:teuthology.orchestra.run.smithi157.stdout:5613 trunc from 0xca74e00 to 0x570c800 2024-09-13T09:13:43.554 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:43.555 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:43.563 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:43.586 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:13:43.604 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:43.602+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:43.702 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:43.701+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:43.705 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:43.705 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:43.705 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:43.706 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:43.706 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:43.714 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:43.722 INFO:teuthology.orchestra.run.smithi157.stdout:5614 trunc from 0x570c800 to 0x933800 2024-09-13T09:13:43.779 INFO:teuthology.orchestra.run.smithi157.stdout:5618 write 0xe5b7600 thru 0xe5c6923 (0xf324 bytes) 2024-09-13T09:13:43.791 INFO:teuthology.orchestra.run.smithi157.stdout:5619 trunc from 0xe5c6a00 to 0x46e8e00 2024-09-13T09:13:43.804 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:43.803+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:43.818 INFO:teuthology.orchestra.run.smithi157.stdout:5622 write 0xc6b000 thru 0xc7a937 (0xf938 bytes) 2024-09-13T09:13:43.843 INFO:teuthology.orchestra.run.smithi157.stdout:5623 write 0xaf11000 thru 0xaf13868 (0x2869 bytes) 2024-09-13T09:13:43.843 INFO:teuthology.orchestra.run.smithi157.stdout:5624 read 0xa7eac00 thru 0xa7f84c0 (0xd8c1 bytes) 2024-09-13T09:13:43.843 INFO:teuthology.orchestra.run.smithi157.stdout:5626 read 0x81ea00 thru 0x8229f5 (0x3ff6 bytes) 2024-09-13T09:13:43.845 INFO:teuthology.orchestra.run.smithi157.stdout:5630 write 0x678e800 thru 0x679b6be (0xcebf bytes) 2024-09-13T09:13:43.845 INFO:teuthology.orchestra.run.smithi157.stdout:5632 write 0x2632c00 thru 0x26361fb (0x35fc bytes) 2024-09-13T09:13:43.854 INFO:teuthology.orchestra.run.smithi157.stdout:5633 trunc from 0xaf13a00 to 0x312b600 2024-09-13T09:13:43.869 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9035 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2342912~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.121408082999999, 2024-09-13T09:13:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070293606999999994, 2024-09-13T09:13:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9035 2024-09-13T09:13:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745601+0000", 2024-09-13T09:13:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745602+0000", 2024-09-13T09:13:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6199999999999999e-06 2024-09-13T09:13:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745607+0000", 2024-09-13T09:13:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0770000000000001e-06 2024-09-13T09:13:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745609+0000", 2024-09-13T09:13:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3499999999999999e-06 2024-09-13T09:13:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815626+0000", 2024-09-13T09:13:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070017398999999994 2024-09-13T09:13:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815672+0000", 2024-09-13T09:13:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5275999999999998e-05 2024-09-13T09:13:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815895+0000", 2024-09-13T09:13:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022328300000000001 2024-09-13T09:13:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.121395538999998, 2024-09-13T09:13:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:13:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:13:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:13:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:13:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:13:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:13:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:13:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:13:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:13:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:13:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:13:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:13:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:13:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.121386860999998, 2024-09-13T09:13:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:13:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:13:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:13:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:13:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:13:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:13:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:13:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:13:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:13:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:13:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:13:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:13:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:13:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:13:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.121377875, 2024-09-13T09:13:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:13:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:13:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:13:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:13:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:13:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:13:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:13:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:13:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:13:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:13:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:13:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:13:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:13:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:13:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.121369903000002, 2024-09-13T09:13:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:13:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:13:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:13:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:13:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:13:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:13:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:13:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:13:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:13:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:13:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:13:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:13:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:13:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:13:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.119495069000003, 2024-09-13T09:13:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:13:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:13:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:13:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:13:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:13:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:13:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:13:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:13:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:13:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:13:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:13:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:13:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:13:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:13:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.119484407999998, 2024-09-13T09:13:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:13:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:13:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:13:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:13:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:13:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:13:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:13:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:13:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:13:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:13:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:13:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:13:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:13:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:13:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.119462431999999, 2024-09-13T09:13:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:13:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:13:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:13:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:13:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:13:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:13:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:13:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:13:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:13:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:13:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:13:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:13:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:13:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:13:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.119448929999997, 2024-09-13T09:13:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:13:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:13:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:13:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:13:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:13:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:13:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:13:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:13:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:13:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:13:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:13:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:13:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:13:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:13:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9056 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 7159808~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.117448101000001, 2024-09-13T09:13:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070283126000000001, 2024-09-13T09:13:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9056 2024-09-13T09:13:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749561+0000", 2024-09-13T09:13:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749562+0000", 2024-09-13T09:13:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1340000000000001e-06 2024-09-13T09:13:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749565+0000", 2024-09-13T09:13:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8760000000000001e-06 2024-09-13T09:13:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749567+0000", 2024-09-13T09:13:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2340000000000001e-06 2024-09-13T09:13:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819750+0000", 2024-09-13T09:13:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070182537000000003 2024-09-13T09:13:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819792+0000", 2024-09-13T09:13:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2141999999999997e-05 2024-09-13T09:13:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819844+0000", 2024-09-13T09:13:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2428e-05 2024-09-13T09:13:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.117439832000002, 2024-09-13T09:13:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:13:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:13:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:13:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:13:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:13:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:13:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:13:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:13:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:13:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:13:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:13:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:13:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:13:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:13:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.117415510000001, 2024-09-13T09:13:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:13:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:13:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:13:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:13:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:13:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:13:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:13:43.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:13:43.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:13:43.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:13:43.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:13:43.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:13:43.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:13:43.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:13:43.925 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.925 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.925 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.925 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.925 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:43.925 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.117404536000002, 2024-09-13T09:13:43.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:13:43.926 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.926 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.926 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.926 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.926 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.926 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:13:43.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.926 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:43.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:43.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:13:43.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:13:43.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:13:43.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:13:43.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:13:43.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:13:43.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:13:43.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:13:43.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:13:43.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:13:43.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:13:43.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:13:43.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:13:43.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.930 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.930 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9063 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1589248~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.930 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:43.930 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.115500894, 2024-09-13T09:13:43.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070391924999999994, 2024-09-13T09:13:43.930 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.930 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.930 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.930 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.931 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9063 2024-09-13T09:13:43.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.931 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:43.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:43.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751508+0000", 2024-09-13T09:13:43.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:43.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751509+0000", 2024-09-13T09:13:43.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.142e-06 2024-09-13T09:13:43.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751513+0000", 2024-09-13T09:13:43.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6669999999999998e-06 2024-09-13T09:13:43.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751520+0000", 2024-09-13T09:13:43.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5910000000000001e-06 2024-09-13T09:13:43.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821593+0000", 2024-09-13T09:13:43.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070073519000000001 2024-09-13T09:13:43.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821648+0000", 2024-09-13T09:13:43.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:13:43.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821900+0000", 2024-09-13T09:13:43.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025263999999999999 2024-09-13T09:13:43.934 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.934 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.934 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.935 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.935 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:43.935 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.115486910999998, 2024-09-13T09:13:43.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:13:43.935 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.935 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.935 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:13:43.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.936 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:43.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:43.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:43.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:43.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:13:43.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:13:43.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:13:43.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:43.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:13:43.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:13:43.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:13:43.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:13:43.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:13:43.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:13:43.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:13:43.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:13:43.939 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.939 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.939 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.939 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.939 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:43.939 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.115477513999998, 2024-09-13T09:13:43.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:13:43.940 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.940 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.940 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:13:43.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.940 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:43.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:43.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:43.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:43.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:13:43.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:13:43.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:13:43.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:13:43.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:13:43.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:13:43.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:13:43.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:13:43.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:13:43.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:13:43.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:13:43.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:13:43.943 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.943 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.944 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9071 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:43.944 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:43.944 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.113494496999998, 2024-09-13T09:13:43.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070386377999999999, 2024-09-13T09:13:43.944 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.944 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:43.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:43.944 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9071 2024-09-13T09:13:43.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.945 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:43.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:43.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753515+0000", 2024-09-13T09:13:43.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:43.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753516+0000", 2024-09-13T09:13:43.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1519999999999999e-06 2024-09-13T09:13:43.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753519+0000", 2024-09-13T09:13:43.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6559999999999998e-06 2024-09-13T09:13:43.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.753522+0000", 2024-09-13T09:13:43.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:13:43.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823798+0000", 2024-09-13T09:13:43.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070275907999999998 2024-09-13T09:13:43.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823845+0000", 2024-09-13T09:13:43.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7593000000000003e-05 2024-09-13T09:13:43.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.823901+0000", 2024-09-13T09:13:43.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5934999999999999e-05 2024-09-13T09:13:43.948 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.948 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.948 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.948 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3181 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T09:13:43.949 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:43.949 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.845113338000001, 2024-09-13T09:13:43.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.480957603, 2024-09-13T09:13:43.949 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.949 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:43.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:13:43.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:13:43.949 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3181 2024-09-13T09:13:43.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.949 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:43.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:43.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021894+0000", 2024-09-13T09:13:43.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:13:43.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021903+0000", 2024-09-13T09:13:43.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4109999999999996e-06 2024-09-13T09:13:43.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021919+0000", 2024-09-13T09:13:43.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6225000000000001e-05 2024-09-13T09:13:43.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021926+0000", 2024-09-13T09:13:43.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4120000000000004e-06 2024-09-13T09:13:43.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021975+0000", 2024-09-13T09:13:43.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8526000000000003e-05 2024-09-13T09:13:43.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022049+0000", 2024-09-13T09:13:43.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4018999999999997e-05 2024-09-13T09:13:43.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:13:43.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022230+0000", 2024-09-13T09:13:43.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018085599999999999 2024-09-13T09:13:43.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:13:43.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022796+0000", 2024-09-13T09:13:43.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056638999999999995 2024-09-13T09:13:43.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499540+0000", 2024-09-13T09:13:43.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.47674342800000002 2024-09-13T09:13:43.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:43.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499547+0000", 2024-09-13T09:13:43.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:13:43.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502794+0000", 2024-09-13T09:13:43.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.003246818 2024-09-13T09:13:43.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502854+0000", 2024-09-13T09:13:43.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9925e-05 2024-09-13T09:13:43.955 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.955 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.955 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.955 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3182 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 139818970156544 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:13:43.955 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:43.955 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.371878783, 2024-09-13T09:13:43.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0089058390000001, 2024-09-13T09:13:43.955 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.955 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.956 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:13:43.956 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:13:43.956 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3182 2024-09-13T09:13:43.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.956 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:43.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:43.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495128+0000", 2024-09-13T09:13:43.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:13:43.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495139+0000", 2024-09-13T09:13:43.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0599e-05 2024-09-13T09:13:43.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495157+0000", 2024-09-13T09:13:43.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7830999999999999e-05 2024-09-13T09:13:43.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495167+0000", 2024-09-13T09:13:43.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0689999999999999e-05 2024-09-13T09:13:43.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495229+0000", 2024-09-13T09:13:43.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1898999999999998e-05 2024-09-13T09:13:43.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:13:43.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495243+0000", 2024-09-13T09:13:43.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3349e-05 2024-09-13T09:13:43.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499575+0000", 2024-09-13T09:13:43.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043324410000000002 2024-09-13T09:13:43.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:43.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499582+0000", 2024-09-13T09:13:43.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0849999999999999e-06 2024-09-13T09:13:43.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502872+0000", 2024-09-13T09:13:43.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0032895150000001 2024-09-13T09:13:43.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503016+0000", 2024-09-13T09:13:43.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014394200000000001 2024-09-13T09:13:43.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504036+0000", 2024-09-13T09:13:43.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010208680000000001 2024-09-13T09:13:43.961 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.961 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.961 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.961 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:19151 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 140383089882784 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:13:43.961 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:43.961 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.371867324, 2024-09-13T09:13:43.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.009071287, 2024-09-13T09:13:43.961 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:43.961 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:43.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:43.962 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:13:43.962 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:13:43.962 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19151 2024-09-13T09:13:43.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.962 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:43.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:43.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:43.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:43.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:43.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:43.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:43.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495140+0000", 2024-09-13T09:13:43.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:13:43.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:43.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495149+0000", 2024-09-13T09:13:43.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0759999999999992e-06 2024-09-13T09:13:43.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:43.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495165+0000", 2024-09-13T09:13:43.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5322e-05 2024-09-13T09:13:43.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:43.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495173+0000", 2024-09-13T09:13:43.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0979999999999994e-06 2024-09-13T09:13:43.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495318+0000", 2024-09-13T09:13:43.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014553 2024-09-13T09:13:43.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:13:43.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495325+0000", 2024-09-13T09:13:43.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:13:43.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499598+0000", 2024-09-13T09:13:43.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004273064 2024-09-13T09:13:43.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:43.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499602+0000", 2024-09-13T09:13:43.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8539999999999999e-06 2024-09-13T09:13:43.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:43.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503507+0000", 2024-09-13T09:13:43.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0039045520000001 2024-09-13T09:13:43.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:43.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503583+0000", 2024-09-13T09:13:43.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6283000000000001e-05 2024-09-13T09:13:43.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:43.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:43.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:43.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504213+0000", 2024-09-13T09:13:43.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00063044700000000002 2024-09-13T09:13:43.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.967 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:43.967 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:43.967 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:43.968 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:43.967+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:43.970 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:43.970 INFO:teuthology.orchestra.run.smithi157.stdout:5635 trunc from 0x312b600 to 0x76e4a00 2024-09-13T09:13:43.970 INFO:teuthology.orchestra.run.smithi157.stdout:5636 write 0xa530c00 thru 0xa538f56 (0x8357 bytes) 2024-09-13T09:13:43.970 INFO:teuthology.orchestra.run.smithi157.stdout:5638 read 0x8c1f400 thru 0x8c25617 (0x6218 bytes) 2024-09-13T09:13:43.971 INFO:teuthology.orchestra.run.smithi157.stdout:5639 read 0x51de600 thru 0x51e62ef (0x7cf0 bytes) 2024-09-13T09:13:43.971 INFO:teuthology.orchestra.run.smithi157.stdout:5641 write 0x1eda800 thru 0x1edcc2b (0x242c bytes) 2024-09-13T09:13:43.971 INFO:teuthology.orchestra.run.smithi157.stdout:5644 punch from 0x1929800 to 0x1936800, (0xd000 bytes) 2024-09-13T09:13:43.971 INFO:teuthology.orchestra.run.smithi157.stdout:5645 write 0xbb9d600 thru 0xbb9fba2 (0x25a3 bytes) 2024-09-13T09:13:43.971 INFO:teuthology.orchestra.run.smithi157.stdout:5646 read 0x4ab1200 thru 0x4ab92f1 (0x80f2 bytes) 2024-09-13T09:13:43.971 INFO:teuthology.orchestra.run.smithi157.stdout:5647 read 0x836e800 thru 0x837384a (0x504b bytes) 2024-09-13T09:13:43.971 INFO:teuthology.orchestra.run.smithi157.stdout:5649 write 0xcfae00 thru 0xcfdb1c (0x2d1d bytes) 2024-09-13T09:13:43.971 INFO:teuthology.orchestra.run.smithi157.stdout:5650 trunc from 0xbb9fc00 to 0x62b3200 2024-09-13T09:13:43.972 INFO:teuthology.orchestra.run.smithi157.stdout:5651 write 0xbebea00 thru 0xbec34c7 (0x4ac8 bytes) 2024-09-13T09:13:43.981 INFO:teuthology.orchestra.run.smithi157.stdout:5652 trunc from 0xbec3600 to 0xec75a00 2024-09-13T09:13:43.993 INFO:teuthology.orchestra.run.smithi157.stdout:5653 clone 35 order 21 su 1048576 sc 4 2024-09-13T09:13:44.067 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:13:44.070 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:44.068+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:44.080 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:44.080 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:44.080 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:44.080 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:44.090 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:44.174 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:44.173+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:44.206 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:44.206 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:44.238 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:44.239 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:44.239 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:13:44.255 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:44.256 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:44.270 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:44.271 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:44.271 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:44.269+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:44.283 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:44.285 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:44.285 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:44.301 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:44.301 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:44.304 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:13:44.305 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:13:44.372 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:44.371+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:44.391 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:44.391 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:44.400 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:44.473 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:44.472+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:44.507 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:44.507 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:44.515 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:44.575 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:44.574+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:44.587 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-09-13T09:13:44.624 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:44.624 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:44.633 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:44.655 INFO:teuthology.orchestra.run.smithi157.stdout:flattening image image_client.0-clone34 2024-09-13T09:13:44.676 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:44.675+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:44.777 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:44.775+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:44.779 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:44.779 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:44.788 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:44.878 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:44.877+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:44.897 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:44.897 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:44.905 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:44.990 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:44.988+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:45.050 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:45.051 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:45.053 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-13T09:13:45.060 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:45.102 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:45.099+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:45.227 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:45.178+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:45.227 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:45.227 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:45.237 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:45.321 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:45.319+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:45.382 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:45.380+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:45.397 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:45.397 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:45.406 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:45.493 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:45.491+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:45.552 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:45.552 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:45.561 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:45.584 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:45.583+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:45.685 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:45.683+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:45.717 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:45.717 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:45.725 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:45.810 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:45.808+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:45.868 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:45.869 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:45.875 INFO:teuthology.orchestra.run.smithi157.stdout:5654 write 0x5bdbc00 thru 0x5be910f (0xd510 bytes) 2024-09-13T09:13:45.877 INFO:teuthology.orchestra.run.smithi157.stdout:5655 write 0x49e4e00 thru 0x49f30cb (0xe2cc bytes) 2024-09-13T09:13:45.878 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:45.878 INFO:teuthology.orchestra.run.smithi157.stdout:5656 read 0xa05b000 thru 0xa06139b (0x639c bytes) 2024-09-13T09:13:45.880 INFO:teuthology.orchestra.run.smithi157.stdout:5658 read 0x7941600 thru 0x7951311 (0xfd12 bytes) 2024-09-13T09:13:45.880 INFO:teuthology.orchestra.run.smithi157.stdout:5659 read 0x48acc00 thru 0x48af654 (0x2a55 bytes) 2024-09-13T09:13:45.880 INFO:teuthology.orchestra.run.smithi157.stdout:5661 read 0x39f3000 thru 0x39fd02f (0xa030 bytes) 2024-09-13T09:13:45.881 INFO:teuthology.orchestra.run.smithi157.stdout:5663 read 0x41c7e00 thru 0x41ccd03 (0x4f04 bytes) 2024-09-13T09:13:45.882 INFO:teuthology.orchestra.run.smithi157.stdout:5666 write 0x61c3000 thru 0x61d0597 (0xd598 bytes) 2024-09-13T09:13:45.882 INFO:teuthology.orchestra.run.smithi157.stdout:5667 write 0x59a7200 thru 0x59b32fb (0xc0fc bytes) 2024-09-13T09:13:45.883 INFO:teuthology.orchestra.run.smithi157.stdout:5668 read 0x9b92800 thru 0x9b9dc60 (0xb461 bytes) 2024-09-13T09:13:45.883 INFO:teuthology.orchestra.run.smithi157.stdout:5669 write 0xd054e00 thru 0xd05ffad (0xb1ae bytes) 2024-09-13T09:13:45.884 INFO:teuthology.orchestra.run.smithi157.stdout:5670 read 0x708ba00 thru 0x70922e1 (0x68e2 bytes) 2024-09-13T09:13:45.893 INFO:teuthology.orchestra.run.smithi157.stdout:5672 trunc from 0xec75a00 to 0x76a1c00 2024-09-13T09:13:45.919 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:45.917+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:45.938 INFO:teuthology.orchestra.run.smithi157.stdout:5674 write 0x8116600 thru 0x8116f83 (0x984 bytes) 2024-09-13T09:13:45.939 INFO:teuthology.orchestra.run.smithi157.stdout:5675 read 0x789b200 thru 0x78a2a6a (0x786b bytes) 2024-09-13T09:13:45.939 INFO:teuthology.orchestra.run.smithi157.stdout:5676 write 0x66afc00 thru 0x66b2e10 (0x3211 bytes) 2024-09-13T09:13:45.939 INFO:teuthology.orchestra.run.smithi157.stdout:5678 read 0xadc400 thru 0xae4129 (0x7d2a bytes) 2024-09-13T09:13:45.946 INFO:teuthology.orchestra.run.smithi157.stdout:5681 trunc from 0x8117000 to 0xcf55800 2024-09-13T09:13:45.967 INFO:teuthology.orchestra.run.smithi157.stdout:5682 trunc from 0xcf55800 to 0x66b2a00 2024-09-13T09:13:45.983 INFO:teuthology.orchestra.run.smithi157.stdout:5683 read 0x5a5600 thru 0x5ab994 (0x6395 bytes) 2024-09-13T09:13:45.985 INFO:teuthology.orchestra.run.smithi157.stdout:5684 punch from 0x51ca00 to 0x51f000, (0x2600 bytes) 2024-09-13T09:13:45.985 INFO:teuthology.orchestra.run.smithi157.stdout:5686 read 0x25a0400 thru 0x25a73b1 (0x6fb2 bytes) 2024-09-13T09:13:45.987 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:45.986+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:45.989 INFO:teuthology.orchestra.run.smithi157.stdout:5687 trunc from 0x66b2a00 to 0x7dcf200 2024-09-13T09:13:46.002 INFO:teuthology.orchestra.run.smithi157.stdout:5690 read 0x5906400 thru 0x590981c (0x341d bytes) 2024-09-13T09:13:46.003 INFO:teuthology.orchestra.run.smithi157.stdout:5691 read 0x163e200 thru 0x164af90 (0xcd91 bytes) 2024-09-13T09:13:46.009 INFO:teuthology.orchestra.run.smithi157.stdout:5692 trunc from 0x7dcf200 to 0x45c1e00 2024-09-13T09:13:46.029 INFO:teuthology.orchestra.run.smithi157.stdout:5693 punch from 0x2a3ec00 to 0x2a40a00, (0x1e00 bytes) 2024-09-13T09:13:46.034 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:46.034 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:46.043 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:46.052 INFO:teuthology.orchestra.run.smithi157.stdout:5695 write 0xed1d200 thru 0xed232b6 (0x60b7 bytes) 2024-09-13T09:13:46.065 INFO:teuthology.orchestra.run.smithi157.stdout:5698 trunc from 0xed23400 to 0x4b74c00 2024-09-13T09:13:46.097 INFO:teuthology.orchestra.run.smithi157.stdout:5699 write 0xc2df200 thru 0xc2eea57 (0xf858 bytes) 2024-09-13T09:13:46.098 INFO:teuthology.orchestra.run.smithi157.stdout:5700 punch from 0x5206e00 to 0x5211400, (0xa600 bytes) 2024-09-13T09:13:46.098 INFO:teuthology.orchestra.run.smithi157.stdout:5701 read 0x2669800 thru 0x26726af (0x8eb0 bytes) 2024-09-13T09:13:46.113 INFO:teuthology.orchestra.run.smithi157.stdout:5703 trunc from 0xc2eec00 to 0x1d9000 2024-09-13T09:13:46.129 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:46.127+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:46.135 INFO:teuthology.orchestra.run.smithi157.stdout:5704 punch from 0x108200 to 0x115400, (0xd200 bytes) 2024-09-13T09:13:46.156 INFO:teuthology.orchestra.run.smithi157.stdout:5706 write 0x844a200 thru 0x8459ab0 (0xf8b1 bytes) 2024-09-13T09:13:46.156 INFO:teuthology.orchestra.run.smithi157.stdout:5708 read 0x2614400 thru 0x261a152 (0x5d53 bytes) 2024-09-13T09:13:46.163 INFO:teuthology.orchestra.run.smithi157.stdout:5713 trunc from 0x8459c00 to 0x29c6200 2024-09-13T09:13:46.178 INFO:teuthology.orchestra.run.smithi157.stdout:5714 punch from 0x24ae600 to 0x24b6a00, (0x8400 bytes) 2024-09-13T09:13:46.180 INFO:teuthology.orchestra.run.smithi157.stdout:5715 read 0xefaa00 thru 0xf0792f (0xcf30 bytes) 2024-09-13T09:13:46.180 INFO:teuthology.orchestra.run.smithi157.stdout:5716 read 0x1e24e00 thru 0x1e31b87 (0xcd88 bytes) 2024-09-13T09:13:46.180 INFO:teuthology.orchestra.run.smithi157.stdout:5718 punch from 0x2087600 to 0x2094c00, (0xd600 bytes) 2024-09-13T09:13:46.180 INFO:teuthology.orchestra.run.smithi157.stdout:5719 write 0x61600 thru 0x61ceb (0x6ec bytes) 2024-09-13T09:13:46.180 INFO:teuthology.orchestra.run.smithi157.stdout:5722 read 0x14c7e00 thru 0x14d32c9 (0xb4ca bytes) 2024-09-13T09:13:46.189 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:46.188+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:46.198 INFO:teuthology.orchestra.run.smithi157.stdout:5725 write 0x81fba00 thru 0x81fe18d (0x278e bytes) 2024-09-13T09:13:46.198 INFO:teuthology.orchestra.run.smithi157.stdout:5726 punch from 0x1dae200 to 0x1db3e00, (0x5c00 bytes) 2024-09-13T09:13:46.199 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:46.199 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:46.203 INFO:teuthology.orchestra.run.smithi157.stdout:5727 write 0x665b000 thru 0x6666c5f (0xbc60 bytes) 2024-09-13T09:13:46.203 INFO:teuthology.orchestra.run.smithi157.stdout:5730 punch from 0x4c96800 to 0x4c9a200, (0x3a00 bytes) 2024-09-13T09:13:46.203 INFO:teuthology.orchestra.run.smithi157.stdout:5732 read 0x6d42400 thru 0x6d4a415 (0x8016 bytes) 2024-09-13T09:13:46.203 INFO:teuthology.orchestra.run.smithi157.stdout:5733 read 0x2f66800 thru 0x2f7177b (0xaf7c bytes) 2024-09-13T09:13:46.203 INFO:teuthology.orchestra.run.smithi157.stdout:5736 punch from 0x6aa4a00 to 0x6aab000, (0x6600 bytes) 2024-09-13T09:13:46.209 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:46.217 INFO:teuthology.orchestra.run.smithi157.stdout:5737 write 0x9ba4800 thru 0x9ba89a0 (0x41a1 bytes) 2024-09-13T09:13:46.218 INFO:teuthology.orchestra.run.smithi157.stdout:5739 write 0x641c000 thru 0x642444a (0x844b bytes) 2024-09-13T09:13:46.218 INFO:teuthology.orchestra.run.smithi157.stdout:5740 write 0x7b92800 thru 0x7b96fc7 (0x47c8 bytes) 2024-09-13T09:13:46.218 INFO:teuthology.orchestra.run.smithi157.stdout:5743 write 0x52dde00 thru 0x52ed9cd (0xfbce bytes) 2024-09-13T09:13:46.218 INFO:teuthology.orchestra.run.smithi157.stdout:5745 punch from 0x7858000 to 0x785a400, (0x2400 bytes) 2024-09-13T09:13:46.222 INFO:teuthology.orchestra.run.smithi157.stdout:5749 read 0x196ee00 thru 0x197581a (0x6a1b bytes) 2024-09-13T09:13:46.223 INFO:teuthology.orchestra.run.smithi157.stdout:5750 write 0x5d44600 thru 0x5d48736 (0x4137 bytes) 2024-09-13T09:13:46.223 INFO:teuthology.orchestra.run.smithi157.stdout:5753 trunc from 0x9ba8a00 to 0x9f73600 2024-09-13T09:13:46.234 INFO:teuthology.orchestra.run.smithi157.stdout:5754 punch from 0x92a5800 to 0x92b2000, (0xc800 bytes) 2024-09-13T09:13:46.239 INFO:teuthology.orchestra.run.smithi157.stdout:5755 write 0x817e00 thru 0x822721 (0xa922 bytes) 2024-09-13T09:13:46.240 INFO:teuthology.orchestra.run.smithi157.stdout:5756 read 0x8d2a200 thru 0x8d33e26 (0x9c27 bytes) 2024-09-13T09:13:46.240 INFO:teuthology.orchestra.run.smithi157.stdout:5759 read 0x5fe3800 thru 0x5ff24ca (0xeccb bytes) 2024-09-13T09:13:46.249 INFO:teuthology.orchestra.run.smithi157.stdout:5760 trunc from 0x9f73600 to 0x2193c00 2024-09-13T09:13:46.267 INFO:teuthology.orchestra.run.smithi157.stdout:5762 read 0x10c4e00 thru 0x10c517a (0x37b bytes) 2024-09-13T09:13:46.279 INFO:teuthology.orchestra.run.smithi157.stdout:5765 trunc from 0x2193c00 to 0xb99ca00 2024-09-13T09:13:46.288 INFO:teuthology.orchestra.run.smithi157.stdout:5766 read 0x281000 thru 0x2867cf (0x57d0 bytes) 2024-09-13T09:13:46.290 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:46.288+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:46.298 INFO:teuthology.orchestra.run.smithi157.stdout:5767 trunc from 0xb99ca00 to 0x443a400 2024-09-13T09:13:46.305 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:13:46.307 INFO:teuthology.orchestra.run.smithi157.stdout:5768 punch from 0x3643200 to 0x364d200, (0xa000 bytes) 2024-09-13T09:13:46.320 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:46.320 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:46.325 INFO:teuthology.orchestra.run.smithi157.stdout:5769 write 0xb48d000 thru 0xb49932e (0xc32f bytes) 2024-09-13T09:13:46.326 INFO:teuthology.orchestra.run.smithi157.stdout:5770 read 0x9706a00 thru 0x9715b01 (0xf102 bytes) 2024-09-13T09:13:46.326 INFO:teuthology.orchestra.run.smithi157.stdout:5771 punch from 0x1535600 to 0x153d000, (0x7a00 bytes) 2024-09-13T09:13:46.326 INFO:teuthology.orchestra.run.smithi157.stdout:5773 punch from 0x7629a00 to 0x7634a00, (0xb000 bytes) 2024-09-13T09:13:46.326 INFO:teuthology.orchestra.run.smithi157.stdout:5777 read 0xaa39600 thru 0xaa45486 (0xbe87 bytes) 2024-09-13T09:13:46.326 INFO:teuthology.orchestra.run.smithi157.stdout:5779 write 0x3612a00 thru 0x36141d7 (0x17d8 bytes) 2024-09-13T09:13:46.327 INFO:teuthology.orchestra.run.smithi157.stdout:5782 write 0x6b67800 thru 0x6b6bfda (0x47db bytes) 2024-09-13T09:13:46.327 INFO:teuthology.orchestra.run.smithi157.stdout:5783 read 0xa222600 thru 0xa227bb0 (0x55b1 bytes) 2024-09-13T09:13:46.327 INFO:teuthology.orchestra.run.smithi157.stdout:5786 write 0x7009800 thru 0x7019443 (0xfc44 bytes) 2024-09-13T09:13:46.328 INFO:teuthology.orchestra.run.smithi157.stdout:5788 write 0xd33e00 thru 0xd4320c (0xf40d bytes) 2024-09-13T09:13:46.328 INFO:teuthology.orchestra.run.smithi157.stdout:5789 write 0x78e600 thru 0x791d95 (0x3796 bytes) 2024-09-13T09:13:46.328 INFO:teuthology.orchestra.run.smithi157.stdout:5795 trunc from 0xb499400 to 0xaa92a00 2024-09-13T09:13:46.331 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:46.345 INFO:teuthology.orchestra.run.smithi157.stdout:5797 punch from 0x3dda200 to 0x3de5000, (0xae00 bytes) 2024-09-13T09:13:46.357 INFO:teuthology.orchestra.run.smithi157.stdout:5799 write 0xc939400 thru 0xc948031 (0xec32 bytes) 2024-09-13T09:13:46.357 INFO:teuthology.orchestra.run.smithi157.stdout:5802 read 0x7f53a00 thru 0x7f606cf (0xccd0 bytes) 2024-09-13T09:13:46.358 INFO:teuthology.orchestra.run.smithi157.stdout:5804 read 0x9bb3800 thru 0x9bbb02b (0x782c bytes) 2024-09-13T09:13:46.358 INFO:teuthology.orchestra.run.smithi157.stdout:5805 punch from 0x1024000 to 0x1032200, (0xe200 bytes) 2024-09-13T09:13:46.358 INFO:teuthology.orchestra.run.smithi157.stdout:5807 punch from 0xc6cac00 to 0xc6d3e00, (0x9200 bytes) 2024-09-13T09:13:46.363 INFO:teuthology.orchestra.run.smithi157.stdout:5808 trunc from 0xc948200 to 0x8127e00 2024-09-13T09:13:46.381 INFO:teuthology.orchestra.run.smithi157.stdout:5809 punch from 0x117de00 to 0x1181200, (0x3400 bytes) 2024-09-13T09:13:46.381 INFO:teuthology.orchestra.run.smithi157.stdout:5811 read 0x44f800 thru 0x458b79 (0x937a bytes) 2024-09-13T09:13:46.381 INFO:teuthology.orchestra.run.smithi157.stdout:5812 read 0x5dd3800 thru 0x5dd93ec (0x5bed bytes) 2024-09-13T09:13:46.381 INFO:teuthology.orchestra.run.smithi157.stdout:5813 read 0xcc3000 thru 0xcc3907 (0x908 bytes) 2024-09-13T09:13:46.381 INFO:teuthology.orchestra.run.smithi157.stdout:5814 read 0x790c000 thru 0x7919515 (0xd516 bytes) 2024-09-13T09:13:46.385 INFO:teuthology.orchestra.run.smithi157.stdout:5816 trunc from 0x8127e00 to 0xb482c00 2024-09-13T09:13:46.400 INFO:teuthology.orchestra.run.smithi157.stdout:5817 write 0x6b3c00 thru 0x6bb1c1 (0x75c2 bytes) 2024-09-13T09:13:46.415 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:46.414+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:46.418 INFO:teuthology.orchestra.run.smithi157.stdout:5819 write 0xb729400 thru 0xb7363d3 (0xcfd4 bytes) 2024-09-13T09:13:46.430 INFO:teuthology.orchestra.run.smithi157.stdout:5820 trunc from 0xb736400 to 0x1dfec00 2024-09-13T09:13:46.433 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:46.433 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:46.449 INFO:teuthology.orchestra.run.smithi157.stdout:5823 punch from 0xc8a400 to 0xc8d800, (0x3400 bytes) 2024-09-13T09:13:46.452 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:46.453 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:46.453 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:13:46.468 INFO:teuthology.orchestra.run.smithi157.stdout:5824 write 0x4892e00 thru 0x48a1484 (0xe685 bytes) 2024-09-13T09:13:46.472 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:46.472 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:46.486 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:46.486 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:46.486 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:46.486 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:46.486 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:46.487 INFO:teuthology.orchestra.run.smithi157.stdout:5828 write 0xa3aec00 thru 0xa3aedc0 (0x1c1 bytes) 2024-09-13T09:13:46.488 INFO:teuthology.orchestra.run.smithi157.stdout:5829 write 0x5e05600 thru 0x5e06862 (0x1263 bytes) 2024-09-13T09:13:46.488 INFO:teuthology.orchestra.run.smithi157.stdout:5830 write 0x7dec600 thru 0x7df7148 (0xab49 bytes) 2024-09-13T09:13:46.490 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:46.490 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:46.491 INFO:teuthology.orchestra.run.smithi157.stdout:5832 trunc from 0xa3aee00 to 0xc317000 2024-09-13T09:13:46.492 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:46.490+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:46.495 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:46.509 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:46.510 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:46.510 INFO:teuthology.orchestra.run.smithi157.stdout:5835 read 0x67d0200 thru 0x67d1605 (0x1406 bytes) 2024-09-13T09:13:46.512 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:13:46.512 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:13:46.516 INFO:teuthology.orchestra.run.smithi157.stdout:5836 trunc from 0xc317000 to 0x769d800 2024-09-13T09:13:46.531 INFO:teuthology.orchestra.run.smithi157.stdout:5839 trunc from 0x769d800 to 0x4a87000 2024-09-13T09:13:46.552 INFO:teuthology.orchestra.run.smithi157.stdout:5841 read 0x990400 thru 0x99bf4f (0xbb50 bytes) 2024-09-13T09:13:46.552 INFO:teuthology.orchestra.run.smithi157.stdout:5842 clone 36 order 18 su 1024 sc 12 2024-09-13T09:13:46.593 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:46.592+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:46.601 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:13:46.622 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:46.622 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:46.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:46.622 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:46.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.622 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.622 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:46.622 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.090654768999997, 2024-09-13T09:13:46.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:13:46.623 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.623 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.623 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.623 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.623 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.623 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:13:46.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.623 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:46.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:46.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:13:46.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:13:46.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:13:46.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:13:46.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:13:46.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:13:46.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:13:46.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:13:46.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:13:46.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:13:46.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:13:46.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:13:46.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:13:46.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:13:46.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:13:46.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:13:46.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:13:46.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:13:46.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:13:46.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:13:46.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:13:46.628 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.628 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.628 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.628 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.629 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:46.629 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.090636734, 2024-09-13T09:13:46.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:13:46.629 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.629 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.629 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.629 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.629 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.629 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:13:46.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.629 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:46.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:46.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:46.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:13:46.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:13:46.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:13:46.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:13:46.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:13:46.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:13:46.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:13:46.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:13:46.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:13:46.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:13:46.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:13:46.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:13:46.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:13:46.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:13:46.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:13:46.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:13:46.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:13:46.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:13:46.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:13:46.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:13:46.634 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.634 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.634 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.635 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.635 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:46.635 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.090620064000007, 2024-09-13T09:13:46.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:13:46.635 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.635 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.635 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.635 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.635 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.635 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:13:46.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.636 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:46.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:46.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:13:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:13:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:13:46.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:13:46.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:13:46.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:13:46.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:13:46.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:13:46.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:13:46.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:13:46.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:13:46.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:13:46.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:13:46.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:13:46.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:13:46.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:13:46.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:13:46.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:13:46.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:13:46.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:13:46.641 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.641 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.641 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.642 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.642 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:46.642 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.090602431999997, 2024-09-13T09:13:46.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:13:46.642 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.642 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.642 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.642 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.642 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.642 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:13:46.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.642 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:46.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:46.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:46.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:13:46.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:13:46.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:13:46.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:13:46.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:13:46.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:13:46.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:13:46.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:13:46.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:13:46.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:13:46.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:13:46.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:13:46.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:13:46.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:13:46.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:13:46.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:13:46.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:13:46.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:13:46.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:13:46.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:13:46.647 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.647 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.647 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.648 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.648 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:46.648 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.090556126999999, 2024-09-13T09:13:46.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:13:46.648 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.648 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.648 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.648 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.648 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.648 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:13:46.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.648 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:46.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:46.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:13:46.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:13:46.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:13:46.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:13:46.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:13:46.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:13:46.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:13:46.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:13:46.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:13:46.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:13:46.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:13:46.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:13:46.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:13:46.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:13:46.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:13:46.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:13:46.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:13:46.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:13:46.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:13:46.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:13:46.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:13:46.653 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.654 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.654 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.654 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.654 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:46.654 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.090530826000006, 2024-09-13T09:13:46.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:13:46.654 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.654 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.654 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.654 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.654 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.654 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:13:46.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.655 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:46.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:46.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:13:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:13:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:13:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:13:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:13:46.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:13:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:13:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:13:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:13:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:13:46.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:13:46.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:13:46.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:13:46.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:13:46.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:13:46.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:13:46.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:13:46.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:13:46.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:13:46.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:13:46.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:13:46.659 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.659 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.660 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.660 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.660 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:46.660 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.090512597, 2024-09-13T09:13:46.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:13:46.660 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.660 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.660 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.660 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.660 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.660 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:13:46.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.661 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:46.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:46.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:46.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:13:46.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:13:46.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:13:46.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:13:46.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:13:46.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:13:46.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:13:46.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:13:46.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:13:46.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:13:46.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:13:46.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:13:46.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:13:46.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:13:46.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:13:46.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:13:46.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:13:46.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:13:46.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:13:46.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:13:46.665 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.665 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.666 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.666 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.666 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:46.666 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.090465277999996, 2024-09-13T09:13:46.666 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:13:46.666 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.666 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.666 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.666 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.666 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.666 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:13:46.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.667 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:46.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:46.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:46.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:13:46.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:13:46.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:13:46.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:13:46.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:13:46.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:13:46.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:13:46.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:13:46.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:13:46.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:13:46.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:13:46.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:13:46.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:13:46.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:13:46.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:13:46.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:13:46.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:13:46.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:13:46.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:13:46.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:13:46.671 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.672 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.672 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.673 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.673 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:46.673 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.089951040000003, 2024-09-13T09:13:46.673 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:13:46.673 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.673 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.673 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.673 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.673 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.674 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:13:46.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.674 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:46.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:46.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:13:46.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:13:46.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:13:46.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:13:46.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:13:46.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:13:46.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:13:46.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:13:46.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:13:46.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:13:46.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:13:46.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:13:46.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:13:46.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:13:46.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:13:46.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:13:46.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:13:46.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:13:46.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:13:46.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:13:46.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:13:46.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:13:46.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.678 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.679 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.679 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.679 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:46.679 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.089916509999995, 2024-09-13T09:13:46.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:13:46.680 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.680 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.680 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.680 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:13:46.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.681 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:46.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:46.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:46.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:13:46.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:13:46.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:13:46.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:13:46.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:13:46.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:13:46.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:13:46.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:13:46.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:13:46.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:13:46.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:13:46.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:13:46.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:13:46.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:13:46.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:13:46.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:13:46.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:13:46.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:13:46.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.686 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:13:46.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:13:46.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.686 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.686 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.686 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:46.686 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.089898602999995, 2024-09-13T09:13:46.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:13:46.686 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.686 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.686 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.687 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.687 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.687 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:13:46.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.687 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:46.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:46.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:46.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:13:46.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:13:46.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:13:46.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:13:46.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:13:46.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:13:46.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:13:46.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:13:46.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:13:46.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:13:46.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:13:46.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:13:46.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:13:46.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:13:46.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:13:46.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:13:46.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:13:46.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:13:46.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:13:46.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:13:46.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.692 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.693 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.693 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:46.693 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.080771755000001, 2024-09-13T09:13:46.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:13:46.693 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.693 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.693 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.693 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.693 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.693 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:13:46.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:13:46.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:13:46.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:13:46.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:13:46.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:13:46.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:13:46.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:13:46.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:13:46.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:13:46.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:13:46.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:13:46.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:13:46.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:13:46.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:13:46.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:13:46.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:13:46.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:13:46.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:13:46.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:13:46.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:13:46.698 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.698 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.698 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.699 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.699 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:46.699 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.080720174999996, 2024-09-13T09:13:46.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:13:46.699 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.699 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.699 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.699 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.699 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.699 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:13:46.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.699 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:46.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:46.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:13:46.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:13:46.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:13:46.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:13:46.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:13:46.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:13:46.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:13:46.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:13:46.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:13:46.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:13:46.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:13:46.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:13:46.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:13:46.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:13:46.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:13:46.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:13:46.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:13:46.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:13:46.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:13:46.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:13:46.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:13:46.705 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.705 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.705 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.705 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.705 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:46.705 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.080703899, 2024-09-13T09:13:46.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:13:46.706 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.706 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.706 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.706 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.706 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.706 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:13:46.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.706 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:46.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:46.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:13:46.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:13:46.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:13:46.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:13:46.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:13:46.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:13:46.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:13:46.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:13:46.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:13:46.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:13:46.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:13:46.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:13:46.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:13:46.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:13:46.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:13:46.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:13:46.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:13:46.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:13:46.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:13:46.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:13:46.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:13:46.711 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.711 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.711 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.711 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.711 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:46.712 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.080688124999995, 2024-09-13T09:13:46.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:13:46.712 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.712 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.712 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.712 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.712 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.712 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:13:46.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.712 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:46.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:46.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:13:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:13:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:13:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:13:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:13:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:13:46.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:13:46.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:13:46.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:13:46.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:13:46.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:13:46.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:13:46.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:13:46.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:13:46.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:13:46.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:13:46.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:13:46.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:13:46.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:13:46.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:13:46.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:13:46.718 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.718 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.718 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.718 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.718 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:46.718 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.080673250000004, 2024-09-13T09:13:46.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:13:46.718 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.718 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.719 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.719 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.719 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.719 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:13:46.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.719 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:46.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:46.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:46.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.721 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:13:46.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:13:46.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:13:46.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:13:46.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:13:46.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:13:46.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:13:46.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:13:46.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:13:46.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:13:46.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:13:46.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:13:46.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:13:46.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:13:46.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:13:46.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:13:46.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:13:46.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:13:46.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:13:46.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:13:46.729 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.729 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.729 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.730 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.730 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:46.730 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.080655788000001, 2024-09-13T09:13:46.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:13:46.730 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.730 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.731 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.731 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.731 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.731 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:13:46.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.731 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.732 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.732 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:46.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.733 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.733 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:46.733 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.733 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:13:46.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.734 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:13:46.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:13:46.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:13:46.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:13:46.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:13:46.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:13:46.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:13:46.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:13:46.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.737 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:13:46.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:13:46.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:13:46.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:13:46.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:13:46.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:13:46.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:13:46.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:13:46.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.739 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.739 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:13:46.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:13:46.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.739 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.739 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:13:46.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:13:46.739 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.739 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.739 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.740 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.740 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:46.740 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.079055155999995, 2024-09-13T09:13:46.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:13:46.740 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.740 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.740 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.740 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.740 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.740 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:13:46.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.740 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:46.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:46.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:13:46.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:13:46.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:13:46.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:13:46.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:13:46.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:13:46.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:13:46.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:13:46.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:13:46.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:13:46.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:13:46.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.744 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.744 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:13:46.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:13:46.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.744 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.744 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:13:46.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:13:46.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.744 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.744 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:13:46.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:13:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:13:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:13:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:13:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:13:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.746 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.746 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:46.746 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.079015104999996, 2024-09-13T09:13:46.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:13:46.746 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.746 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.746 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.746 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.746 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.746 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:13:46.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.746 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:46.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:46.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.748 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.748 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:13:46.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.748 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.749 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:13:46.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:13:46.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.749 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.749 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:13:46.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:13:46.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.749 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:13:46.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:13:46.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:13:46.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:13:46.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:13:46.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:13:46.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:13:46.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:13:46.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:13:46.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:13:46.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:13:46.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:13:46.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.752 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:13:46.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:13:46.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.752 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:13:46.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:13:46.752 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.752 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.752 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.752 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:46.753 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:46.753 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.078971342000003, 2024-09-13T09:13:46.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:13:46.753 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:46.753 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:46.753 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:46.753 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:46.754 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:46.754 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:13:46.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.754 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:46.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.754 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:46.754 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:46.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:46.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:46.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:46.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:46.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:46.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:46.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:46.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:13:46.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:13:46.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:46.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:13:46.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:13:46.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:46.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:13:46.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:13:46.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:13:46.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:13:46.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:13:46.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:13:46.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:13:46.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:13:46.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:46.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:13:46.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:13:46.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:46.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:13:46.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:13:46.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:46.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:13:46.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:13:46.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:46.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:46.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:46.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:13:46.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:13:46.759 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.759 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.759 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.759 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:46.759 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:46.759 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:46.760 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone35 from 0x4a87000 (overlap 0x1d9000) to 0xbd200 2024-09-13T09:13:46.761 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:46.762 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:46.759+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:46.861 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:46.860+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:46.912 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:46.912 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:46.913 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:46.913 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:46.922 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:46.942 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-13T09:13:46.962 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:46.960+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:47.063 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:47.062+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:47.073 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:47.073 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:47.073 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:47.084 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:47.164 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:47.163+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:47.251 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:47.251 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:47.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:47.251 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:47.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.252 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.252 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:47.252 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.74574294, 2024-09-13T09:13:47.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:13:47.252 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.252 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.252 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.252 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.252 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.252 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:13:47.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.252 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:47.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:47.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:13:47.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:13:47.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:13:47.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:13:47.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:13:47.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:13:47.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:13:47.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:13:47.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:13:47.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:13:47.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:13:47.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:13:47.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:13:47.255 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.255 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.256 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.256 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.256 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:47.256 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.745715255999997, 2024-09-13T09:13:47.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:13:47.256 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.256 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.256 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.256 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.256 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.256 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:13:47.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.257 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:47.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:47.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:47.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:13:47.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:13:47.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:13:47.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:47.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:13:47.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:13:47.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:13:47.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:13:47.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:13:47.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:13:47.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:13:47.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:13:47.260 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.260 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.260 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.260 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.260 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:47.260 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.745690914999997, 2024-09-13T09:13:47.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:13:47.260 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.261 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.261 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.261 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.261 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.261 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:13:47.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.261 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:47.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:47.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:13:47.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:13:47.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:13:47.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:13:47.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:13:47.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:13:47.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:13:47.264 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.264 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.264 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.264 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.264 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:47.264 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.745674594999997, 2024-09-13T09:13:47.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:13:47.265 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.265 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.265 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.265 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.265 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.265 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:13:47.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.265 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:47.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:13:47.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:13:47.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:13:47.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:13:47.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:13:47.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:13:47.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:13:47.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:13:47.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.268 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:13:47.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:13:47.268 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.268 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.268 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.268 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.268 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:47.268 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.745658180000007, 2024-09-13T09:13:47.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:13:47.269 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.269 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.269 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.269 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.269 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.269 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:13:47.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.269 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:47.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:13:47.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:13:47.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:13:47.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:13:47.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:13:47.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:13:47.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:13:47.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:13:47.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:13:47.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:13:47.272 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.272 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.272 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.272 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.745642071999995, 2024-09-13T09:13:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:13:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:13:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.274 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.274 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:47.274 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.274 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.274 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:47.274 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.274 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.274 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:13:47.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:13:47.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:13:47.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:13:47.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:13:47.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:13:47.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:13:47.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:13:47.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:13:47.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:13:47.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:13:47.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:13:47.276 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.276 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.276 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.277 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.277 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:47.277 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.745626510999998, 2024-09-13T09:13:47.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:13:47.277 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.277 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.277 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.277 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.277 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.277 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:13:47.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.278 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.278 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.278 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:47.278 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.278 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.278 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:47.278 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.278 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.278 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:13:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:13:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:13:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:13:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.279 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:13:47.280 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:13:47.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.280 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.280 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:13:47.280 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:13:47.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.280 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.280 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:13:47.280 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:13:47.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.280 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:13:47.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:13:47.281 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.281 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.281 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.281 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.281 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:47.281 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.745609673000004, 2024-09-13T09:13:47.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:13:47.282 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.282 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.282 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.282 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.282 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.282 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:13:47.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.282 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.282 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.282 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:47.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:47.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:13:47.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:13:47.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:13:47.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:13:47.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:13:47.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:13:47.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:13:47.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:13:47.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:13:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:13:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:13:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:13:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:13:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.285 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.286 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.286 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:47.288 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.745593681000003, 2024-09-13T09:13:47.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:13:47.288 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.288 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.288 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.288 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.288 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.288 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:13:47.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.288 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:47.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:47.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:13:47.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:13:47.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:13:47.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:13:47.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:13:47.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:13:47.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:13:47.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.291 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:13:47.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:13:47.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.291 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:13:47.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:13:47.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.291 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:13:47.292 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:13:47.292 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.292 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.292 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.292 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.292 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:47.292 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.745566182999994, 2024-09-13T09:13:47.292 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:13:47.292 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.292 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.293 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.293 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:13:47.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.293 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:47.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:47.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:47.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:13:47.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:13:47.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:13:47.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:13:47.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:13:47.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:13:47.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:13:47.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:13:47.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:13:47.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:13:47.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:13:47.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:13:47.296 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.296 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.296 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.297 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.297 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:47.297 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.745540324000004, 2024-09-13T09:13:47.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:13:47.297 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.297 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.297 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.297 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.297 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.297 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:13:47.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.297 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:13:47.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:13:47.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:13:47.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:13:47.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:13:47.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:13:47.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:13:47.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:13:47.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:13:47.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:13:47.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:13:47.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:13:47.301 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.301 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.301 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.301 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.301 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:47.301 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.745498834000003, 2024-09-13T09:13:47.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:13:47.301 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.301 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.301 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.302 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.302 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.302 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:13:47.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.302 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.302 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.302 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:47.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.302 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.302 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:47.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:13:47.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:13:47.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:13:47.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:13:47.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:13:47.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:13:47.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:13:47.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:13:47.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:13:47.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:13:47.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:13:47.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:13:47.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:13:47.305 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.305 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.305 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.306 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.306 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:47.306 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.745477596000001, 2024-09-13T09:13:47.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:13:47.306 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.306 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.306 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.306 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.306 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.306 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:13:47.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.306 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:47.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:47.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:47.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:13:47.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:13:47.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:13:47.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:13:47.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:13:47.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:13:47.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:13:47.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:13:47.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:13:47.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:13:47.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:13:47.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:13:47.310 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.310 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.310 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.310 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.310 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:47.310 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.745454796000004, 2024-09-13T09:13:47.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:13:47.310 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.310 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.311 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.311 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.311 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.311 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:13:47.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.311 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.311 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.311 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:47.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.311 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.311 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:47.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:13:47.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.312 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:13:47.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:13:47.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:13:47.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:47.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:13:47.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:13:47.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:13:47.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:13:47.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:13:47.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:13:47.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:13:47.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:13:47.314 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.314 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.314 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.315 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.315 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:47.315 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.745438340000007, 2024-09-13T09:13:47.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:13:47.315 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.315 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.315 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.315 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.315 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.315 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:13:47.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.315 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:47.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:47.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:47.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:13:47.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:13:47.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:13:47.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:13:47.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:13:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:13:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:13:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:13:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:13:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:13:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:13:47.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:13:47.319 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.319 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.319 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.319 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.319 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:47.319 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.745193202999999, 2024-09-13T09:13:47.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:13:47.319 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.319 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.319 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.320 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.320 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.320 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:13:47.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.320 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:47.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:47.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:13:47.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:13:47.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:13:47.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:13:47.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:13:47.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:13:47.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:13:47.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:47.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:13:47.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:13:47.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:13:47.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:13:47.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:13:47.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:13:47.323 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.323 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.323 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.323 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.324 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:47.324 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.743307643999998, 2024-09-13T09:13:47.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:13:47.324 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.324 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.324 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.324 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.324 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.324 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:13:47.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.324 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.324 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:13:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:13:47.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:13:47.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:13:47.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:13:47.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:13:47.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:13:47.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:13:47.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:13:47.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:13:47.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:13:47.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:13:47.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:13:47.328 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.328 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.328 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.328 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.328 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:47.328 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.743286277999999, 2024-09-13T09:13:47.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:13:47.328 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.328 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.328 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.328 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.329 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.329 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:13:47.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.329 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:47.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:47.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:47.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:13:47.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:13:47.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:13:47.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:13:47.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:13:47.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:13:47.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:13:47.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:13:47.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:13:47.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:13:47.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:13:47.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:13:47.332 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.332 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.333 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.333 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:47.333 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:47.333 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 64.743080910000003, 2024-09-13T09:13:47.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:13:47.333 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.333 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:47.333 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.333 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:47.333 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.333 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:13:47.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.334 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:47.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:47.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:13:47.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:13:47.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:13:47.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:13:47.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:13:47.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:13:47.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:13:47.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:13:47.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:13:47.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:13:47.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:13:47.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:13:47.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:13:47.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:13:47.337 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.337 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.337 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.337 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5272.0:1775 3.0 3:02a42fca:::rbd_data.11235782ee8d.0000000000000030:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2244608~40448] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:13:47.338 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:47.338 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.001539069000003, 2024-09-13T09:13:47.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.666843732, 2024-09-13T09:13:47.338 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:47.338 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:47.338 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:47.338 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5272", 2024-09-13T09:13:47.338 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4089902097", 2024-09-13T09:13:47.338 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1775 2024-09-13T09:13:47.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.338 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:47.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:47.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:47.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:47.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:47.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:47.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:47.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247780+0000", 2024-09-13T09:13:47.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:13:47.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:47.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247865+0000", 2024-09-13T09:13:47.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4359999999999999e-05 2024-09-13T09:13:47.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:47.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247876+0000", 2024-09-13T09:13:47.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1456e-05 2024-09-13T09:13:47.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247883+0000", 2024-09-13T09:13:47.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:13:47.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247926+0000", 2024-09-13T09:13:47.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3203e-05 2024-09-13T09:13:47.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247932+0000", 2024-09-13T09:13:47.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:13:47.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909639+0000", 2024-09-13T09:13:47.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6617069099999999 2024-09-13T09:13:47.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:13:47.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909828+0000", 2024-09-13T09:13:47.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000189075 2024-09-13T09:13:47.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:47.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911676+0000", 2024-09-13T09:13:47.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001847235 2024-09-13T09:13:47.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:47.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911765+0000", 2024-09-13T09:13:47.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9530000000000005e-05 2024-09-13T09:13:47.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:13:47.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911901+0000", 2024-09-13T09:13:47.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013603899999999999 2024-09-13T09:13:47.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:13:47.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.912889+0000", 2024-09-13T09:13:47.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098807400000000003 2024-09-13T09:13:47.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:47.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914438+0000", 2024-09-13T09:13:47.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015491070000000001 2024-09-13T09:13:47.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:13:47.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914505+0000", 2024-09-13T09:13:47.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:13:47.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:47.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:47.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:47.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914625+0000", 2024-09-13T09:13:47.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120108 2024-09-13T09:13:47.344 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.344 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.344 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.344 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:47.345 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:47.345 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:47.346 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:47.347 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:47.344+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:47.447 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:47.445+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:47.454 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:47.455 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:47.455 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:47.463 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:47.547 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:47.546+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:47.594 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5] 2024-09-13T09:13:47.594 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:13:47.594 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T09:13:47.594 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 1.000000 2024-09-13T09:13:47.594 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-13T09:13:47.599 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-09-13T09:13:47.604 INFO:teuthology.orchestra.run.smithi157.stdout:5844 read 0x2665200 thru 0x266b8b3 (0x66b4 bytes) 2024-09-13T09:13:47.606 INFO:teuthology.orchestra.run.smithi157.stdout:5846 write 0x3907800 thru 0x390b41a (0x3c1b bytes) 2024-09-13T09:13:47.607 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:47.607 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:47.608 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:47.608 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:47.608 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:47.608 INFO:teuthology.orchestra.run.smithi157.stdout:5848 read 0x402be00 thru 0x40396fc (0xd8fd bytes) 2024-09-13T09:13:47.617 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:47.643 INFO:teuthology.orchestra.run.smithi157.stdout:5852 write 0x9eb1a00 thru 0x9eb1a6e (0x6f bytes) 2024-09-13T09:13:47.644 INFO:teuthology.orchestra.run.smithi157.stdout:5853 write 0x4599400 thru 0x45a3246 (0x9e47 bytes) 2024-09-13T09:13:47.644 INFO:teuthology.orchestra.run.smithi157.stdout:5857 punch from 0x527c600 to 0x5283200, (0x6c00 bytes) 2024-09-13T09:13:47.651 INFO:teuthology.orchestra.run.smithi157.stdout:5861 read 0x16c4000 thru 0x16ce8f4 (0xa8f5 bytes) 2024-09-13T09:13:47.651 INFO:teuthology.orchestra.run.smithi157.stdout:5864 read 0x3128600 thru 0x312928f (0xc90 bytes) 2024-09-13T09:13:47.661 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:47.659+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:47.700 INFO:teuthology.orchestra.run.smithi157.stdout:5865 write 0xdf9fc00 thru 0xdfadac1 (0xdec2 bytes) 2024-09-13T09:13:47.700 INFO:teuthology.orchestra.run.smithi157.stdout:5866 read 0x9073200 thru 0x9082b3f (0xf940 bytes) 2024-09-13T09:13:47.715 INFO:teuthology.orchestra.run.smithi157.stdout:5871 trunc from 0xdfadc00 to 0xaefc00 2024-09-13T09:13:47.749 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:47.748+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:47.769 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:47.769 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:47.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:47.769 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:47.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.769 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.770 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:47.770 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.245545637000006, 2024-09-13T09:13:47.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:13:47.770 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.770 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.770 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.770 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.770 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.770 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:13:47.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.770 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:47.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:47.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:47.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:13:47.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:13:47.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:13:47.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:13:47.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:13:47.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:13:47.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:13:47.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:13:47.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:13:47.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:13:47.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:13:47.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:13:47.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:13:47.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:13:47.774 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.774 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.774 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.774 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.774 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:47.774 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.245496580999998, 2024-09-13T09:13:47.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:13:47.774 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.775 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.775 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:13:47.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.775 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:47.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:47.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:47.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:13:47.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:13:47.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:13:47.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:13:47.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:13:47.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:13:47.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:13:47.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:13:47.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:13:47.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:13:47.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:13:47.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:13:47.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:13:47.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:13:47.779 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.779 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.779 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.779 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.779 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:47.779 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.245456505999996, 2024-09-13T09:13:47.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:13:47.779 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.779 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.779 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.779 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.780 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.780 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:13:47.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.780 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:47.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:47.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:47.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:13:47.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:13:47.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:13:47.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:13:47.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:13:47.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:13:47.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:13:47.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:13:47.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:13:47.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:13:47.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:13:47.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:13:47.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:13:47.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:13:47.784 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.784 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.784 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.784 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.784 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:47.784 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.245426691000006, 2024-09-13T09:13:47.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:13:47.785 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.785 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.785 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.785 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.785 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.785 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:13:47.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.785 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:47.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:47.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:47.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:13:47.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:13:47.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:13:47.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:47.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:13:47.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:13:47.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:13:47.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:13:47.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:13:47.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:13:47.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:13:47.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:13:47.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:13:47.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:13:47.789 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.789 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.789 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.789 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.790 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:47.790 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.245397030000007, 2024-09-13T09:13:47.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:13:47.790 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.790 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.790 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:13:47.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.790 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:47.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:47.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:47.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:13:47.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:13:47.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:13:47.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:13:47.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:13:47.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:13:47.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:13:47.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:13:47.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:13:47.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:13:47.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:13:47.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:13:47.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:13:47.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:13:47.794 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.794 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.794 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.794 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.794 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:47.794 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.245367822999995, 2024-09-13T09:13:47.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:13:47.794 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.794 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.795 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.795 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.795 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.795 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:13:47.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.795 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:47.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:47.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:47.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:13:47.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:13:47.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:13:47.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:13:47.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:13:47.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:13:47.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:13:47.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:13:47.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:13:47.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:13:47.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:13:47.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:13:47.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:13:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:13:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.245311642000004, 2024-09-13T09:13:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:13:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.799 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.800 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.800 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.800 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.800 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:13:47.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.800 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:47.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:47.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:47.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:13:47.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:13:47.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:13:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:13:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:13:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:13:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:13:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:13:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:13:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:13:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:13:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:13:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:13:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:13:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.245258180999997, 2024-09-13T09:13:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:13:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.805 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.805 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:13:47.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.805 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:47.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:47.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:13:47.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:13:47.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:13:47.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:13:47.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:13:47.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:13:47.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:13:47.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:13:47.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:13:47.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:13:47.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:13:47.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:13:47.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:13:47.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:13:47.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:13:47.809 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.809 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.809 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.809 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.810 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:47.810 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.245201351000006, 2024-09-13T09:13:47.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:13:47.810 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.810 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.810 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.810 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.810 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.810 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:13:47.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.810 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:47.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:47.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:13:47.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:13:47.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:13:47.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:13:47.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:13:47.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:13:47.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:13:47.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:13:47.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:13:47.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:13:47.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:13:47.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:13:47.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:13:47.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:13:47.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:13:47.814 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.814 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.814 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.815 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.815 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:47.815 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.245150323999994, 2024-09-13T09:13:47.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:13:47.815 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.815 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.815 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.815 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.815 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.815 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:13:47.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.815 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:47.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:47.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:13:47.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:13:47.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:13:47.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:13:47.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:13:47.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:13:47.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:13:47.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:13:47.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:13:47.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:13:47.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:13:47.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:13:47.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:13:47.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:13:47.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:13:47.819 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.819 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.819 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.819 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.819 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:47.819 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.245109134000003, 2024-09-13T09:13:47.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:13:47.820 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.820 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.820 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:13:47.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.820 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:47.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:47.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:47.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:13:47.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:13:47.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:13:47.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:13:47.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:13:47.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:13:47.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:13:47.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:13:47.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:13:47.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:13:47.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:13:47.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:13:47.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:13:47.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:13:47.824 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.824 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.824 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.824 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.824 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:47.824 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.245067018, 2024-09-13T09:13:47.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:13:47.825 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.825 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.825 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:13:47.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.825 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:47.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:47.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:47.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:47.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:13:47.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:13:47.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:13:47.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:13:47.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:13:47.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:13:47.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:13:47.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:13:47.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:13:47.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:13:47.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:13:47.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:13:47.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:13:47.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:13:47.829 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.829 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.829 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.829 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.829 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:47.829 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.245023138999997, 2024-09-13T09:13:47.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:13:47.829 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.829 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.830 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:13:47.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.830 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:47.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:47.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:47.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:47.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:13:47.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:13:47.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:13:47.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:13:47.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:13:47.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:13:47.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:13:47.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:13:47.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:13:47.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:13:47.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:13:47.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:13:47.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:13:47.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:13:47.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.833 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.834 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.834 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.834 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:47.834 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.244977222000003, 2024-09-13T09:13:47.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:13:47.834 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.834 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.834 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:13:47.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.835 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:47.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:47.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:13:47.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:13:47.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:13:47.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:13:47.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:13:47.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:13:47.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:13:47.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:13:47.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:13:47.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:13:47.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:13:47.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:13:47.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:13:47.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:13:47.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:13:47.838 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.838 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.839 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.839 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:47.839 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.244927450999995, 2024-09-13T09:13:47.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:13:47.839 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.839 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.839 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.839 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.839 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.839 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:13:47.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.840 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:47.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:47.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:47.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:47.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:13:47.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:13:47.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:13:47.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:13:47.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:13:47.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:13:47.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:13:47.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:13:47.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:13:47.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:13:47.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:13:47.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:13:47.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:13:47.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:13:47.843 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.843 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.843 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.844 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.844 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:47.844 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.244882047000004, 2024-09-13T09:13:47.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:13:47.844 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.844 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.844 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:13:47.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.844 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:47.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:47.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:13:47.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:13:47.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:13:47.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:13:47.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:13:47.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:13:47.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:13:47.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:13:47.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:13:47.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:13:47.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:13:47.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:13:47.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:13:47.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:13:47.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.848 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.849 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.849 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:47.849 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.244831409, 2024-09-13T09:13:47.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:13:47.849 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.849 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.849 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:13:47.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.849 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:47.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:47.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:13:47.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:13:47.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:13:47.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:13:47.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:13:47.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:13:47.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:13:47.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:13:47.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:13:47.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:13:47.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:13:47.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:13:47.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:13:47.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:13:47.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:13:47.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.853 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.854 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.854 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:47.854 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.244775566000001, 2024-09-13T09:13:47.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:13:47.854 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.854 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.854 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:13:47.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.854 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:47.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:47.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:47.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:13:47.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:13:47.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:13:47.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:13:47.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:13:47.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:13:47.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:13:47.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:13:47.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:13:47.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:13:47.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:13:47.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:13:47.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:13:47.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:13:47.859 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.859 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.859 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.859 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.859 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:47.859 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.244718644000002, 2024-09-13T09:13:47.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:13:47.859 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.860 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.860 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:13:47.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.860 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:47.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:47.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:47.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:47.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:13:47.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:13:47.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:13:47.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:13:47.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:13:47.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:13:47.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:13:47.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:13:47.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:13:47.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:13:47.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:13:47.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:13:47.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:13:47.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:13:47.864 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.864 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.864 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.864 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:47.864 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:47.864 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 65.244665080999994, 2024-09-13T09:13:47.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:13:47.865 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:47.865 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:47.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:47.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:47.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:47.865 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:13:47.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.865 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:47.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:47.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:47.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:47.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:47.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:47.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:47.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:47.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:47.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:47.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:13:47.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:13:47.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:47.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:13:47.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:13:47.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:47.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:13:47.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:13:47.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:13:47.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:13:47.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:47.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:13:47.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:13:47.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:47.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:13:47.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:13:47.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:47.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:47.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:47.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:13:47.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:13:47.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.869 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:47.869 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:47.869 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:47.870 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:47.869+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:47.871 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:47.971 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:47.970+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:47.972 INFO:teuthology.orchestra.run.smithi157.stdout:5873 write 0x774cc00 thru 0x774df8f (0x1390 bytes) 2024-09-13T09:13:47.979 INFO:teuthology.orchestra.run.smithi157.stdout:5875 trunc from 0x774e000 to 0x3d56c00 2024-09-13T09:13:48.036 INFO:teuthology.orchestra.run.smithi157.stdout:5876 read 0x1b5a000 thru 0x1b64bd1 (0xabd2 bytes) 2024-09-13T09:13:48.036 INFO:teuthology.orchestra.run.smithi157.stdout:5877 punch from 0x2303e00 to 0x230aa00, (0x6c00 bytes) 2024-09-13T09:13:48.036 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:48.037 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:48.037 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:48.037 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:48.047 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:48.048 INFO:teuthology.orchestra.run.smithi157.stdout:5878 punch from 0x1db7a00 to 0x1dc4200, (0xc800 bytes) 2024-09-13T09:13:48.088 INFO:teuthology.orchestra.run.smithi157.stdout:5879 write 0x99f7000 thru 0x9a03a3b (0xca3c bytes) 2024-09-13T09:13:48.088 INFO:teuthology.orchestra.run.smithi157.stdout:5881 read 0x61c2000 thru 0x61c275d (0x75e bytes) 2024-09-13T09:13:48.089 INFO:teuthology.orchestra.run.smithi157.stdout:5882 punch from 0x2a54e00 to 0x2a63200, (0xe400 bytes) 2024-09-13T09:13:48.091 INFO:teuthology.orchestra.run.smithi157.stdout:5883 read 0x570ca00 thru 0x5711ad2 (0x50d3 bytes) 2024-09-13T09:13:48.091 INFO:teuthology.orchestra.run.smithi157.stdout:5884 write 0x197ac00 thru 0x197b2b6 (0x6b7 bytes) 2024-09-13T09:13:48.091 INFO:teuthology.orchestra.run.smithi157.stdout:5885 write 0x9164200 thru 0x9172435 (0xe236 bytes) 2024-09-13T09:13:48.091 INFO:teuthology.orchestra.run.smithi157.stdout:5887 read 0x8645400 thru 0x865099b (0xb59c bytes) 2024-09-13T09:13:48.091 INFO:teuthology.orchestra.run.smithi157.stdout:5889 punch from 0x63c8e00 to 0x63d4200, (0xb400 bytes) 2024-09-13T09:13:48.091 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:48.089+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:48.092 INFO:teuthology.orchestra.run.smithi157.stdout:5891 punch from 0x977600 to 0x97c800, (0x5200 bytes) 2024-09-13T09:13:48.092 INFO:teuthology.orchestra.run.smithi157.stdout:5892 read 0x35a2200 thru 0x35af49d (0xd29e bytes) 2024-09-13T09:13:48.092 INFO:teuthology.orchestra.run.smithi157.stdout:5893 punch from 0x28dc800 to 0x28e4200, (0x7a00 bytes) 2024-09-13T09:13:48.172 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:48.171+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:48.178 INFO:teuthology.orchestra.run.smithi157.stdout:5894 write 0xac79200 thru 0xac860bf (0xcec0 bytes) 2024-09-13T09:13:48.178 INFO:teuthology.orchestra.run.smithi157.stdout:5896 read 0xa8f9600 thru 0xa9091bf (0xfbc0 bytes) 2024-09-13T09:13:48.179 INFO:teuthology.orchestra.run.smithi157.stdout:5897 read 0x4018000 thru 0x40268cf (0xe8d0 bytes) 2024-09-13T09:13:48.179 INFO:teuthology.orchestra.run.smithi157.stdout:5898 punch from 0x323aa00 to 0x323da00, (0x3000 bytes) 2024-09-13T09:13:48.179 INFO:teuthology.orchestra.run.smithi157.stdout:5899 write 0x3591a00 thru 0x359bb50 (0xa151 bytes) 2024-09-13T09:13:48.179 INFO:teuthology.orchestra.run.smithi157.stdout:5900 punch from 0x2715400 to 0x2725200, (0xfe00 bytes) 2024-09-13T09:13:48.182 INFO:teuthology.orchestra.run.smithi157.stdout:5902 trunc from 0xac86200 to 0xcd33800 2024-09-13T09:13:48.204 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:48.205 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:48.217 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:48.229 INFO:teuthology.orchestra.run.smithi157.stdout:5905 read 0x2091e00 thru 0x2095cd7 (0x3ed8 bytes) 2024-09-13T09:13:48.229 INFO:teuthology.orchestra.run.smithi157.stdout:5907 clone 37 order 24 su 4194304 sc 8 2024-09-13T09:13:48.303 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:48.301+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:48.363 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:48.363 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2185 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835586604, 2024-09-13T09:13:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063348994000000006, 2024-09-13T09:13:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.364 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2185 2024-09-13T09:13:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525765+0000", 2024-09-13T09:13:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2390000000000001e-06 2024-09-13T09:13:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525772+0000", 2024-09-13T09:13:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:13:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525777+0000", 2024-09-13T09:13:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0799999999999996e-06 2024-09-13T09:13:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589020+0000", 2024-09-13T09:13:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063242501000000007 2024-09-13T09:13:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589063+0000", 2024-09-13T09:13:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2673999999999998e-05 2024-09-13T09:13:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589112+0000", 2024-09-13T09:13:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9286000000000002e-05 2024-09-13T09:13:48.368 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2187 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835567056999999, 2024-09-13T09:13:48.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063500102000000003, 2024-09-13T09:13:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2187 2024-09-13T09:13:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525784+0000", 2024-09-13T09:13:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1629999999999999e-06 2024-09-13T09:13:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525792+0000", 2024-09-13T09:13:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525796+0000", 2024-09-13T09:13:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7389999999999999e-06 2024-09-13T09:13:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589132+0000", 2024-09-13T09:13:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063336119999999996 2024-09-13T09:13:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589187+0000", 2024-09-13T09:13:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4126000000000003e-05 2024-09-13T09:13:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589283+0000", 2024-09-13T09:13:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6032999999999993e-05 2024-09-13T09:13:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.373 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2188 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835549010999998, 2024-09-13T09:13:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063724617999999997, 2024-09-13T09:13:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2188 2024-09-13T09:13:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.374 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525802+0000", 2024-09-13T09:13:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0439999999999998e-06 2024-09-13T09:13:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525809+0000", 2024-09-13T09:13:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.238e-06 2024-09-13T09:13:48.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525814+0000", 2024-09-13T09:13:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-13T09:13:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589327+0000", 2024-09-13T09:13:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063513257000000004 2024-09-13T09:13:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589418+0000", 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0796000000000003e-05 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589525+0000", 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010693500000000001 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2190 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835530925, 2024-09-13T09:13:48.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063863981, 2024-09-13T09:13:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2190 2024-09-13T09:13:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525818+0000", 2024-09-13T09:13:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525820+0000", 2024-09-13T09:13:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:13:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525827+0000", 2024-09-13T09:13:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0589999999999999e-06 2024-09-13T09:13:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525831+0000", 2024-09-13T09:13:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7870000000000001e-06 2024-09-13T09:13:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589562+0000", 2024-09-13T09:13:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063731038000000004 2024-09-13T09:13:48.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589612+0000", 2024-09-13T09:13:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0087999999999999e-05 2024-09-13T09:13:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589683+0000", 2024-09-13T09:13:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0121000000000002e-05 2024-09-13T09:13:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.383 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2192 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835513964999997, 2024-09-13T09:13:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064050299000000005, 2024-09-13T09:13:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2192 2024-09-13T09:13:48.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525835+0000", 2024-09-13T09:13:48.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525837+0000", 2024-09-13T09:13:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:13:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525845+0000", 2024-09-13T09:13:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3660000000000003e-06 2024-09-13T09:13:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525850+0000", 2024-09-13T09:13:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0030000000000002e-06 2024-09-13T09:13:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589720+0000", 2024-09-13T09:13:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063870176000000001 2024-09-13T09:13:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589801+0000", 2024-09-13T09:13:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0889000000000004e-05 2024-09-13T09:13:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589886+0000", 2024-09-13T09:13:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5183000000000001e-05 2024-09-13T09:13:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835495862000002, 2024-09-13T09:13:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:13:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.390 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:13:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:13:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:13:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:13:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:13:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:13:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:13:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:13:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:13:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:13:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:13:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:13:48.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:13:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:13:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:48.394 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835478164000001, 2024-09-13T09:13:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:13:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:13:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:13:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:13:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:13:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:13:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:13:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:13:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:13:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:13:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:13:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:13:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:13:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:13:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835457548999997, 2024-09-13T09:13:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:13:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.399 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:13:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:13:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:13:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:13:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:13:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:13:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:13:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:13:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:13:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:13:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:13:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:13:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:13:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.403 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835438592999999, 2024-09-13T09:13:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:13:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.404 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:13:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:13:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:13:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:13:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:13:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:13:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:13:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:13:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:13:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:13:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:13:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:13:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:13:48.408 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835421334999999, 2024-09-13T09:13:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:13:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:13:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:48.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:13:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:13:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:13:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:13:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:13:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:13:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:13:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:13:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:13:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:13:48.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:13:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:13:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835401320999999, 2024-09-13T09:13:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:13:48.415 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:13:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:48.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:13:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:13:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:13:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:13:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:13:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:13:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:13:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:13:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:13:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:13:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:13:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:13:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.420 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835375739, 2024-09-13T09:13:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:13:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:13:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:13:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:13:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:13:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:13:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:13:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:13:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:13:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:13:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:13:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:13:48.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:13:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:13:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:13:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835356455000003, 2024-09-13T09:13:48.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:13:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:13:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:13:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:13:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:13:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:13:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:13:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:13:48.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:13:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:13:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:13:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:13:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:13:48.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:13:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835339199000003, 2024-09-13T09:13:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:13:48.433 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:13:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:13:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:13:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:13:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:13:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:13:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:13:48.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:13:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:13:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:13:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:13:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:13:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:13:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835319718999997, 2024-09-13T09:13:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:13:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.438 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:13:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:48.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:13:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:13:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:13:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:13:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:13:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:13:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:13:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:13:48.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:13:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:13:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:13:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:13:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:13:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.83530279, 2024-09-13T09:13:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:13:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:13:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.443 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:13:48.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:13:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:13:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:13:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:13:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:13:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:13:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:13:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:13:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:13:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:13:48.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:13:48.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:13:48.447 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.447 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.447 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.447 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.447 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:48.447 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835283640999997, 2024-09-13T09:13:48.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:13:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:13:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:13:48.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:13:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:13:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:13:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:13:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:13:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:13:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:13:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:13:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:13:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:13:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:13:48.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.835265501999999, 2024-09-13T09:13:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:13:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:13:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:13:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:13:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:13:48.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:13:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:13:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:13:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:13:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:13:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:13:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:13:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:13:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:13:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.456 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.834855007000002, 2024-09-13T09:13:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:13:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:13:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:13:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:13:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:13:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:13:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:13:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:13:48.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:13:48.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:13:48.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:13:48.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:13:48.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:13:48.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:13:48.461 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.461 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.461 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.461 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:48.461 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:48.461 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.834832106, 2024-09-13T09:13:48.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:13:48.461 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.461 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:48.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:48.462 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:13:48.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.462 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:48.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:48.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:48.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:13:48.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:13:48.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:13:48.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:13:48.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:13:48.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:13:48.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:13:48.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:13:48.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:13:48.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:13:48.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:13:48.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:13:48.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.465 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.466 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.466 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:48.467 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:48.468 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:48.465+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:48.514 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:13:48.568 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:48.566+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:48.601 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-13T09:13:48.614 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:48.614 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:48.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.614 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:24106 3.1d 3:ba4a2174:::rbd_header.112360a70c4c:head [call rbd.snapshot_get in=23b out=52b,call rbd.parent_overlap_get in=29b out=9b,call rbd.get_flags in=20b out=8b,call rbd.get_protection_status in=32b out=1b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:13:48.614 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.613512+0000", 2024-09-13T09:13:48.615 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.000412366, 2024-09-13T09:13:48.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00043405100000000002, 2024-09-13T09:13:48.615 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:48.615 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:48.615 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:48.615 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:13:48.615 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:13:48.615 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 24106 2024-09-13T09:13:48.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.615 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:48.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:48.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.613512+0000", 2024-09-13T09:13:48.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:48.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.613512+0000", 2024-09-13T09:13:48.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:48.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:48.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.613511+0000", 2024-09-13T09:13:48.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:48.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:48.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.613514+0000", 2024-09-13T09:13:48.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8990000000000001e-06 2024-09-13T09:13:48.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:48.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.613519+0000", 2024-09-13T09:13:48.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:13:48.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:48.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.613522+0000", 2024-09-13T09:13:48.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1860000000000001e-06 2024-09-13T09:13:48.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:48.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.613681+0000", 2024-09-13T09:13:48.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00015925600000000001 2024-09-13T09:13:48.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:48.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.613729+0000", 2024-09-13T09:13:48.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7549999999999997e-05 2024-09-13T09:13:48.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:48.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:48.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:48.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.613951+0000", 2024-09-13T09:13:48.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000222112 2024-09-13T09:13:48.619 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.619 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:48.619 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.619 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:48.619 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-13T09:13:48.619 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 1 2024-09-13T09:13:48.619 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:48.624 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:48.641 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:48.641 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:48.658 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:48.658 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:48.658 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:13:48.671 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone36 from 0xcd33800 (overlap 0xaefc00) to 0x9d7c00 2024-09-13T09:13:48.674 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:48.675 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:48.699 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:48.699 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:48.708 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:48.706+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:48.721 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:48.722 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:48.725 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:13:48.726 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:13:48.769 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:48.767+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:48.806 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:48.806 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:48.817 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:48.901 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:48.899+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:48.962 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:48.963 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:48.971 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:48.969+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:48.971 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:49.072 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:49.070+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:49.079 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:49.079 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:49.088 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:49.130 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-13T09:13:49.173 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:49.171+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:49.195 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:49.195 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:49.195 INFO:teuthology.orchestra.run.smithi157.stdout:5909 trunc from 0xcd33800 to 0xbba5a00 2024-09-13T09:13:49.204 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:49.241 INFO:teuthology.orchestra.run.smithi157.stdout:5911 read 0x1118400 thru 0x1124273 (0xbe74 bytes) 2024-09-13T09:13:49.241 INFO:teuthology.orchestra.run.smithi157.stdout:5913 write 0xa437e00 thru 0xa43a829 (0x2a2a bytes) 2024-09-13T09:13:49.241 INFO:teuthology.orchestra.run.smithi157.stdout:5916 punch from 0xf02000 to 0xf02e00, (0xe00 bytes) 2024-09-13T09:13:49.244 INFO:teuthology.orchestra.run.smithi157.stdout:5918 read 0x20cb000 thru 0x20cb659 (0x65a bytes) 2024-09-13T09:13:49.255 INFO:teuthology.orchestra.run.smithi157.stdout:5919 trunc from 0xbba5a00 to 0x23e5a00 2024-09-13T09:13:49.273 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:49.272+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:49.314 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:49.314 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:49.323 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:49.374 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:49.372+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:49.467 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:49.468 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:49.475 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:49.473+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:49.477 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:49.576 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:49.574+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:49.614 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:13:49.615 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:49.615 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:49.627 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:49.711 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:49.710+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:49.778 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:49.777+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:49.829 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:49.829 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:49.838 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:49.882 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:49.881+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:49.946 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:49.947 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:49.956 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:49.998 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:49.996+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:50.018 INFO:teuthology.orchestra.run.smithi157.stdout:5920 read 0xbf5400 thru 0xbf941d (0x401e bytes) 2024-09-13T09:13:50.018 INFO:teuthology.orchestra.run.smithi157.stdout:5922 punch from 0xbd8800 to 0xbe1400, (0x8c00 bytes) 2024-09-13T09:13:50.019 INFO:teuthology.orchestra.run.smithi157.stdout:5923 read 0x1641400 thru 0x165134f (0xff50 bytes) 2024-09-13T09:13:50.019 INFO:teuthology.orchestra.run.smithi157.stdout:5924 read 0x3a5000 thru 0x3ad5d9 (0x85da bytes) 2024-09-13T09:13:50.030 INFO:teuthology.orchestra.run.smithi157.stdout:5925 write 0x3251200 thru 0x325166d (0x46e bytes) 2024-09-13T09:13:50.045 INFO:teuthology.orchestra.run.smithi157.stdout:5928 write 0x67cd600 thru 0x67d24a1 (0x4ea2 bytes) 2024-09-13T09:13:50.046 INFO:teuthology.orchestra.run.smithi157.stdout:5929 read 0x1487000 thru 0x148f66b (0x866c bytes) 2024-09-13T09:13:50.060 INFO:teuthology.orchestra.run.smithi157.stdout:5934 write 0xa2d0600 thru 0xa2d15e3 (0xfe4 bytes) 2024-09-13T09:13:50.061 INFO:teuthology.orchestra.run.smithi157.stdout:5936 punch from 0x47fa000 to 0x47ffe00, (0x5e00 bytes) 2024-09-13T09:13:50.061 INFO:teuthology.orchestra.run.smithi157.stdout:5937 punch from 0x8721c00 to 0x8723c00, (0x2000 bytes) 2024-09-13T09:13:50.061 INFO:teuthology.orchestra.run.smithi157.stdout:5939 read 0x3617a00 thru 0x362447b (0xca7c bytes) 2024-09-13T09:13:50.062 INFO:teuthology.orchestra.run.smithi157.stdout:5945 read 0x3fe0e00 thru 0x3fe4ec2 (0x40c3 bytes) 2024-09-13T09:13:50.062 INFO:teuthology.orchestra.run.smithi157.stdout:5947 read 0x5d8ea00 thru 0x5d9b602 (0xcc03 bytes) 2024-09-13T09:13:50.071 INFO:teuthology.orchestra.run.smithi157.stdout:5948 write 0xb302000 thru 0xb30526e (0x326f bytes) 2024-09-13T09:13:50.072 INFO:teuthology.orchestra.run.smithi157.stdout:5949 read 0x2cdd000 thru 0x2ce580b (0x880c bytes) 2024-09-13T09:13:50.072 INFO:teuthology.orchestra.run.smithi157.stdout:5950 write 0x44a4c00 thru 0x44af2a6 (0xa6a7 bytes) 2024-09-13T09:13:50.073 INFO:teuthology.orchestra.run.smithi157.stdout:5951 punch from 0x4f2fc00 to 0x4f32a00, (0x2e00 bytes) 2024-09-13T09:13:50.073 INFO:teuthology.orchestra.run.smithi157.stdout:5952 write 0x34d4000 thru 0x34d50f3 (0x10f4 bytes) 2024-09-13T09:13:50.074 INFO:teuthology.orchestra.run.smithi157.stdout:5954 punch from 0x8d84200 to 0x8d91400, (0xd200 bytes) 2024-09-13T09:13:50.074 INFO:teuthology.orchestra.run.smithi157.stdout:5955 read 0x81f1000 thru 0x81fb892 (0xa893 bytes) 2024-09-13T09:13:50.075 INFO:teuthology.orchestra.run.smithi157.stdout:5956 read 0xa867e00 thru 0xa8726fc (0xa8fd bytes) 2024-09-13T09:13:50.075 INFO:teuthology.orchestra.run.smithi157.stdout:5957 read 0x2af9c00 thru 0x2afe387 (0x4788 bytes) 2024-09-13T09:13:50.082 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:50.080+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:50.086 INFO:teuthology.orchestra.run.smithi157.stdout:5959 write 0xcc3da00 thru 0xcc432c1 (0x58c2 bytes) 2024-09-13T09:13:50.087 INFO:teuthology.orchestra.run.smithi157.stdout:5960 read 0x85e8000 thru 0x85f3ff1 (0xbff2 bytes) 2024-09-13T09:13:50.088 INFO:teuthology.orchestra.run.smithi157.stdout:5961 write 0x8a21200 thru 0x8a28625 (0x7426 bytes) 2024-09-13T09:13:50.101 INFO:teuthology.orchestra.run.smithi157.stdout:5962 write 0xec07000 thru 0xec0c66c (0x566d bytes) 2024-09-13T09:13:50.102 INFO:teuthology.orchestra.run.smithi157.stdout:5964 write 0x8a00 thru 0x14984 (0xbf85 bytes) 2024-09-13T09:13:50.102 INFO:teuthology.orchestra.run.smithi157.stdout:5966 read 0xb941400 thru 0xb949b63 (0x8764 bytes) 2024-09-13T09:13:50.103 INFO:teuthology.orchestra.run.smithi157.stdout:5967 read 0x803600 thru 0x80cc71 (0x9672 bytes) 2024-09-13T09:13:50.103 INFO:teuthology.orchestra.run.smithi157.stdout:5968 read 0x84be200 thru 0x84c68dd (0x86de bytes) 2024-09-13T09:13:50.103 INFO:teuthology.orchestra.run.smithi157.stdout:5971 read 0x3e61200 thru 0x3e6d924 (0xc725 bytes) 2024-09-13T09:13:50.115 INFO:teuthology.orchestra.run.smithi157.stdout:5972 write 0xec99e00 thru 0xec9cdef (0x2ff0 bytes) 2024-09-13T09:13:50.116 INFO:teuthology.orchestra.run.smithi157.stdout:5974 read 0x86f5000 thru 0x8701c83 (0xcc84 bytes) 2024-09-13T09:13:50.117 INFO:teuthology.orchestra.run.smithi157.stdout:5975 punch from 0x1dfb000 to 0x1e04c00, (0x9c00 bytes) 2024-09-13T09:13:50.117 INFO:teuthology.orchestra.run.smithi157.stdout:5976 read 0x310a00 thru 0x31ccfc (0xc2fd bytes) 2024-09-13T09:13:50.118 INFO:teuthology.orchestra.run.smithi157.stdout:5977 write 0xb828400 thru 0xb82bf71 (0x3b72 bytes) 2024-09-13T09:13:50.118 INFO:teuthology.orchestra.run.smithi157.stdout:5978 write 0xc47ee00 thru 0xc483029 (0x422a bytes) 2024-09-13T09:13:50.119 INFO:teuthology.orchestra.run.smithi157.stdout:5979 write 0x10000 thru 0x19e37 (0x9e38 bytes) 2024-09-13T09:13:50.120 INFO:teuthology.orchestra.run.smithi157.stdout:5980 write 0xa6b200 thru 0xa73085 (0x7e86 bytes) 2024-09-13T09:13:50.120 INFO:teuthology.orchestra.run.smithi157.stdout:5981 read 0xe6ada00 thru 0xe6b8175 (0xa776 bytes) 2024-09-13T09:13:50.123 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:50.123 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:50.132 INFO:teuthology.orchestra.run.smithi157.stdout:5984 trunc from 0xec9ce00 to 0x5425000 2024-09-13T09:13:50.133 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:50.169 INFO:teuthology.orchestra.run.smithi157.stdout:5985 write 0x994ba00 thru 0x9957aea (0xc0eb bytes) 2024-09-13T09:13:50.170 INFO:teuthology.orchestra.run.smithi157.stdout:5986 punch from 0x35da000 to 0x35e0600, (0x6600 bytes) 2024-09-13T09:13:50.170 INFO:teuthology.orchestra.run.smithi157.stdout:5988 write 0x9461600 thru 0x9469bdf (0x85e0 bytes) 2024-09-13T09:13:50.171 INFO:teuthology.orchestra.run.smithi157.stdout:5993 write 0x553a200 thru 0x55482a0 (0xe0a1 bytes) 2024-09-13T09:13:50.172 INFO:teuthology.orchestra.run.smithi157.stdout:5994 read 0x8179400 thru 0x81804ae (0x70af bytes) 2024-09-13T09:13:50.172 INFO:teuthology.orchestra.run.smithi157.stdout:5995 read 0x430f200 thru 0x431ce73 (0xdc74 bytes) 2024-09-13T09:13:50.173 INFO:teuthology.orchestra.run.smithi157.stdout:5998 clone 38 order 18 su 8192 sc 15 2024-09-13T09:13:50.219 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:50.217+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:50.277 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:50.277 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:50.283 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:50.281+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:50.285 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:50.384 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:50.382+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:50.428 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:50.428 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:50.437 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:50.522 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:50.520+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:50.581 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:50.581 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:50.585 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:50.584+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:50.589 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:50.614 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-09-13T09:13:50.686 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:50.684+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:50.694 INFO:teuthology.orchestra.run.smithi157.stdout:truncating image image_client.0-clone37 from 0x9957c00 (overlap 0x23e5a00) to 0x72de00 2024-09-13T09:13:50.727 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:13:50.739 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:50.739 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:50.749 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:50.791 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:50.789+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:50.851 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:50.851 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:50.867 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:13:50.884 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:50.885 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:50.888 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:50.886+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:50.896 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:50.896 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:50.901 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:50.901 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:50.905 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:50.917 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:50.917 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:50.920 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:13:50.920 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:13:50.990 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:50.989+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:51.056 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:51.056 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:51.056 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:51.056 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:51.056 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:51.066 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:51.086 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-13T09:13:51.089 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:51.088+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:51.191 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:51.189+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:51.260 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:51.260 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:51.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:51.260 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:51.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.260 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:51.261 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:51.261 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.463126955, 2024-09-13T09:13:51.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:13:51.261 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.261 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:51.261 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.261 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:51.261 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:51.261 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:13:51.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.262 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:51.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:51.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:51.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:13:51.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:51.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:13:51.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:13:51.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:13:51.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:13:51.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:13:51.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:13:51.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:13:51.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:13:51.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:13:51.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:13:51.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.265 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.265 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:51.266 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:51.266 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.463118277, 2024-09-13T09:13:51.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:13:51.266 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.266 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:51.266 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.266 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:51.266 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:51.266 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:13:51.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.266 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:51.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:51.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:51.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:51.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:13:51.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:13:51.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:13:51.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:13:51.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:13:51.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:13:51.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:13:51.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:13:51.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:13:51.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:13:51.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:13:51.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:13:51.271 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.271 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.271 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.271 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:51.271 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:51.271 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.463109291000002, 2024-09-13T09:13:51.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:13:51.272 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.272 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:51.272 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.272 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:51.272 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:51.272 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:13:51.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.272 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:51.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:51.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:51.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:13:51.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:13:51.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:13:51.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:13:51.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:13:51.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:13:51.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:13:51.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:13:51.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:13:51.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:13:51.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:13:51.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:13:51.276 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.276 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.276 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.276 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:51.276 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:51.276 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.463101319000003, 2024-09-13T09:13:51.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:13:51.276 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.276 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:51.277 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.277 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:51.277 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:51.277 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:13:51.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.277 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:51.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:51.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:51.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:51.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:13:51.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:13:51.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:13:51.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:13:51.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:13:51.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:13:51.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:13:51.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:13:51.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:13:51.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:13:51.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:13:51.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:13:51.281 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.281 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.281 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.281 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:51.281 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:51.281 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.461226484999997, 2024-09-13T09:13:51.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:13:51.282 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.282 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:51.282 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.282 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:51.282 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:51.282 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:13:51.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.282 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:51.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:51.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:51.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:13:51.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:13:51.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:13:51.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:13:51.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:13:51.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:13:51.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:13:51.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:13:51.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:13:51.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:13:51.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:13:51.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:13:51.286 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.286 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.286 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.286 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:51.286 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:51.286 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.461215824, 2024-09-13T09:13:51.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:13:51.287 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.287 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:51.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:51.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:51.287 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:13:51.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.287 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:51.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:51.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:51.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:51.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:13:51.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:13:51.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:13:51.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:13:51.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:13:51.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:13:51.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:13:51.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:13:51.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:13:51.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:13:51.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:13:51.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:13:51.291 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.291 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.291 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.291 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:51.291 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:51.291 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.461193848000001, 2024-09-13T09:13:51.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:13:51.291 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.291 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:51.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:51.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:51.292 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:13:51.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.292 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:51.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:51.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:51.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:13:51.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:13:51.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:13:51.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:13:51.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:13:51.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:13:51.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:13:51.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:13:51.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:13:51.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:13:51.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:13:51.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:13:51.296 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.296 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.296 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.297 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:51.297 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:51.297 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.461180345999999, 2024-09-13T09:13:51.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:13:51.297 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.297 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:51.297 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.297 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:51.297 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:51.297 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:13:51.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.298 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:51.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:51.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:51.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:13:51.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:13:51.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:13:51.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:13:51.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:13:51.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:13:51.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:13:51.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:13:51.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:13:51.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:13:51.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:13:51.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:13:51.302 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.302 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.302 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.302 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:51.302 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:51.302 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.459171247999997, 2024-09-13T09:13:51.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:13:51.302 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.302 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:51.303 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.303 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:51.303 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:51.303 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:13:51.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.303 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:51.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:51.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:51.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:13:51.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:13:51.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:13:51.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:13:51.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:13:51.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:13:51.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:13:51.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:13:51.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:13:51.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:13:51.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:13:51.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:13:51.306 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.306 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.306 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.306 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:51.307 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:51.307 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.459146926000003, 2024-09-13T09:13:51.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:13:51.307 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.307 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:51.307 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.307 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:51.307 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:51.307 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:13:51.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.307 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:51.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:51.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:51.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:13:51.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:13:51.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:13:51.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:13:51.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:13:51.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:13:51.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:13:51.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:13:51.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:13:51.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:13:51.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:13:51.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:13:51.311 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.311 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.311 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.311 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:51.311 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:51.311 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.459135951999997, 2024-09-13T09:13:51.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:13:51.311 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.311 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:51.311 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.311 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:51.311 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:51.312 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:13:51.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.312 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:51.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:51.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:13:51.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:13:51.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:13:51.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:13:51.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:13:51.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:13:51.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:13:51.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:13:51.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:13:51.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:13:51.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:13:51.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:13:51.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:13:51.315 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.315 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.315 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.316 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:51.316 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:51.316 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.457218327, 2024-09-13T09:13:51.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:13:51.316 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.316 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:51.316 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.316 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:51.316 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:51.316 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:13:51.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.317 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:51.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:51.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:51.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:13:51.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:13:51.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:13:51.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:51.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:13:51.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:13:51.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:13:51.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:13:51.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:13:51.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:13:51.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:13:51.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:13:51.320 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.320 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.320 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.320 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:51.320 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:51.320 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.45720893, 2024-09-13T09:13:51.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:13:51.320 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.320 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:51.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:51.321 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:51.321 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:13:51.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.321 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:51.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:51.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:51.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:51.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:13:51.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:13:51.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:13:51.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:13:51.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:13:51.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:13:51.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:13:51.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:13:51.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:13:51.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:13:51.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:13:51.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:13:51.324 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.324 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.324 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.324 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3181 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T09:13:51.324 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:51.324 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 21.186844753999999, 2024-09-13T09:13:51.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.480957603, 2024-09-13T09:13:51.325 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.325 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:51.325 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.325 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:13:51.325 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:13:51.325 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3181 2024-09-13T09:13:51.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.325 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:51.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:51.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021894+0000", 2024-09-13T09:13:51.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:13:51.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021903+0000", 2024-09-13T09:13:51.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4109999999999996e-06 2024-09-13T09:13:51.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021919+0000", 2024-09-13T09:13:51.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6225000000000001e-05 2024-09-13T09:13:51.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021926+0000", 2024-09-13T09:13:51.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4120000000000004e-06 2024-09-13T09:13:51.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021975+0000", 2024-09-13T09:13:51.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8526000000000003e-05 2024-09-13T09:13:51.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022049+0000", 2024-09-13T09:13:51.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4018999999999997e-05 2024-09-13T09:13:51.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:13:51.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022230+0000", 2024-09-13T09:13:51.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018085599999999999 2024-09-13T09:13:51.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:13:51.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022796+0000", 2024-09-13T09:13:51.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056638999999999995 2024-09-13T09:13:51.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499540+0000", 2024-09-13T09:13:51.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.47674342800000002 2024-09-13T09:13:51.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499547+0000", 2024-09-13T09:13:51.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:13:51.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502794+0000", 2024-09-13T09:13:51.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.003246818 2024-09-13T09:13:51.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502854+0000", 2024-09-13T09:13:51.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9925e-05 2024-09-13T09:13:51.331 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.331 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.331 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.331 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3182 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 139818970156544 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:13:51.331 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:51.331 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.713610199000001, 2024-09-13T09:13:51.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0089058390000001, 2024-09-13T09:13:51.332 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.332 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:51.332 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.332 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:13:51.332 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:13:51.332 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3182 2024-09-13T09:13:51.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.332 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:51.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:51.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495128+0000", 2024-09-13T09:13:51.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:13:51.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495139+0000", 2024-09-13T09:13:51.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0599e-05 2024-09-13T09:13:51.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495157+0000", 2024-09-13T09:13:51.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7830999999999999e-05 2024-09-13T09:13:51.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495167+0000", 2024-09-13T09:13:51.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0689999999999999e-05 2024-09-13T09:13:51.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495229+0000", 2024-09-13T09:13:51.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1898999999999998e-05 2024-09-13T09:13:51.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:13:51.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495243+0000", 2024-09-13T09:13:51.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3349e-05 2024-09-13T09:13:51.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499575+0000", 2024-09-13T09:13:51.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043324410000000002 2024-09-13T09:13:51.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499582+0000", 2024-09-13T09:13:51.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0849999999999999e-06 2024-09-13T09:13:51.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502872+0000", 2024-09-13T09:13:51.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0032895150000001 2024-09-13T09:13:51.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503016+0000", 2024-09-13T09:13:51.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014394200000000001 2024-09-13T09:13:51.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504036+0000", 2024-09-13T09:13:51.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010208680000000001 2024-09-13T09:13:51.337 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.337 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.337 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.338 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:19151 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 140383089882784 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:13:51.338 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:51.338 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 20.713598739999998, 2024-09-13T09:13:51.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.009071287, 2024-09-13T09:13:51.338 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.338 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:51.338 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.338 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:13:51.338 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:13:51.338 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19151 2024-09-13T09:13:51.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.338 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:51.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:51.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495140+0000", 2024-09-13T09:13:51.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:13:51.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495149+0000", 2024-09-13T09:13:51.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0759999999999992e-06 2024-09-13T09:13:51.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495165+0000", 2024-09-13T09:13:51.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5322e-05 2024-09-13T09:13:51.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495173+0000", 2024-09-13T09:13:51.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0979999999999994e-06 2024-09-13T09:13:51.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495318+0000", 2024-09-13T09:13:51.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014553 2024-09-13T09:13:51.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:13:51.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495325+0000", 2024-09-13T09:13:51.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:13:51.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499598+0000", 2024-09-13T09:13:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004273064 2024-09-13T09:13:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499602+0000", 2024-09-13T09:13:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8539999999999999e-06 2024-09-13T09:13:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503507+0000", 2024-09-13T09:13:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0039045520000001 2024-09-13T09:13:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503583+0000", 2024-09-13T09:13:51.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6283000000000001e-05 2024-09-13T09:13:51.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504213+0000", 2024-09-13T09:13:51.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00063044700000000002 2024-09-13T09:13:51.344 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.344 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.344 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.345 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3908 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:13:51.345 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:13:51.345 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.1160618489999998, 2024-09-13T09:13:51.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070530317999999995, 2024-09-13T09:13:51.345 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.345 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:51.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:13:51.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:13:51.345 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3908 2024-09-13T09:13:51.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.346 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:13:51.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:13:51.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092678+0000", 2024-09-13T09:13:51.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092681+0000", 2024-09-13T09:13:51.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.903e-06 2024-09-13T09:13:51.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092687+0000", 2024-09-13T09:13:51.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0859999999999998e-06 2024-09-13T09:13:51.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092692+0000", 2024-09-13T09:13:51.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-13T09:13:51.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.159711+0000", 2024-09-13T09:13:51.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067018922999999994 2024-09-13T09:13:51.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.159796+0000", 2024-09-13T09:13:51.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5247000000000006e-05 2024-09-13T09:13:51.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:51.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160111+0000", 2024-09-13T09:13:51.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00031449499999999998 2024-09-13T09:13:51.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:13:51.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162219+0000", 2024-09-13T09:13:51.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021085570000000001 2024-09-13T09:13:51.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:51.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163127+0000", 2024-09-13T09:13:51.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00090723400000000004 2024-09-13T09:13:51.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:13:51.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163172+0000", 2024-09-13T09:13:51.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5383000000000002e-05 2024-09-13T09:13:51.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163209+0000", 2024-09-13T09:13:51.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7132000000000003e-05 2024-09-13T09:13:51.355 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.355 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.355 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.355 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3909 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:13:51.355 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:13:51.355 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.1160443579999999, 2024-09-13T09:13:51.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070616834000000003, 2024-09-13T09:13:51.355 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.355 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:51.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:13:51.356 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:13:51.356 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3909 2024-09-13T09:13:51.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.356 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:13:51.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:13:51.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:13:51.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092699+0000", 2024-09-13T09:13:51.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:13:51.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092704+0000", 2024-09-13T09:13:51.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5940000000000001e-06 2024-09-13T09:13:51.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092709+0000", 2024-09-13T09:13:51.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5109999999999997e-06 2024-09-13T09:13:51.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160604+0000", 2024-09-13T09:13:51.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067895633999999996 2024-09-13T09:13:51.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160658+0000", 2024-09-13T09:13:51.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3292999999999999e-05 2024-09-13T09:13:51.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:51.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160935+0000", 2024-09-13T09:13:51.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000276879 2024-09-13T09:13:51.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:13:51.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163223+0000", 2024-09-13T09:13:51.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022886320000000001 2024-09-13T09:13:51.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:51.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163258+0000", 2024-09-13T09:13:51.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5101000000000002e-05 2024-09-13T09:13:51.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:13:51.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163292+0000", 2024-09-13T09:13:51.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3237e-05 2024-09-13T09:13:51.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163313+0000", 2024-09-13T09:13:51.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1642e-05 2024-09-13T09:13:51.361 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.361 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.361 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.361 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3910 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:13:51.361 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:13:51.361 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.1160272940000002, 2024-09-13T09:13:51.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733837999999993, 2024-09-13T09:13:51.361 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.361 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:51.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:13:51.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:13:51.362 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3910 2024-09-13T09:13:51.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.362 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:13:51.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:13:51.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:13:51.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092716+0000", 2024-09-13T09:13:51.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.757e-06 2024-09-13T09:13:51.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092722+0000", 2024-09-13T09:13:51.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7869999999999998e-06 2024-09-13T09:13:51.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092726+0000", 2024-09-13T09:13:51.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.622e-06 2024-09-13T09:13:51.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161487+0000", 2024-09-13T09:13:51.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068761166999999998 2024-09-13T09:13:51.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161569+0000", 2024-09-13T09:13:51.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1185000000000003e-05 2024-09-13T09:13:51.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:51.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161868+0000", 2024-09-13T09:13:51.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029979799999999999 2024-09-13T09:13:51.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:51.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163366+0000", 2024-09-13T09:13:51.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001497888 2024-09-13T09:13:51.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:13:51.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163373+0000", 2024-09-13T09:13:51.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1640000000000004e-06 2024-09-13T09:13:51.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:13:51.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163412+0000", 2024-09-13T09:13:51.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8423999999999997e-05 2024-09-13T09:13:51.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163447+0000", 2024-09-13T09:13:51.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5441000000000002e-05 2024-09-13T09:13:51.367 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.367 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.367 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.367 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3911 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:13:51.367 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:13:51.367 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.1160073779999999, 2024-09-13T09:13:51.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.072284372999999999, 2024-09-13T09:13:51.367 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:51.368 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:51.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:51.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:13:51.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:13:51.368 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3911 2024-09-13T09:13:51.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.368 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:51.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:51.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:13:51.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:51.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:13:51.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:51.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:51.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:13:51.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:51.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092736+0000", 2024-09-13T09:13:51.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9510000000000002e-06 2024-09-13T09:13:51.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092743+0000", 2024-09-13T09:13:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9e-06 2024-09-13T09:13:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092748+0000", 2024-09-13T09:13:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999998e-06 2024-09-13T09:13:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162257+0000", 2024-09-13T09:13:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.069508890000000004 2024-09-13T09:13:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:51.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162342+0000", 2024-09-13T09:13:51.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5886999999999999e-05 2024-09-13T09:13:51.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:51.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162682+0000", 2024-09-13T09:13:51.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00033971700000000001 2024-09-13T09:13:51.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:13:51.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164897+0000", 2024-09-13T09:13:51.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022152370000000001 2024-09-13T09:13:51.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:51.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164945+0000", 2024-09-13T09:13:51.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7243999999999997e-05 2024-09-13T09:13:51.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:13:51.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164985+0000", 2024-09-13T09:13:51.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0599000000000002e-05 2024-09-13T09:13:51.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:51.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:51.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:51.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.165018+0000", 2024-09-13T09:13:51.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2549000000000003e-05 2024-09-13T09:13:51.373 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.373 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.373 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.373 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:51.373 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:51.373 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:51.373 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:51.375 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:51.373+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:51.475 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:51.474+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:51.483 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:51.483 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:51.483 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:51.484 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:51.493 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:51.576 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:51.574+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:51.653 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:51.653 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:51.653 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:51.653 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:51.653 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:51.664 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:51.677 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:51.676+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:51.778 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:51.777+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:51.793 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:51.793 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:51.793 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:51.793 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:51.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.793 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.793 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:51.793 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.261665562000005, 2024-09-13T09:13:51.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:13:51.794 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.794 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.794 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.794 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.794 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.794 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:13:51.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.794 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.794 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.794 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:51.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:51.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:13:51.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:13:51.796 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:13:51.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.796 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.796 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:13:51.796 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:13:51.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.796 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.796 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:13:51.796 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:13:51.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.796 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:13:51.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:13:51.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.797 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:13:51.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:13:51.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.797 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:13:51.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:13:51.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.798 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.798 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:13:51.798 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:13:51.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.798 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.798 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:13:51.798 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:13:51.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.798 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.799 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:13:51.799 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:13:51.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.799 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.799 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:13:51.799 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:13:51.799 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.799 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.799 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.799 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.800 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:51.800 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.261647526999994, 2024-09-13T09:13:51.800 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:13:51.800 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.800 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.800 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.800 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.800 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.800 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:13:51.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.800 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.800 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:51.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.801 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:51.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.801 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:51.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.802 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.802 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:13:51.802 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:13:51.802 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.802 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.802 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:13:51.802 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:13:51.802 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.802 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.802 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:13:51.802 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:13:51.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:13:51.803 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:13:51.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:13:51.803 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:13:51.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:13:51.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:13:51.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.804 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:13:51.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:13:51.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.804 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:13:51.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:13:51.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.805 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.805 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:13:51.805 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:13:51.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.808 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.808 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.808 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:13:51.808 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:13:51.808 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.808 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.808 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.809 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.809 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.809 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:51.809 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.261630857, 2024-09-13T09:13:51.809 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:13:51.809 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.810 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.810 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.810 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.810 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.810 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:13:51.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.810 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.810 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.810 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.810 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:51.810 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.810 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.810 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.810 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:51.811 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.811 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.811 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.811 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.811 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:13:51.811 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.811 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.811 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.811 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.811 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:13:51.812 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:51.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.812 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.812 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:13:51.812 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:13:51.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.813 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.813 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:13:51.813 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:13:51.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.813 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.813 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:13:51.813 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:13:51.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.813 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.813 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:13:51.813 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:13:51.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.814 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.814 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:13:51.814 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:13:51.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.814 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.814 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:13:51.814 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:13:51.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.814 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.814 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:13:51.815 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:13:51.815 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.815 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.815 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.815 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:13:51.815 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:13:51.815 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.815 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.815 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.815 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:13:51.815 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:13:51.815 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.815 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.815 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.816 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.816 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.816 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:51.816 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.261613225000005, 2024-09-13T09:13:51.816 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:13:51.816 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.816 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.816 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.816 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.816 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.816 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:13:51.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.816 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.817 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.817 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:51.817 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.817 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.817 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:51.817 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.817 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.817 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:51.818 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.818 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.818 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:13:51.818 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:13:51.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.818 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.818 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:13:51.818 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:13:51.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.818 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.819 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:13:51.819 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:13:51.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.819 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.819 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:13:51.819 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:13:51.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.819 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.819 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:13:51.819 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:13:51.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.820 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.820 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:13:51.820 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:13:51.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.820 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.820 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.820 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:13:51.820 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:13:51.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.820 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.820 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.820 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:13:51.820 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:13:51.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.821 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.821 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:13:51.821 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:13:51.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.821 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.821 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:13:51.821 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:13:51.821 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.821 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.821 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.261566920000007, 2024-09-13T09:13:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:13:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:13:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.822 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:13:51.823 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.824 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.824 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:13:51.824 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:13:51.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.824 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.824 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:13:51.824 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:13:51.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.824 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.825 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:13:51.825 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:13:51.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.825 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.825 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:13:51.825 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:13:51.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.825 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.825 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:13:51.825 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:13:51.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.826 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.826 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:13:51.826 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:13:51.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.826 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.826 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.826 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:13:51.826 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:13:51.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.826 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.826 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.826 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:13:51.826 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:13:51.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.826 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.827 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.827 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:13:51.827 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:13:51.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.827 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.827 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.827 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:13:51.827 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:13:51.827 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.827 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.827 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.827 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.827 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.827 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:51.828 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.261541618999999, 2024-09-13T09:13:51.828 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:13:51.828 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.828 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.828 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.828 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.828 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.828 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:13:51.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.828 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.828 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.828 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:51.828 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.829 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.829 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:51.829 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.829 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.829 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:13:51.829 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.829 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.829 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:13:51.830 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:13:51.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.830 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.830 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:13:51.830 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:13:51.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.830 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.830 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:13:51.830 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:13:51.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.830 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.831 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:13:51.831 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:13:51.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.831 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.831 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.831 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:13:51.831 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:13:51.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.831 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.831 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.831 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:13:51.831 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:13:51.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.831 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.832 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.832 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:13:51.832 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:13:51.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.832 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.832 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.832 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:13:51.832 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:13:51.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.832 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.832 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.832 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:13:51.832 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:13:51.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.833 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.833 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.833 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:13:51.833 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:13:51.833 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.833 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.833 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.833 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.833 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.833 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:51.833 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.261523389999994, 2024-09-13T09:13:51.834 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:13:51.834 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.834 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.834 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.834 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.834 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.834 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:13:51.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.834 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.834 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.834 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.834 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:51.834 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.835 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.835 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:51.835 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.835 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.835 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:51.835 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.835 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.836 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:13:51.836 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:13:51.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.836 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.836 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.836 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:13:51.836 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:13:51.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.836 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.836 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.836 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:13:51.837 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:13:51.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.837 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.837 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.837 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:13:51.837 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:13:51.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.837 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.838 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.838 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:13:51.838 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:13:51.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.838 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.838 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.838 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:13:51.838 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:13:51.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.839 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.839 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.839 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:13:51.839 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:13:51.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.839 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.839 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.839 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:13:51.840 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:13:51.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.840 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.840 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.840 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:13:51.840 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:13:51.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.840 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.840 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.841 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:13:51.841 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:13:51.841 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.841 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.841 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.841 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.841 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.842 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:51.842 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.261476071000004, 2024-09-13T09:13:51.842 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:13:51.842 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.842 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.842 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.842 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.842 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.843 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:13:51.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.843 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.843 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.843 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:51.843 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.844 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.844 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:51.844 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.844 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.844 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.844 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:51.844 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.845 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.845 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.845 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:13:51.845 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:13:51.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.845 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.845 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.846 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:13:51.846 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:13:51.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.846 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.846 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.846 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:13:51.846 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:13:51.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.846 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.847 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.847 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:13:51.847 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:13:51.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.847 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.847 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.847 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:13:51.847 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:13:51.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.848 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.848 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:13:51.848 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:13:51.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.848 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.848 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:13:51.849 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:13:51.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.849 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.849 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.849 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:13:51.849 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:13:51.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.849 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.850 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.850 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:13:51.850 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:13:51.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.850 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.850 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:13:51.850 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:13:51.851 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.851 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.851 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.851 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.851 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.851 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.852 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:51.852 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.260961832999996, 2024-09-13T09:13:51.852 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:13:51.852 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.852 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.852 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.852 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.852 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.852 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:13:51.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.853 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.853 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.853 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:51.853 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.853 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.853 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:51.853 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.853 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.854 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:13:51.854 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:13:51.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.854 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.854 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:13:51.854 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:13:51.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.855 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.855 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:13:51.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:13:51.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.855 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.855 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:13:51.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:13:51.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.856 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.856 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:13:51.856 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:13:51.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.856 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.856 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.856 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:13:51.856 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:13:51.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.857 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.857 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:13:51.857 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:13:51.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.857 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.857 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:13:51.857 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:13:51.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.857 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:13:51.858 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:13:51.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.858 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:13:51.858 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:13:51.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.858 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:13:51.859 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:13:51.859 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.859 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.859 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.859 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.859 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:51.859 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.260927303000003, 2024-09-13T09:13:51.859 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:13:51.859 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.859 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.859 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.859 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.860 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.860 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:13:51.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.860 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.860 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.860 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.860 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:51.860 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.860 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.860 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.860 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:51.860 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:51.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:13:51.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:13:51.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:13:51.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:13:51.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:13:51.862 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:13:51.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:13:51.862 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:13:51.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:13:51.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:13:51.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.863 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.863 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.863 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:13:51.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:13:51.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.863 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.863 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.863 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:13:51.864 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:13:51.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.864 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.864 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:13:51.864 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:13:51.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.864 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.864 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:13:51.864 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:13:51.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.864 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.865 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:13:51.865 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:13:51.865 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.865 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.865 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.865 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.865 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:51.865 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.260909396000002, 2024-09-13T09:13:51.865 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:13:51.865 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.865 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.866 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.866 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.866 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.866 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:13:51.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.866 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.866 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:51.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:51.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:51.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:13:51.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:13:51.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.868 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.868 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:13:51.868 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:13:51.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.868 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.868 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.868 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:13:51.868 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:13:51.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.868 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.868 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.868 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:13:51.868 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:13:51.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.869 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.869 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.869 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:13:51.869 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:13:51.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.869 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.869 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.869 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:13:51.869 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:13:51.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.869 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.869 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.869 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:13:51.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:13:51.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:13:51.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:13:51.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:13:51.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:13:51.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.871 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:13:51.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:13:51.871 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.871 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.871 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.871 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.871 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:51.871 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.251782547999994, 2024-09-13T09:13:51.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:13:51.872 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.872 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.872 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.872 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.872 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.872 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:13:51.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.872 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.873 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:51.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.873 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.873 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:51.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.873 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.873 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:51.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:13:51.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:13:51.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:13:51.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:13:51.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:13:51.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:13:51.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.875 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:13:51.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:13:51.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.875 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:13:51.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:13:51.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.875 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:13:51.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:13:51.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.876 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.876 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:13:51.876 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:13:51.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.876 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.876 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:13:51.876 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:13:51.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.876 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.876 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:13:51.877 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:13:51.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.877 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.877 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:13:51.877 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:13:51.877 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.877 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.877 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.877 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.877 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:51.878 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.251730968000004, 2024-09-13T09:13:51.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:13:51.878 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.878 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.878 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.878 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.878 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.878 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:13:51.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.878 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:51.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:51.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:13:51.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.880 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:13:51.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:13:51.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.880 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:13:51.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:13:51.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.880 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:13:51.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:13:51.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.881 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.881 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:13:51.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:13:51.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.881 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.881 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:13:51.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:13:51.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:13:51.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:13:51.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:13:51.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:13:51.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:13:51.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:13:51.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:13:51.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:13:51.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:13:51.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:13:51.883 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.883 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.883 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.884 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.884 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:51.884 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.251714691999993, 2024-09-13T09:13:51.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:13:51.884 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.884 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.884 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.884 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.884 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.884 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:13:51.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.885 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:51.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:51.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.886 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:13:51.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.886 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:13:51.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:13:51.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.886 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:13:51.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:13:51.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:13:51.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:13:51.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:13:51.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:13:51.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:13:51.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:13:51.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:13:51.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:13:51.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:13:51.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:13:51.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:13:51.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:13:51.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:13:51.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:13:51.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:13:51.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:13:51.890 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.890 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.890 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.890 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.890 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:51.891 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.251698918000002, 2024-09-13T09:13:51.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:13:51.891 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.891 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.891 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.891 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.891 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.891 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:13:51.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.891 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:51.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:51.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:13:51.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:13:51.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:13:51.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:13:51.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:13:51.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:13:51.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:13:51.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:13:51.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:13:51.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:13:51.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:13:51.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:13:51.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:13:51.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:13:51.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:13:51.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:13:51.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:13:51.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:13:51.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:13:51.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:13:51.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:13:51.896 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.896 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.896 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.896 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.897 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:51.897 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.251684042999997, 2024-09-13T09:13:51.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:13:51.897 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.897 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.897 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.897 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.897 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.897 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:13:51.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.897 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:51.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:51.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:51.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:13:51.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:13:51.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:13:51.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:13:51.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:13:51.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:13:51.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:13:51.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:13:51.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:13:51.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:13:51.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:13:51.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:13:51.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:13:51.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:13:51.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:13:51.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:13:51.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:13:51.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:13:51.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:13:51.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:13:51.902 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.902 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.902 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.903 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.903 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:51.903 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.251666580999995, 2024-09-13T09:13:51.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:13:51.903 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.903 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.903 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.903 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.903 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.903 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:13:51.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.903 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:51.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:51.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:13:51.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:13:51.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:13:51.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:13:51.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:13:51.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:13:51.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:13:51.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:13:51.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:13:51.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:13:51.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:13:51.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:13:51.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:13:51.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:13:51.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:13:51.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:13:51.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:13:51.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:13:51.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:13:51.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:13:51.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:13:51.908 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.908 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.908 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.909 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.909 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:51.909 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.250065949000003, 2024-09-13T09:13:51.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:13:51.909 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.909 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.909 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.909 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.909 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.909 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:13:51.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.909 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:51.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:51.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:13:51.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:13:51.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:13:51.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:13:51.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:13:51.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:13:51.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:13:51.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:13:51.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:13:51.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:13:51.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:13:51.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:13:51.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:13:51.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:13:51.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:13:51.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:13:51.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:13:51.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:13:51.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:13:51.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:13:51.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:13:51.915 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.915 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.915 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.915 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.915 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:51.915 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.250025898000004, 2024-09-13T09:13:51.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:13:51.916 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.916 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.916 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.916 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.916 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.916 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:13:51.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.916 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:51.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:51.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:13:51.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:13:51.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:13:51.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:13:51.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:13:51.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:13:51.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:13:51.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:13:51.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:13:51.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:13:51.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:13:51.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:13:51.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:13:51.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:13:51.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:13:51.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:13:51.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:13:51.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:13:51.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:13:51.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:13:51.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:13:51.921 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.921 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.921 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.921 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:51.922 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:51.922 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.249982134999996, 2024-09-13T09:13:51.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:13:51.922 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:51.922 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:51.922 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:51.922 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:51.922 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:51.922 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:13:51.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.922 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:51.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:51.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:51.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:51.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:51.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:51.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:51.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:51.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:51.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:51.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:13:51.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:13:51.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:51.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:13:51.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:13:51.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:51.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:13:51.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:13:51.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:13:51.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:13:51.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:13:51.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:13:51.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:13:51.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:13:51.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:51.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:13:51.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:13:51.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:51.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:13:51.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:13:51.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:51.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:13:51.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:13:51.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:51.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:51.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:51.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:13:51.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:13:51.927 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.927 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.927 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.927 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:51.927 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:51.927 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:51.929 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:51.929 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:51.927+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:52.030 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:52.029+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:52.076 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:52.076 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:52.076 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:52.076 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:52.085 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:52.169 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:52.168+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:52.231 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:52.230+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:52.239 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:52.239 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:52.239 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:52.239 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:52.239 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:52.248 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:52.333 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:52.332+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:52.404 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:52.404 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:52.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:52.404 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:52.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.404 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.404 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:52.404 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.898441711000004, 2024-09-13T09:13:52.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:13:52.405 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.405 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.405 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.405 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.405 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.405 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:13:52.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.405 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.405 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:52.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:52.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:13:52.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:13:52.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:13:52.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:13:52.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:13:52.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:13:52.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:13:52.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:13:52.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:13:52.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:13:52.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:13:52.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:13:52.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:13:52.409 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.409 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.409 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.409 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.409 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:52.409 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.898414027000001, 2024-09-13T09:13:52.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:13:52.409 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.409 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.409 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.410 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.410 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.410 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:13:52.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.410 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:52.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:52.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.411 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.411 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:52.411 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.411 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.411 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:13:52.411 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:13:52.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.411 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.411 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:13:52.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:52.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:13:52.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:13:52.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:13:52.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:13:52.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:13:52.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:13:52.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:13:52.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:13:52.413 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.413 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.413 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.413 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.414 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:52.414 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.898389686000002, 2024-09-13T09:13:52.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:13:52.414 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.414 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.414 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.414 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.414 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.414 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:13:52.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.414 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:13:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:13:52.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:13:52.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:13:52.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:13:52.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:13:52.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:13:52.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.417 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.417 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:13:52.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:13:52.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.417 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.417 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:13:52.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:13:52.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.417 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.418 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:13:52.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:13:52.418 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.418 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.418 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.418 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.418 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:52.418 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.898373366000001, 2024-09-13T09:13:52.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:13:52.418 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.418 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.419 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.419 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.419 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.419 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:13:52.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.419 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:52.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:52.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:52.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:13:52.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:52.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:13:52.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:13:52.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:13:52.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:13:52.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:13:52.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:13:52.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:13:52.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:13:52.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:13:52.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:13:52.422 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.422 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.423 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.423 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.423 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:52.423 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.898356950999997, 2024-09-13T09:13:52.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:13:52.423 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.423 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.423 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.423 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.423 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.423 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:13:52.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.424 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.424 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:52.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.424 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:52.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:52.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:13:52.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:13:52.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:13:52.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:13:52.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:13:52.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:13:52.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:13:52.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:13:52.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:13:52.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:13:52.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:13:52.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:13:52.427 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.427 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.427 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.427 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.427 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:52.428 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.898340843, 2024-09-13T09:13:52.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:13:52.428 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.428 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.428 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.428 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.428 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.428 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:13:52.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.428 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:52.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:52.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:52.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.430 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:13:52.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:13:52.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.430 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.430 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:13:52.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:13:52.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.430 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.430 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:13:52.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:13:52.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:13:52.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:13:52.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:13:52.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:13:52.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:13:52.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:13:52.431 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.432 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.432 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.432 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.432 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:52.432 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.898325282000002, 2024-09-13T09:13:52.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:13:52.432 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.432 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.432 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.432 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.432 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.433 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:13:52.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.433 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:52.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:52.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:52.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:13:52.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:13:52.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:13:52.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:13:52.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:13:52.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:13:52.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:13:52.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:13:52.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:13:52.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:13:52.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:13:52.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:13:52.436 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.436 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.436 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.437 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.437 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:52.437 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.898308443999994, 2024-09-13T09:13:52.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:13:52.437 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.437 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.437 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.437 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.437 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.437 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:13:52.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.437 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.438 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.438 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:52.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.438 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.438 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:52.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.438 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.438 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:13:52.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:13:52.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:13:52.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:13:52.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:13:52.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:13:52.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:13:52.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:13:52.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:13:52.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:13:52.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:13:52.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:13:52.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:13:52.441 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.441 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.441 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.441 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.441 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:52.441 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.898292452000007, 2024-09-13T09:13:52.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:13:52.442 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.442 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.442 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.442 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.442 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.442 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:13:52.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.442 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.442 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.442 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:52.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.443 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:52.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.443 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:13:52.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:13:52.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:13:52.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.444 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:13:52.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:13:52.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.444 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:13:52.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:13:52.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:13:52.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:13:52.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:13:52.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:13:52.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:13:52.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:13:52.446 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.446 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.446 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.446 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.446 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:52.446 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.898264953999998, 2024-09-13T09:13:52.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:13:52.446 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.446 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.446 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.446 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.447 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.447 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:13:52.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.447 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:52.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:52.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:52.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:13:52.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:13:52.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:13:52.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:13:52.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:13:52.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:13:52.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:13:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:13:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:13:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:13:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:13:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:13:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.451 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.451 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.451 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.451 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:52.451 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.898239094999994, 2024-09-13T09:13:52.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:13:52.451 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.451 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.451 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.451 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.451 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.452 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:13:52.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.452 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.452 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:52.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.452 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:52.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:52.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:13:52.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:13:52.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:13:52.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:13:52.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:13:52.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:13:52.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:13:52.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:13:52.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:13:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:13:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:13:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:13:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.898197605000007, 2024-09-13T09:13:52.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:13:52.456 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.456 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.456 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.456 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.456 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.456 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:13:52.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.456 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.456 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:52.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:52.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:13:52.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:13:52.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:13:52.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:13:52.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:13:52.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:13:52.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:13:52.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:13:52.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:13:52.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:13:52.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:13:52.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:13:52.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:13:52.459 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.459 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.459 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.460 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.460 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:52.460 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.898176367000005, 2024-09-13T09:13:52.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:13:52.460 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.460 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.460 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.460 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.460 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.460 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:13:52.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.460 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:52.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:52.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:52.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:13:52.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:13:52.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:13:52.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:13:52.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:13:52.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:13:52.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.463 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:13:52.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:13:52.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.463 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:13:52.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:13:52.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.464 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:13:52.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:13:52.464 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.464 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.464 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.464 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.464 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:52.464 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.898153566999994, 2024-09-13T09:13:52.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:13:52.465 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.465 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.465 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.465 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.465 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.465 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:13:52.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.465 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:52.465 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.466 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:52.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.466 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:13:52.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.466 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:13:52.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:13:52.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.467 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:13:52.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:52.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.467 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:13:52.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:13:52.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.468 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.468 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:13:52.468 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:13:52.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.468 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.468 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:13:52.468 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:13:52.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.468 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.468 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:13:52.468 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:13:52.469 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.469 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.469 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.469 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.469 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:52.469 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.898137110999997, 2024-09-13T09:13:52.469 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:13:52.469 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.469 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.469 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.469 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:52.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:13:52.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:13:52.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:13:52.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:13:52.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.472 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.472 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:13:52.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:13:52.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.472 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.472 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.472 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:13:52.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:13:52.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.472 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.472 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.472 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:13:52.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:13:52.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.473 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.473 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.473 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:13:52.473 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:13:52.473 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.473 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.473 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.473 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.473 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.473 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:52.473 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.897891974000004, 2024-09-13T09:13:52.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:13:52.474 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.474 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.474 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.474 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.474 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.474 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:13:52.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.474 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.474 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:52.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:52.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:13:52.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:13:52.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:13:52.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:13:52.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:13:52.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:13:52.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:13:52.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:52.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:13:52.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:13:52.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:13:52.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:13:52.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:13:52.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:13:52.478 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.478 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.478 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.478 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.478 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:52.478 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.896006415000002, 2024-09-13T09:13:52.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:13:52.478 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.478 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.478 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.478 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.479 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.479 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:13:52.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.479 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:52.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:52.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:13:52.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:13:52.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:13:52.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:13:52.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:13:52.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:13:52.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:13:52.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:13:52.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:13:52.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:13:52.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:13:52.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:13:52.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:13:52.482 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.482 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.482 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.483 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.483 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:52.483 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.895985049000004, 2024-09-13T09:13:52.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:13:52.483 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.483 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.483 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.483 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.483 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.483 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:13:52.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.483 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:52.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:52.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:52.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:13:52.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:13:52.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:13:52.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:13:52.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:13:52.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:13:52.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:13:52.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:13:52.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:13:52.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:13:52.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:13:52.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:13:52.487 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.487 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.487 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.487 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:52.487 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:52.487 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 69.895779680999993, 2024-09-13T09:13:52.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:13:52.487 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.487 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:52.488 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.488 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:52.488 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.488 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:13:52.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.488 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:52.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:52.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:13:52.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:13:52.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:13:52.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:13:52.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:13:52.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:13:52.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:13:52.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:13:52.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:13:52.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:13:52.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:13:52.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:13:52.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:13:52.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:13:52.491 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.491 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.491 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.492 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5272.0:1775 3.0 3:02a42fca:::rbd_data.11235782ee8d.0000000000000030:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2244608~40448] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:13:52.492 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:52.492 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 42.15423784, 2024-09-13T09:13:52.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.666843732, 2024-09-13T09:13:52.492 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:52.492 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:52.492 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:52.492 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5272", 2024-09-13T09:13:52.492 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4089902097", 2024-09-13T09:13:52.492 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1775 2024-09-13T09:13:52.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.493 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:52.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:52.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:52.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:52.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:52.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:52.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:52.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247780+0000", 2024-09-13T09:13:52.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:13:52.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:52.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247865+0000", 2024-09-13T09:13:52.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4359999999999999e-05 2024-09-13T09:13:52.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:52.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247876+0000", 2024-09-13T09:13:52.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1456e-05 2024-09-13T09:13:52.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247883+0000", 2024-09-13T09:13:52.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:13:52.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247926+0000", 2024-09-13T09:13:52.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3203e-05 2024-09-13T09:13:52.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247932+0000", 2024-09-13T09:13:52.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:13:52.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909639+0000", 2024-09-13T09:13:52.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6617069099999999 2024-09-13T09:13:52.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:13:52.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909828+0000", 2024-09-13T09:13:52.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000189075 2024-09-13T09:13:52.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:52.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911676+0000", 2024-09-13T09:13:52.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001847235 2024-09-13T09:13:52.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:52.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911765+0000", 2024-09-13T09:13:52.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9530000000000005e-05 2024-09-13T09:13:52.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:13:52.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911901+0000", 2024-09-13T09:13:52.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013603899999999999 2024-09-13T09:13:52.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:13:52.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.912889+0000", 2024-09-13T09:13:52.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098807400000000003 2024-09-13T09:13:52.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914438+0000", 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015491070000000001 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914505+0000", 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914625+0000", 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120108 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.498 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.499 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.499 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:52.499 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:52.499 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:52.500 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:52.501 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:52.499+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:52.601 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:52.599+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:52.604 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:52.604 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:52.613 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:52.630 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:13:52.701 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:52.700+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:52.765 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:52.766 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:52.777 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:52.822 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:52.819+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:52.889 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.365350778000007, 2024-09-13T09:13:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:13:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:13:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:13:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:13:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:13:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:13:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:13:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:13:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:13:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:13:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:13:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:13:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:13:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:13:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:13:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:13:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:13:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.365301721999998, 2024-09-13T09:13:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:13:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:13:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:13:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:13:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:13:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:13:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:13:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:13:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:13:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:13:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:13:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:13:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:13:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:13:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:13:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:13:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:13:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.365261646999997, 2024-09-13T09:13:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:13:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:13:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:13:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:13:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:13:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:13:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:13:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:13:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:13:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:13:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:13:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:13:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:13:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:13:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:13:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:13:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:13:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.365231832000006, 2024-09-13T09:13:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:13:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:13:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:13:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:13:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:13:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:13:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:13:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:13:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:13:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:13:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:13:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:13:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:13:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:13:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:13:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:13:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.365202171000007, 2024-09-13T09:13:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:13:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:13:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:13:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:13:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:13:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:13:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:13:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:13:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:13:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:13:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:13:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:13:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:13:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:13:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:13:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:13:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:13:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.365172963999996, 2024-09-13T09:13:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:13:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:13:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:13:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:13:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:13:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:13:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:13:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:13:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:13:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:13:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:13:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:13:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:13:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:13:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:13:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:13:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:13:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.365116783000005, 2024-09-13T09:13:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:13:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:13:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:13:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:13:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:13:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:13:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:13:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:13:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:13:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:13:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:13:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:13:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:13:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:13:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:13:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:13:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:13:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.365063321999997, 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:13:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:13:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:13:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:13:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:13:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:13:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:13:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:13:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:13:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:13:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:13:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:13:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:13:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:13:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:13:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:13:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.365006492000006, 2024-09-13T09:13:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:13:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:13:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:13:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:13:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:13:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:13:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:13:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:13:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:13:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:13:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:13:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:13:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:13:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:13:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:13:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:13:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:13:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:13:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.364955464999994, 2024-09-13T09:13:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:13:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:13:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:13:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:13:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:13:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:13:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:13:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:13:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:13:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:13:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:13:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:13:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:13:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:13:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:13:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:13:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:13:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:13:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.364914275000004, 2024-09-13T09:13:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:13:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:13:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:13:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:13:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:13:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:13:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:13:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:13:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:13:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:13:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:13:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:13:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:13:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:13:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:13:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:13:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:13:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.364872159000001, 2024-09-13T09:13:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:13:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:13:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:13:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:13:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:13:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:13:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:13:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:13:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:13:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:13:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:13:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:13:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:13:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:13:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:13:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.947 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:13:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:13:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:13:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.364828279999998, 2024-09-13T09:13:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:13:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:13:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:13:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:13:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:13:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:13:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:13:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:13:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:13:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:13:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:13:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:13:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:13:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:13:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:13:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:13:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:13:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.364782363000003, 2024-09-13T09:13:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:13:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:13:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:13:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:13:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:13:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:13:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:13:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:13:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:13:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:13:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:13:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:13:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:13:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:13:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:13:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:13:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:13:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:13:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.364732591999996, 2024-09-13T09:13:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:13:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:13:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:13:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:13:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:13:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:13:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:13:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:13:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:13:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:13:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:13:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:13:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:13:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:13:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:13:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:13:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:13:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.364687188000005, 2024-09-13T09:13:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:13:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:13:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:13:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:13:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:13:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:13:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:13:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:13:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:13:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:13:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:13:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:13:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:13:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:13:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:13:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:13:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:13:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.36463655, 2024-09-13T09:13:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:13:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:13:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:13:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:13:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:13:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:13:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:13:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:13:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:13:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:13:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:13:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:13:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:13:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:13:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:13:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:13:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:13:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:13:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.364580707000002, 2024-09-13T09:13:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:13:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:13:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:13:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:13:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:13:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:13:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:13:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:13:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:13:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:13:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:13:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:13:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:13:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:13:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:13:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:13:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:13:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.364523785000003, 2024-09-13T09:13:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:13:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:13:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:13:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:13:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:13:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:13:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:13:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:13:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:13:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:13:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:13:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:13:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:13:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:13:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:13:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:13:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:13:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:13:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 70.364470221999994, 2024-09-13T09:13:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:13:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:13:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:13:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:13:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:13:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:13:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:13:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:13:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:13:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:13:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:13:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:13:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:13:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:13:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:13:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:13:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:13:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:13:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:52.992 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:52.993 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:52.993 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:52.946+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:53.049 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:53.047+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:53.060 INFO:teuthology.orchestra.run.smithi157.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-13T09:13:53.070 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:53.086 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:53.086 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:53.087 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:13:53.103 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:53.104 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:53.108 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:53.108 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:53.108 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:53.109 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:53.118 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:53.120 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:53.120 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:53.136 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:53.137 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:53.139 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:13:53.140 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:13:53.162 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:53.161+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:53.227 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:53.227 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:53.237 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:53.280 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:53.278+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:53.351 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:53.350+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:53.382 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:53.382 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:53.391 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:53.452 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:53.450+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:53.494 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:53.494 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:53.503 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:53.589 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:53.587+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:53.620 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5] 2024-09-13T09:13:53.620 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:13:53.621 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-13T09:13:53.621 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:13:53.621 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:13:53.621 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-13T09:13:53.622 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-09-13T09:13:53.658 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:53.658 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:53.659 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:53.657+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:53.667 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:53.738 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:53.738 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "" 2024-09-13T09:13:53.738 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:53.754 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:53.753+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:53.815 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:53.815 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:53.824 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:53.856 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:53.854+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:53.932 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:53.933 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:53.941 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:53.985 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:53.984+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:54.048 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:54.048 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:54.057 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:54.058 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:54.057+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:54.159 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:54.158+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:54.203 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:54.204 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:54.213 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:54.260 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:54.259+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:54.320 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:54.320 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:54.329 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:54.361 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:54.359+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:54.462 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:54.460+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:54.475 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:54.476 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:54.484 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:54.562 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:54.561+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:54.590 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:54.591 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:54.599 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:54.663 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:54.662+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:54.742 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:54.742 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:54.751 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:54.793 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:54.791+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:54.864 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:54.863+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:54.895 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:54.895 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:54.903 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:54.988 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:54.986+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:55.046 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:55.046 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:55.055 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:55.066 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:55.065+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:55.141 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:13:55.167 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:55.165+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:55.197 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:55.198 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:55.206 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:55.267 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:55.267 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:55.284 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:55.285 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:55.285 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:13:55.290 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:55.289+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:55.300 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:55.300 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:55.316 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:55.317 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:55.333 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:55.334 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:55.337 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:13:55.337 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:13:55.350 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:55.351 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:55.351 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:55.351 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:55.351 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:55.359 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:55.401 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:55.399+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:55.469 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:55.468+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:55.505 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:55.506 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:55.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:55.506 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:55.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.506 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2185 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.506 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:55.506 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977909699999998, 2024-09-13T09:13:55.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063348994000000006, 2024-09-13T09:13:55.506 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.506 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.506 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.506 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.507 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.507 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2185 2024-09-13T09:13:55.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.507 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:55.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:55.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:13:55.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525765+0000", 2024-09-13T09:13:55.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2390000000000001e-06 2024-09-13T09:13:55.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525772+0000", 2024-09-13T09:13:55.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:13:55.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525777+0000", 2024-09-13T09:13:55.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0799999999999996e-06 2024-09-13T09:13:55.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589020+0000", 2024-09-13T09:13:55.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063242501000000007 2024-09-13T09:13:55.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589063+0000", 2024-09-13T09:13:55.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2673999999999998e-05 2024-09-13T09:13:55.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589112+0000", 2024-09-13T09:13:55.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9286000000000002e-05 2024-09-13T09:13:55.510 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.511 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.511 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.511 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2187 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.511 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:55.511 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977890152999997, 2024-09-13T09:13:55.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063500102000000003, 2024-09-13T09:13:55.511 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.511 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.511 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.511 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.511 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.511 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2187 2024-09-13T09:13:55.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.512 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:55.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:55.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:13:55.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525784+0000", 2024-09-13T09:13:55.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1629999999999999e-06 2024-09-13T09:13:55.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525792+0000", 2024-09-13T09:13:55.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:55.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525796+0000", 2024-09-13T09:13:55.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7389999999999999e-06 2024-09-13T09:13:55.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589132+0000", 2024-09-13T09:13:55.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063336119999999996 2024-09-13T09:13:55.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589187+0000", 2024-09-13T09:13:55.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4126000000000003e-05 2024-09-13T09:13:55.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589283+0000", 2024-09-13T09:13:55.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6032999999999993e-05 2024-09-13T09:13:55.516 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.516 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.516 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.517 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2188 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.517 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:55.517 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977872107000003, 2024-09-13T09:13:55.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063724617999999997, 2024-09-13T09:13:55.517 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.517 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.517 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.517 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.517 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.517 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2188 2024-09-13T09:13:55.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.518 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:55.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:55.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:13:55.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525802+0000", 2024-09-13T09:13:55.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0439999999999998e-06 2024-09-13T09:13:55.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525809+0000", 2024-09-13T09:13:55.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.238e-06 2024-09-13T09:13:55.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525814+0000", 2024-09-13T09:13:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-13T09:13:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589327+0000", 2024-09-13T09:13:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063513257000000004 2024-09-13T09:13:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589418+0000", 2024-09-13T09:13:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0796000000000003e-05 2024-09-13T09:13:55.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589525+0000", 2024-09-13T09:13:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010693500000000001 2024-09-13T09:13:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2190 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977854020999999, 2024-09-13T09:13:55.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063863981, 2024-09-13T09:13:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2190 2024-09-13T09:13:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:13:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525818+0000", 2024-09-13T09:13:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525820+0000", 2024-09-13T09:13:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:13:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525827+0000", 2024-09-13T09:13:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0589999999999999e-06 2024-09-13T09:13:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525831+0000", 2024-09-13T09:13:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7870000000000001e-06 2024-09-13T09:13:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589562+0000", 2024-09-13T09:13:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063731038000000004 2024-09-13T09:13:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589612+0000", 2024-09-13T09:13:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0087999999999999e-05 2024-09-13T09:13:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589683+0000", 2024-09-13T09:13:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0121000000000002e-05 2024-09-13T09:13:55.525 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2192 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977837061000002, 2024-09-13T09:13:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064050299000000005, 2024-09-13T09:13:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2192 2024-09-13T09:13:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:13:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525835+0000", 2024-09-13T09:13:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525837+0000", 2024-09-13T09:13:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:13:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525845+0000", 2024-09-13T09:13:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3660000000000003e-06 2024-09-13T09:13:55.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525850+0000", 2024-09-13T09:13:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0030000000000002e-06 2024-09-13T09:13:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589720+0000", 2024-09-13T09:13:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063870176000000001 2024-09-13T09:13:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589801+0000", 2024-09-13T09:13:55.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0889000000000004e-05 2024-09-13T09:13:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589886+0000", 2024-09-13T09:13:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5183000000000001e-05 2024-09-13T09:13:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977818958, 2024-09-13T09:13:55.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:13:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:13:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:13:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:13:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:13:55.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:13:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:13:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:13:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:13:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:13:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:13:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:13:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:13:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:13:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:13:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:13:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.534 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.97780126, 2024-09-13T09:13:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:13:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.535 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:13:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:13:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:13:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:13:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:13:55.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:13:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:13:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:13:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:13:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:13:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:13:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:13:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:13:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:13:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.539 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977780645000003, 2024-09-13T09:13:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:13:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:13:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:13:55.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:13:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:13:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:13:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:13:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:13:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:13:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:13:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:13:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:13:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:13:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:13:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:13:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977761688999998, 2024-09-13T09:13:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:13:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:13:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:13:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:13:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:13:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:13:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:13:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:13:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:13:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:13:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:13:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:13:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:13:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:13:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:13:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977744430999998, 2024-09-13T09:13:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:13:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:13:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:13:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:13:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:13:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:13:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:13:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:13:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:13:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:13:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:13:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:13:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:13:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:13:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:13:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.553 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977724416999997, 2024-09-13T09:13:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:13:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:13:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:13:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:13:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:13:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:13:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:13:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:13:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:13:55.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:13:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:13:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:13:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:13:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:13:55.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:13:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977698834999998, 2024-09-13T09:13:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:13:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.558 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:13:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:13:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:13:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:13:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:13:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:13:55.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:13:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:13:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:13:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:13:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:13:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:13:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:13:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:13:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:13:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:55.562 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977679551000001, 2024-09-13T09:13:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:13:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:13:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:13:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:13:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:13:55.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:13:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:13:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:13:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:13:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:13:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:13:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:13:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:13:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:13:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:13:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977662295000002, 2024-09-13T09:13:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:13:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:13:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.567 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:13:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:13:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:13:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:13:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:13:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:13:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:13:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:13:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:13:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:13:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:13:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:13:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:13:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977642815000003, 2024-09-13T09:13:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:13:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.571 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:13:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:13:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:13:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:13:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:13:55.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:13:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:13:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:13:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:13:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:13:55.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:13:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:13:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:13:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:13:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:13:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977625885999998, 2024-09-13T09:13:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:13:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:13:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:13:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:13:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:13:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:13:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:13:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:13:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:13:55.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:13:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:13:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:13:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:13:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:13:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:13:55.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:13:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977606737000002, 2024-09-13T09:13:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:13:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:13:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:13:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:13:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:13:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:13:55.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:13:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:13:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:13:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:13:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:13:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:13:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:13:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:13:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:13:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977588597999997, 2024-09-13T09:13:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:13:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.585 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:13:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:55.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:13:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:13:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:13:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:13:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:13:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:13:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:13:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:13:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:13:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:13:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:13:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:13:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:13:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977178103, 2024-09-13T09:13:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:13:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:13:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:13:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:13:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:13:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:13:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:13:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:13:55.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:13:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:13:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:13:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:13:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:13:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:13:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:13:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:13:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.977155201999999, 2024-09-13T09:13:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:13:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:55.594 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:13:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:13:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:13:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:55.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:13:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:13:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:13:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:13:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:13:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:13:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:13:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:13:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:13:55.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:13:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:13:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:13:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:13:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.598 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:55.599 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:55.599 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:55.600 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:55.601 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:55.599+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:55.637 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:13:55.703 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:55.700+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:55.754 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:55.754 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:55.754 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:55.755 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:55.764 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:55.805 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:55.803+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:55.903 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:55.901+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:55.906 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:55.906 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:55.906 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:55.907 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:55.907 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:55.915 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:56.004 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:56.003+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:56.061 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:56.061 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:13:56.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:13:56.061 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:13:56.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.061 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:56.061 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:56.061 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.313153583000002, 2024-09-13T09:13:56.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:13:56.062 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.062 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:56.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:56.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:56.062 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:13:56.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.062 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:56.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:56.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:13:56.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:13:56.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:13:56.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:13:56.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:13:56.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:13:56.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:13:56.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:13:56.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:13:56.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:13:56.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:13:56.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:13:56.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:13:56.066 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.066 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.066 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.066 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:56.066 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:56.067 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.313144905000001, 2024-09-13T09:13:56.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:13:56.067 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.067 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:56.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:56.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:56.067 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:13:56.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.067 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:56.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:56.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:13:56.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:56.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:13:56.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:13:56.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:13:56.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:13:56.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:13:56.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:13:56.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:13:56.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:13:56.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:13:56.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:13:56.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:13:56.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:13:56.071 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.071 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.071 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.072 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:56.072 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:56.072 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.313135918999997, 2024-09-13T09:13:56.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:13:56.072 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.072 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:56.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:56.073 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:56.073 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:13:56.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.073 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:56.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:56.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:13:56.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:13:56.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:13:56.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:13:56.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:13:56.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:13:56.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:13:56.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:13:56.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:13:56.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:13:56.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:13:56.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:13:56.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:13:56.078 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.078 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.078 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.078 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:56.078 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:56.078 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.313127946999998, 2024-09-13T09:13:56.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:13:56.078 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.078 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:56.078 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:56.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:56.079 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:13:56.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.079 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:56.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:56.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:13:56.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:56.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:13:56.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:13:56.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:13:56.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:13:56.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:13:56.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:13:56.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:13:56.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:13:56.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:13:56.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:13:56.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:13:56.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:13:56.083 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.083 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.083 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.083 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:56.083 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:56.083 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.311253112999999, 2024-09-13T09:13:56.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:13:56.084 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.084 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:56.084 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.084 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:56.084 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:56.084 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:13:56.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.084 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:56.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:56.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:13:56.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:13:56.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:13:56.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:13:56.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:13:56.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:13:56.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:13:56.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:13:56.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:13:56.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:13:56.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:13:56.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:13:56.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:13:56.088 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.088 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.088 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.088 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:56.088 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:56.089 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.311242452000002, 2024-09-13T09:13:56.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:13:56.089 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.089 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:56.089 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.089 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:56.090 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:56.090 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:13:56.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.090 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:56.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:56.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:13:56.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:56.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:13:56.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:13:56.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:13:56.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:13:56.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:13:56.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:13:56.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:13:56.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:13:56.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:13:56.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:13:56.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:13:56.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:13:56.098 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.098 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.098 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.098 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:56.098 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:56.098 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.311220476000003, 2024-09-13T09:13:56.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:13:56.098 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.098 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:56.099 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.099 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:56.099 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:56.099 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:13:56.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.099 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:56.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:56.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:13:56.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:13:56.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:13:56.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:13:56.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:13:56.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:13:56.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:13:56.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:13:56.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:13:56.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:13:56.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:13:56.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:13:56.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:13:56.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.107 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.108 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.109 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:56.109 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:56.109 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.311206974000001, 2024-09-13T09:13:56.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:13:56.110 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.110 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:56.110 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.110 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:56.110 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:56.110 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:13:56.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.111 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:56.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:56.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:13:56.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:13:56.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:13:56.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:13:56.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:13:56.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:13:56.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:13:56.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:13:56.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:13:56.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:13:56.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:13:56.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:13:56.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:13:56.114 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.114 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.114 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.115 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:56.115 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:56.115 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.309197875999999, 2024-09-13T09:13:56.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:13:56.115 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.115 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:56.115 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.115 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:56.115 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:56.115 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:13:56.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.115 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:56.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:56.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:13:56.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:13:56.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:13:56.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:13:56.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:13:56.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:13:56.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:13:56.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:13:56.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:13:56.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:13:56.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:13:56.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:13:56.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:13:56.119 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.119 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.119 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.119 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:56.119 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:56.119 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.309173553999997, 2024-09-13T09:13:56.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:13:56.120 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.120 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:56.120 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.120 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:56.120 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:56.120 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:13:56.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.120 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:56.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:56.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:13:56.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:13:56.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:13:56.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:13:56.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:13:56.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:13:56.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:13:56.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:13:56.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:13:56.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:13:56.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:13:56.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:13:56.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:13:56.123 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.123 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.123 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.124 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:56.124 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:56.124 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.309162579999999, 2024-09-13T09:13:56.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:13:56.124 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.124 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:56.124 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.124 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:56.124 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:56.124 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:13:56.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.125 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:56.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:13:56.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:13:56.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:13:56.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:13:56.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:13:56.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:13:56.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:13:56.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:13:56.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:13:56.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:13:56.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:13:56.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:13:56.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:13:56.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:13:56.128 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.128 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.128 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.128 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:56.128 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:56.128 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.307244955000002, 2024-09-13T09:13:56.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:13:56.129 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.129 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:56.129 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.129 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:56.129 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:56.129 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:13:56.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.129 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:56.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:56.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:13:56.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:13:56.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:13:56.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:13:56.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:13:56.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:13:56.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:13:56.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:13:56.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:13:56.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:13:56.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:13:56.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:13:56.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:13:56.132 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.132 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.132 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.133 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:13:56.133 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:56.133 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.307235558000002, 2024-09-13T09:13:56.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:13:56.133 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.133 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:56.133 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.133 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:13:56.133 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:13:56.133 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:13:56.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.133 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:56.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:56.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:13:56.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:13:56.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:13:56.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:13:56.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:13:56.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:13:56.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:13:56.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:13:56.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:13:56.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:13:56.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:13:56.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:13:56.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:13:56.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:13:56.137 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.137 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.137 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.137 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3181 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T09:13:56.137 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:56.137 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.036871382000001, 2024-09-13T09:13:56.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.480957603, 2024-09-13T09:13:56.138 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.138 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:13:56.138 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.138 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:13:56.138 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:13:56.138 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3181 2024-09-13T09:13:56.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.138 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:56.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:13:56.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021894+0000", 2024-09-13T09:13:56.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:13:56.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021903+0000", 2024-09-13T09:13:56.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4109999999999996e-06 2024-09-13T09:13:56.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021919+0000", 2024-09-13T09:13:56.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6225000000000001e-05 2024-09-13T09:13:56.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021926+0000", 2024-09-13T09:13:56.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4120000000000004e-06 2024-09-13T09:13:56.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021975+0000", 2024-09-13T09:13:56.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8526000000000003e-05 2024-09-13T09:13:56.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022049+0000", 2024-09-13T09:13:56.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4018999999999997e-05 2024-09-13T09:13:56.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:13:56.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022230+0000", 2024-09-13T09:13:56.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018085599999999999 2024-09-13T09:13:56.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:13:56.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022796+0000", 2024-09-13T09:13:56.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056638999999999995 2024-09-13T09:13:56.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499540+0000", 2024-09-13T09:13:56.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.47674342800000002 2024-09-13T09:13:56.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499547+0000", 2024-09-13T09:13:56.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:13:56.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502794+0000", 2024-09-13T09:13:56.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.003246818 2024-09-13T09:13:56.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502854+0000", 2024-09-13T09:13:56.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9925e-05 2024-09-13T09:13:56.144 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.144 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.144 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.144 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3182 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 139818970156544 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:13:56.144 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:56.144 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 25.563636827, 2024-09-13T09:13:56.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0089058390000001, 2024-09-13T09:13:56.144 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.144 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:56.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:13:56.145 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:13:56.145 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3182 2024-09-13T09:13:56.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.145 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:56.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:13:56.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495128+0000", 2024-09-13T09:13:56.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:13:56.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495139+0000", 2024-09-13T09:13:56.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0599e-05 2024-09-13T09:13:56.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495157+0000", 2024-09-13T09:13:56.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7830999999999999e-05 2024-09-13T09:13:56.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495167+0000", 2024-09-13T09:13:56.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0689999999999999e-05 2024-09-13T09:13:56.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495229+0000", 2024-09-13T09:13:56.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1898999999999998e-05 2024-09-13T09:13:56.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:13:56.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495243+0000", 2024-09-13T09:13:56.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3349e-05 2024-09-13T09:13:56.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499575+0000", 2024-09-13T09:13:56.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043324410000000002 2024-09-13T09:13:56.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499582+0000", 2024-09-13T09:13:56.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0849999999999999e-06 2024-09-13T09:13:56.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502872+0000", 2024-09-13T09:13:56.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0032895150000001 2024-09-13T09:13:56.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503016+0000", 2024-09-13T09:13:56.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014394200000000001 2024-09-13T09:13:56.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504036+0000", 2024-09-13T09:13:56.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010208680000000001 2024-09-13T09:13:56.149 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.150 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.150 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.150 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:19151 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 140383089882784 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:13:56.150 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:56.150 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 25.563625368, 2024-09-13T09:13:56.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.009071287, 2024-09-13T09:13:56.150 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.150 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:13:56.150 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.150 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:13:56.150 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:13:56.151 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19151 2024-09-13T09:13:56.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.151 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:56.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:13:56.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495140+0000", 2024-09-13T09:13:56.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:13:56.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495149+0000", 2024-09-13T09:13:56.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0759999999999992e-06 2024-09-13T09:13:56.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495165+0000", 2024-09-13T09:13:56.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5322e-05 2024-09-13T09:13:56.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495173+0000", 2024-09-13T09:13:56.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0979999999999994e-06 2024-09-13T09:13:56.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495318+0000", 2024-09-13T09:13:56.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014553 2024-09-13T09:13:56.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:13:56.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495325+0000", 2024-09-13T09:13:56.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:13:56.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499598+0000", 2024-09-13T09:13:56.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004273064 2024-09-13T09:13:56.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499602+0000", 2024-09-13T09:13:56.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8539999999999999e-06 2024-09-13T09:13:56.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503507+0000", 2024-09-13T09:13:56.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0039045520000001 2024-09-13T09:13:56.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503583+0000", 2024-09-13T09:13:56.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6283000000000001e-05 2024-09-13T09:13:56.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504213+0000", 2024-09-13T09:13:56.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00063044700000000002 2024-09-13T09:13:56.156 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.156 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.156 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.156 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3908 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:13:56.156 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:13:56.156 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9660884769999996, 2024-09-13T09:13:56.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070530317999999995, 2024-09-13T09:13:56.156 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.156 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:56.156 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.157 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:13:56.157 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:13:56.157 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3908 2024-09-13T09:13:56.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.157 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:13:56.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:13:56.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092678+0000", 2024-09-13T09:13:56.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092681+0000", 2024-09-13T09:13:56.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.903e-06 2024-09-13T09:13:56.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092687+0000", 2024-09-13T09:13:56.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0859999999999998e-06 2024-09-13T09:13:56.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092692+0000", 2024-09-13T09:13:56.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-13T09:13:56.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.159711+0000", 2024-09-13T09:13:56.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067018922999999994 2024-09-13T09:13:56.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.159796+0000", 2024-09-13T09:13:56.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5247000000000006e-05 2024-09-13T09:13:56.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:56.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160111+0000", 2024-09-13T09:13:56.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00031449499999999998 2024-09-13T09:13:56.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:13:56.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162219+0000", 2024-09-13T09:13:56.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021085570000000001 2024-09-13T09:13:56.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:56.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163127+0000", 2024-09-13T09:13:56.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00090723400000000004 2024-09-13T09:13:56.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:13:56.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163172+0000", 2024-09-13T09:13:56.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5383000000000002e-05 2024-09-13T09:13:56.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163209+0000", 2024-09-13T09:13:56.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7132000000000003e-05 2024-09-13T09:13:56.162 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.162 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.162 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.162 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3909 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:13:56.162 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:13:56.162 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9660709860000001, 2024-09-13T09:13:56.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070616834000000003, 2024-09-13T09:13:56.162 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.162 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:56.162 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.163 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:13:56.163 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:13:56.163 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3909 2024-09-13T09:13:56.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.163 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:13:56.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:13:56.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:13:56.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092699+0000", 2024-09-13T09:13:56.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:13:56.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092704+0000", 2024-09-13T09:13:56.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5940000000000001e-06 2024-09-13T09:13:56.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092709+0000", 2024-09-13T09:13:56.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5109999999999997e-06 2024-09-13T09:13:56.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160604+0000", 2024-09-13T09:13:56.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067895633999999996 2024-09-13T09:13:56.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160658+0000", 2024-09-13T09:13:56.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3292999999999999e-05 2024-09-13T09:13:56.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:56.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160935+0000", 2024-09-13T09:13:56.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000276879 2024-09-13T09:13:56.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:13:56.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163223+0000", 2024-09-13T09:13:56.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022886320000000001 2024-09-13T09:13:56.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:56.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163258+0000", 2024-09-13T09:13:56.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5101000000000002e-05 2024-09-13T09:13:56.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:13:56.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163292+0000", 2024-09-13T09:13:56.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3237e-05 2024-09-13T09:13:56.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163313+0000", 2024-09-13T09:13:56.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1642e-05 2024-09-13T09:13:56.168 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.168 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.168 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.168 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3910 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:13:56.168 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:13:56.168 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9660539220000004, 2024-09-13T09:13:56.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733837999999993, 2024-09-13T09:13:56.168 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.168 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:56.169 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.169 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:13:56.169 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:13:56.169 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3910 2024-09-13T09:13:56.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.169 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:13:56.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:13:56.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:13:56.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092716+0000", 2024-09-13T09:13:56.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.757e-06 2024-09-13T09:13:56.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092722+0000", 2024-09-13T09:13:56.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7869999999999998e-06 2024-09-13T09:13:56.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092726+0000", 2024-09-13T09:13:56.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.622e-06 2024-09-13T09:13:56.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161487+0000", 2024-09-13T09:13:56.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068761166999999998 2024-09-13T09:13:56.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161569+0000", 2024-09-13T09:13:56.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1185000000000003e-05 2024-09-13T09:13:56.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:56.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161868+0000", 2024-09-13T09:13:56.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029979799999999999 2024-09-13T09:13:56.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:56.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163366+0000", 2024-09-13T09:13:56.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001497888 2024-09-13T09:13:56.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:13:56.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163373+0000", 2024-09-13T09:13:56.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1640000000000004e-06 2024-09-13T09:13:56.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:13:56.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163412+0000", 2024-09-13T09:13:56.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8423999999999997e-05 2024-09-13T09:13:56.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163447+0000", 2024-09-13T09:13:56.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5441000000000002e-05 2024-09-13T09:13:56.174 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.174 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.174 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.175 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3911 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:13:56.175 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:13:56.175 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9660340060000001, 2024-09-13T09:13:56.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.072284372999999999, 2024-09-13T09:13:56.175 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:13:56.175 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:56.175 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:13:56.175 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:13:56.175 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:13:56.175 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3911 2024-09-13T09:13:56.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.175 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:13:56.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:13:56.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:13:56.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:13:56.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:13:56.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:13:56.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:13:56.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:13:56.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:13:56.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092736+0000", 2024-09-13T09:13:56.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9510000000000002e-06 2024-09-13T09:13:56.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:13:56.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092743+0000", 2024-09-13T09:13:56.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9e-06 2024-09-13T09:13:56.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092748+0000", 2024-09-13T09:13:56.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999998e-06 2024-09-13T09:13:56.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:13:56.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162257+0000", 2024-09-13T09:13:56.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.069508890000000004 2024-09-13T09:13:56.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:13:56.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162342+0000", 2024-09-13T09:13:56.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5886999999999999e-05 2024-09-13T09:13:56.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:13:56.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162682+0000", 2024-09-13T09:13:56.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00033971700000000001 2024-09-13T09:13:56.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:13:56.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164897+0000", 2024-09-13T09:13:56.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022152370000000001 2024-09-13T09:13:56.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:56.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164945+0000", 2024-09-13T09:13:56.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7243999999999997e-05 2024-09-13T09:13:56.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:13:56.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164985+0000", 2024-09-13T09:13:56.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0599000000000002e-05 2024-09-13T09:13:56.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:13:56.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:13:56.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:13:56.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.165018+0000", 2024-09-13T09:13:56.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2549000000000003e-05 2024-09-13T09:13:56.180 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.180 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.180 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.180 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:13:56.180 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:13:56.181 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:56.182 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:56.180+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:56.183 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:56.283 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:56.282+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:56.291 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:56.291 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:56.291 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:56.291 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:56.300 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:56.384 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:56.383+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:56.447 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:56.447 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:56.447 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:56.447 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:56.447 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:56.455 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:56.496 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:56.495+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:56.586 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:56.584+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:56.604 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:56.604 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:56.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:56.605 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:56.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.605 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.605 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:56.605 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.073138338999996, 2024-09-13T09:13:56.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:13:56.605 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.605 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.605 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.605 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.605 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.606 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:13:56.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.606 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:56.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:13:56.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:13:56.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:13:56.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:13:56.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:13:56.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:13:56.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:13:56.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:13:56.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:13:56.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:13:56.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:13:56.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:13:56.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:13:56.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:13:56.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:13:56.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:13:56.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:13:56.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:13:56.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:13:56.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:13:56.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:13:56.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:13:56.612 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.612 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.612 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.612 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.612 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:56.612 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.073120304, 2024-09-13T09:13:56.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:13:56.612 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.612 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.612 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.612 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.613 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.613 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:13:56.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.613 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:56.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:56.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:13:56.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:13:56.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:13:56.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:13:56.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:13:56.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:13:56.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:13:56.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:13:56.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:13:56.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:13:56.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:13:56.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:13:56.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:13:56.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:13:56.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:13:56.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:13:56.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:13:56.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:13:56.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:13:56.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:13:56.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:13:56.618 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.618 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.618 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.618 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.618 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:56.618 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.073103634000006, 2024-09-13T09:13:56.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:13:56.618 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.618 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.618 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.618 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.619 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.619 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:13:56.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.619 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:56.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:13:56.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:13:56.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:13:56.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:56.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:13:56.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:13:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:13:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:13:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:13:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:13:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:13:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:13:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:13:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:13:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:13:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:13:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:13:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:13:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:13:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:13:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:13:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:13:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.623 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.073086001999997, 2024-09-13T09:13:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:13:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.624 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:13:56.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.625 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:56.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:56.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:13:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:13:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:13:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:13:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:13:56.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:13:56.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:13:56.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:13:56.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:13:56.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:13:56.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:13:56.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:13:56.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:13:56.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:13:56.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:13:56.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:13:56.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:13:56.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:13:56.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:13:56.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:13:56.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:13:56.629 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.630 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.630 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.630 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.630 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:56.630 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.073039696999999, 2024-09-13T09:13:56.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:13:56.630 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.630 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.630 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.630 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.630 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.630 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:13:56.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.631 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:56.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:13:56.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:13:56.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:13:56.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:13:56.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:13:56.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:13:56.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:13:56.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:13:56.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:13:56.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:13:56.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:13:56.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:13:56.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:13:56.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:13:56.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:13:56.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:13:56.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:13:56.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:13:56.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:13:56.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:13:56.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:13:56.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:13:56.635 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.636 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.636 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.636 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.636 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:56.636 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.073014396000005, 2024-09-13T09:13:56.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:13:56.636 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.636 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.636 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.636 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.636 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.636 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:13:56.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.637 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:56.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:13:56.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:13:56.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:13:56.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:13:56.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:13:56.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:13:56.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:13:56.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:13:56.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:13:56.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:13:56.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:13:56.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:13:56.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:13:56.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:13:56.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:13:56.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:13:56.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:13:56.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:13:56.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:13:56.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:13:56.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:13:56.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:13:56.642 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.642 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.642 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.642 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.642 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:56.642 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.072996166999999, 2024-09-13T09:13:56.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:13:56.642 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.642 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.643 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.643 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.643 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.643 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:13:56.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.643 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:56.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:56.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:13:56.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:13:56.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:13:56.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:13:56.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:13:56.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:13:56.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:13:56.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:13:56.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:13:56.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:13:56.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:13:56.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:13:56.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:13:56.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:13:56.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:13:56.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:13:56.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:13:56.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:13:56.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:13:56.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:13:56.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:13:56.648 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.648 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.648 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.648 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.648 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:56.648 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.072948847999996, 2024-09-13T09:13:56.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:13:56.648 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.648 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.649 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.649 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.649 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.649 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:13:56.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.649 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:56.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:56.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:13:56.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:13:56.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:13:56.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:13:56.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:13:56.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:13:56.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:13:56.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:13:56.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:13:56.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:13:56.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:13:56.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:13:56.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:13:56.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:13:56.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:13:56.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:13:56.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:13:56.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:13:56.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:13:56.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:13:56.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:13:56.654 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.654 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.654 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.654 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.654 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:56.654 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.072434610000002, 2024-09-13T09:13:56.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:13:56.654 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.655 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.655 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.655 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.655 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.655 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:13:56.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.655 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:56.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:13:56.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:13:56.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:13:56.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:13:56.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:13:56.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:13:56.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:13:56.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:13:56.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:13:56.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:13:56.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:13:56.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:13:56.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:13:56.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:13:56.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:13:56.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:13:56.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:13:56.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:13:56.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:13:56.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:13:56.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:13:56.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:13:56.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:13:56.660 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.660 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.660 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.661 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.661 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:56.661 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.072400079999994, 2024-09-13T09:13:56.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:13:56.661 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.661 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.661 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.661 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.661 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.661 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:13:56.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.661 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:56.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:56.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:13:56.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:13:56.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:13:56.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:13:56.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:13:56.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:13:56.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:13:56.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:13:56.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:13:56.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:13:56.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:13:56.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:13:56.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:13:56.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:13:56.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:13:56.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:13:56.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:13:56.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.666 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.666 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:13:56.666 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:13:56.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.666 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.666 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:13:56.666 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:13:56.666 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.666 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.666 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.667 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.667 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:56.667 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.072382172999994, 2024-09-13T09:13:56.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:13:56.667 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.667 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.667 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.667 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.667 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.667 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:13:56.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.667 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:56.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:56.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:13:56.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:13:56.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:13:56.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:13:56.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:13:56.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:13:56.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:13:56.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:13:56.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:13:56.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:13:56.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:13:56.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:13:56.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:13:56.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:13:56.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:13:56.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:13:56.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:13:56.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:13:56.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:13:56.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:13:56.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:13:56.672 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.672 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.672 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.673 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.673 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:56.673 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.063255325, 2024-09-13T09:13:56.673 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:13:56.673 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.673 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.673 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.673 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.673 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.673 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:13:56.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.673 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:56.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:56.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:13:56.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:13:56.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:13:56.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:13:56.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:13:56.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:13:56.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:13:56.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:13:56.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:13:56.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:13:56.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:13:56.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:13:56.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:13:56.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:13:56.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:13:56.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:13:56.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:13:56.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:13:56.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:13:56.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:13:56.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:13:56.679 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.679 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.679 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.679 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.679 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:56.679 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.063203744999996, 2024-09-13T09:13:56.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:13:56.679 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.679 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.680 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:13:56.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.680 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:56.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:13:56.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:13:56.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:13:56.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:13:56.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:13:56.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:13:56.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:13:56.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:13:56.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:13:56.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:13:56.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:13:56.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:13:56.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:13:56.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:13:56.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:13:56.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:13:56.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:13:56.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:13:56.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:13:56.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:13:56.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:13:56.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:13:56.684 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.685 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.685 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.685 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.685 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:56.685 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.063187468999999, 2024-09-13T09:13:56.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:13:56.685 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.685 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.685 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.685 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.685 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.686 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:13:56.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.686 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.686 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.686 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:56.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.686 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.686 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:13:56.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:13:56.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:13:56.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:13:56.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:13:56.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:13:56.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:13:56.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:13:56.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:13:56.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:13:56.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:13:56.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:13:56.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:13:56.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:13:56.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:13:56.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:13:56.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:13:56.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:13:56.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:13:56.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:13:56.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:13:56.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:13:56.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.690 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.691 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.691 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.691 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:56.691 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.063171694999994, 2024-09-13T09:13:56.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:13:56.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.691 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.692 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:13:56.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.692 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:56.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:13:56.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:13:56.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:13:56.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:13:56.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:13:56.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:13:56.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:13:56.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:13:56.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:13:56.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:13:56.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:13:56.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:13:56.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:13:56.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:13:56.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:13:56.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:13:56.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:13:56.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:13:56.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:13:56.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:13:56.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:13:56.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:13:56.697 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.697 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.697 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.697 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.697 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:56.697 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.063156820000003, 2024-09-13T09:13:56.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:13:56.697 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.697 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.697 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.697 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.698 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.698 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:13:56.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.698 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:56.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:56.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:13:56.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:13:56.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:13:56.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:13:56.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:13:56.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:13:56.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:13:56.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:13:56.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:13:56.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:13:56.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:13:56.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:13:56.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:13:56.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:13:56.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:13:56.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:13:56.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:13:56.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:13:56.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:13:56.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:13:56.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:13:56.703 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.703 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.703 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.703 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.703 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:56.704 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.063139358000001, 2024-09-13T09:13:56.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:13:56.704 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.704 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.704 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.704 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.704 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.704 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:13:56.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.704 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:56.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:13:56.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:13:56.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:13:56.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:13:56.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:13:56.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:13:56.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:13:56.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:13:56.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:13:56.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:13:56.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:13:56.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:13:56.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:13:56.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:13:56.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:13:56.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:13:56.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:13:56.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:13:56.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:13:56.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:13:56.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:13:56.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:13:56.709 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.709 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.709 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.710 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.710 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:56.710 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.061538725999995, 2024-09-13T09:13:56.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:13:56.710 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.710 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.710 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.710 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.710 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.710 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:13:56.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.710 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:56.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:13:56.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:13:56.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:13:56.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:13:56.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:13:56.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:13:56.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:13:56.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:13:56.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:13:56.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:13:56.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:13:56.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:13:56.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:13:56.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:13:56.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:13:56.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:13:56.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:13:56.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:13:56.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.718 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:13:56.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:13:56.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.718 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.718 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:13:56.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:13:56.718 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.718 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.718 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.718 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.719 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:56.719 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.061498674999996, 2024-09-13T09:13:56.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:13:56.719 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.719 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.719 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.719 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.719 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.719 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:13:56.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.719 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:56.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:13:56.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:13:56.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.721 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:13:56.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:13:56.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.721 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:13:56.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:13:56.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.721 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:13:56.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:13:56.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:13:56.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:13:56.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:13:56.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:13:56.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:13:56.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:13:56.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:13:56.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:13:56.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:13:56.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:13:56.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:13:56.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:13:56.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:13:56.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:13:56.724 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.724 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.724 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.725 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:56.725 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:56.725 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.061454912000002, 2024-09-13T09:13:56.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:13:56.725 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:56.725 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:56.725 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:56.725 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:56.725 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:56.725 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:13:56.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.725 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:56.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:56.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:56.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:56.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:56.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:56.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:56.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:13:56.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:56.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:56.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:13:56.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:13:56.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:56.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:13:56.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:13:56.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:56.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:13:56.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:13:56.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:13:56.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:13:56.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:13:56.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:13:56.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:13:56.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:13:56.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:56.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:13:56.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:13:56.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:56.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:13:56.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:13:56.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:56.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:13:56.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:13:56.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:56.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:56.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:56.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:13:56.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:13:56.730 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.730 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.731 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.731 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:56.731 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:56.731 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:56.733 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:56.734 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-09-13T09:13:56.734 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:56.731+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:56.834 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:56.832+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:56.880 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:56.880 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:13:56.880 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:56.889 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:56.934 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:56.933+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:56.999 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:56.999 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:13:57.000 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:13:57.000 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:13:57.000 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:57.008 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:57.035 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:57.034+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:57.116 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:13:57.116 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:13:57.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:13:57.116 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:13:57.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.117 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.117 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:57.117 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610807436000002, 2024-09-13T09:13:57.117 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:13:57.117 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.117 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.117 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:13:57.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.117 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:57.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:13:57.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:13:57.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:13:57.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:13:57.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:13:57.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:13:57.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:13:57.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:13:57.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:13:57.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:13:57.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:13:57.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:13:57.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:13:57.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:13:57.124 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.124 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.124 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.124 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.125 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:57.125 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610779751999999, 2024-09-13T09:13:57.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:13:57.125 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.125 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.125 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.125 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.125 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.125 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:13:57.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.125 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:57.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:57.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:13:57.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:13:57.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:13:57.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:13:57.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:13:57.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:13:57.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:13:57.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:13:57.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:13:57.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:13:57.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:13:57.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:13:57.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:13:57.129 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.129 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.129 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.129 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.129 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:57.129 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610755411, 2024-09-13T09:13:57.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:13:57.129 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.129 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.129 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.130 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.130 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.130 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:13:57.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.130 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.130 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:57.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.130 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:13:57.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:13:57.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:13:57.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:13:57.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:13:57.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:13:57.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:13:57.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:13:57.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:13:57.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:13:57.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:13:57.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:13:57.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:13:57.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:13:57.133 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.133 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.133 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.133 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.133 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:57.134 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610739090999999, 2024-09-13T09:13:57.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:13:57.134 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.134 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.134 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.134 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.134 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.134 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:13:57.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.134 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.134 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:57.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.135 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:57.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.135 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:13:57.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:13:57.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:13:57.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:13:57.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:13:57.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:13:57.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:13:57.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:13:57.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:13:57.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:13:57.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:13:57.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:13:57.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:13:57.138 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.138 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.138 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.138 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.138 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:57.138 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610722675999995, 2024-09-13T09:13:57.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:13:57.138 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.138 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.138 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.138 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.138 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.139 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:13:57.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.139 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:57.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:57.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:13:57.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:13:57.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:13:57.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:13:57.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:13:57.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.141 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.141 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:13:57.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:13:57.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.141 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.141 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:13:57.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:13:57.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.141 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:13:57.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:13:57.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:13:57.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:13:57.142 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.142 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.142 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.142 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.143 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:57.143 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610706567999998, 2024-09-13T09:13:57.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:13:57.143 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.143 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.143 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.143 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.143 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.143 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:13:57.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.143 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:57.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:57.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:13:57.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:13:57.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:13:57.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:13:57.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:13:57.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:13:57.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:13:57.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:13:57.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:13:57.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.147 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:13:57.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:13:57.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.147 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.147 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:13:57.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:13:57.147 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.147 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.147 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.147 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.147 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:57.148 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610691007, 2024-09-13T09:13:57.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:13:57.148 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.148 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.148 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.148 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.148 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.148 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:13:57.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.148 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:57.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:57.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:13:57.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:13:57.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:13:57.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:13:57.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:13:57.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:13:57.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:13:57.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:13:57.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:13:57.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:13:57.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:13:57.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:13:57.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:13:57.151 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.152 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.152 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.152 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.152 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:57.152 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610674169000006, 2024-09-13T09:13:57.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:13:57.152 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.152 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.152 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.152 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.152 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.153 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:13:57.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.153 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:57.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:13:57.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:13:57.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:13:57.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:13:57.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:13:57.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:13:57.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:13:57.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:13:57.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:13:57.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:13:57.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:13:57.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:13:57.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:13:57.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:13:57.156 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.156 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.156 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.156 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.156 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:57.156 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610658177000005, 2024-09-13T09:13:57.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:13:57.156 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.157 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.157 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.157 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.157 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.157 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:13:57.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.157 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:57.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:13:57.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:13:57.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:13:57.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:13:57.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:13:57.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:13:57.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:13:57.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:13:57.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:13:57.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:13:57.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:13:57.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:13:57.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:13:57.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:13:57.160 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.160 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.160 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.161 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.161 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:57.161 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610630678999996, 2024-09-13T09:13:57.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:13:57.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.161 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.161 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.161 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.161 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.161 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:13:57.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.161 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:57.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:57.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:13:57.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:13:57.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:13:57.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:13:57.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:13:57.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:13:57.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:13:57.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:13:57.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:13:57.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:13:57.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:13:57.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:13:57.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:13:57.165 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.165 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.165 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.165 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.165 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:57.165 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610604820000006, 2024-09-13T09:13:57.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:13:57.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.165 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.165 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.166 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.166 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.166 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:13:57.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.166 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:57.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:57.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:13:57.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:13:57.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:13:57.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:13:57.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:13:57.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:13:57.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:13:57.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:13:57.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:13:57.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:13:57.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:13:57.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:13:57.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:13:57.169 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.169 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.169 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.169 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.170 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:57.170 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610563330000005, 2024-09-13T09:13:57.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:13:57.170 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.170 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.170 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.170 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.170 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.170 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:13:57.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.170 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:57.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:13:57.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:13:57.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:13:57.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:13:57.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:13:57.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:13:57.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:13:57.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:13:57.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:13:57.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:13:57.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:13:57.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:13:57.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:13:57.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:13:57.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.174 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.174 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.174 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:57.174 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610542092000003, 2024-09-13T09:13:57.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:13:57.174 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.175 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.175 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.175 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.175 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.175 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:13:57.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.175 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:57.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:57.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:13:57.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:13:57.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:13:57.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:13:57.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:13:57.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:13:57.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:13:57.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:13:57.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:13:57.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:13:57.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:13:57.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:13:57.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:13:57.178 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.178 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.178 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.179 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.179 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:57.179 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610519292000006, 2024-09-13T09:13:57.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:13:57.179 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.179 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.179 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.179 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.179 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.179 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:13:57.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.179 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:57.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:13:57.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:13:57.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.181 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.181 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:13:57.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:13:57.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.181 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.181 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:13:57.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:57.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.181 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.181 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:13:57.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:13:57.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:13:57.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:13:57.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:13:57.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:13:57.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:13:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:13:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610502835999995, 2024-09-13T09:13:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:13:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.183 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.184 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:13:57.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.184 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:57.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:57.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:13:57.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:13:57.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:13:57.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:13:57.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:13:57.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:13:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:13:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:13:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:13:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:13:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:13:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.610257699000002, 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.187 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.188 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.188 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:13:57.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.188 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:57.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:13:57.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:13:57.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:13:57.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:13:57.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:13:57.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:13:57.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:13:57.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:13:57.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:13:57.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:13:57.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:13:57.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:13:57.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:13:57.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:13:57.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:13:57.192 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.192 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.192 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.192 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.192 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:57.192 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.60837214, 2024-09-13T09:13:57.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:13:57.192 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.192 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.192 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.193 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.193 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.193 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:13:57.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.193 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:57.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:13:57.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:13:57.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:13:57.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:13:57.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:13:57.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:13:57.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:13:57.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:13:57.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:13:57.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:13:57.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:13:57.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:13:57.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:13:57.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:13:57.196 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.197 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.197 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.197 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.197 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:57.197 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.608350774000002, 2024-09-13T09:13:57.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:13:57.197 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.197 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.197 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.197 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.197 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.197 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:13:57.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.198 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:57.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:57.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:13:57.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:13:57.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:13:57.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:13:57.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:13:57.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:13:57.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:13:57.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:13:57.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:13:57.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:13:57.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:13:57.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:13:57.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:13:57.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:13:57.201 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.201 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.201 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.201 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:13:57.201 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:57.201 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 74.608145406000006, 2024-09-13T09:13:57.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:13:57.202 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.202 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:13:57.202 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.202 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:13:57.202 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:13:57.202 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:13:57.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.202 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:57.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:13:57.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:13:57.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:13:57.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:13:57.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:13:57.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:13:57.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:13:57.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:13:57.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:13:57.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:13:57.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:13:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:13:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:13:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:13:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:13:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.206 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5272.0:1775 3.0 3:02a42fca:::rbd_data.11235782ee8d.0000000000000030:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2244608~40448] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:13:57.206 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:57.206 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 46.866603564999998, 2024-09-13T09:13:57.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.666843732, 2024-09-13T09:13:57.206 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:13:57.206 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:13:57.206 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:13:57.206 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5272", 2024-09-13T09:13:57.206 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4089902097", 2024-09-13T09:13:57.206 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1775 2024-09-13T09:13:57.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.206 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:13:57.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:13:57.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:57.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.207 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:13:57.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:13:57.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:13:57.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.207 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:13:57.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247780+0000", 2024-09-13T09:13:57.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:13:57.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:13:57.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247865+0000", 2024-09-13T09:13:57.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4359999999999999e-05 2024-09-13T09:13:57.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:13:57.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247876+0000", 2024-09-13T09:13:57.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1456e-05 2024-09-13T09:13:57.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:13:57.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247883+0000", 2024-09-13T09:13:57.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:13:57.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247926+0000", 2024-09-13T09:13:57.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3203e-05 2024-09-13T09:13:57.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:13:57.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247932+0000", 2024-09-13T09:13:57.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:13:57.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909639+0000", 2024-09-13T09:13:57.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6617069099999999 2024-09-13T09:13:57.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:13:57.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909828+0000", 2024-09-13T09:13:57.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000189075 2024-09-13T09:13:57.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:13:57.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911676+0000", 2024-09-13T09:13:57.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001847235 2024-09-13T09:13:57.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:13:57.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911765+0000", 2024-09-13T09:13:57.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9530000000000005e-05 2024-09-13T09:13:57.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.211 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:13:57.211 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911901+0000", 2024-09-13T09:13:57.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013603899999999999 2024-09-13T09:13:57.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.211 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:13:57.211 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.912889+0000", 2024-09-13T09:13:57.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098807400000000003 2024-09-13T09:13:57.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.211 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:13:57.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914438+0000", 2024-09-13T09:13:57.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015491070000000001 2024-09-13T09:13:57.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.212 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:13:57.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914505+0000", 2024-09-13T09:13:57.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:13:57.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:13:57.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:13:57.212 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:13:57.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914625+0000", 2024-09-13T09:13:57.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120108 2024-09-13T09:13:57.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.213 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.213 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.213 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:13:57.213 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:13:57.213 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:13:57.214 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:57.215 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:57.213+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:57.315 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:57.314+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:57.319 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:57.319 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:57.319 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:57.320 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:57.329 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:57.338 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:13:57.418 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:57.416+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:57.461 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:57.461 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:57.471 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:57.471 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:57.471 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:57.472 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:57.472 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:57.478 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:13:57.480 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:57.495 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:57.495 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:57.514 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:57.514 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:57.521 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:57.519+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:57.532 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:57.533 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:57.534 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:13:57.534 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:13:57.620 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:57.618+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:57.622 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:57.622 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:57.631 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:57.720 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:57.719+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:57.781 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:57.781 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:57.791 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:57.835 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:57.834+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:57.894 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:57.895 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:57.903 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:57.945 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:57.942+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:58.024 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:58.023+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:58.046 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:58.046 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:58.055 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:13:58.124 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:58.123+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:58.157 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:58.158 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:58.166 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:13:58.225 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:58.224+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:58.309 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:58.309 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:58.317 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:13:58.361 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:13:58.359+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:13:58.420 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:58.420 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:58.427 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:58.425+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:58.428 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:13:58.528 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:58.527+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:58.543 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:58.543 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:58.552 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:13:58.636 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:58.635+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:58.706 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:58.707 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:58.716 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:13:58.734 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:13:58.749 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5] 2024-09-13T09:13:58.749 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:13:58.749 INFO:tasks.thrashosds.thrasher:Reweighting osd 1 to 0.8604602816730518 2024-09-13T09:13:58.750 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 0.8604602816730518 2024-09-13T09:13:58.761 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:58.756+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:58.829 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:58.828+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:58.871 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:58.871 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:58.882 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:13:58.930 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:58.929+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:58.990 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:58.990 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:58.998 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:13:59.032 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:59.031+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:59.132 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:59.131+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:59.144 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:59.145 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:59.153 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:13:59.234 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:13:59.233+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:13:59.259 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:59.259 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:59.268 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:13:59.334 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:59.334+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:59.372 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:59.372 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:59.381 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:13:59.436 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:59.434+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:59.483 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:59.484 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:59.492 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:13:59.535 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:13:59.577 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:13:59.575+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:13:59.635 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:13:59.636 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:13:59.638 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:13:59.636+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:13:59.644 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:13:59.659 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:13:59.659 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:13:59.663 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.1 to 0.86046 (dc47) 2024-09-13T09:13:59.664 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-09-13T09:13:59.680 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:13:59.681 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:13:59.681 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:13:59.700 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:13:59.700 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:13:59.717 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:13:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:13:59.733 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:13:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:13:59.736 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:13:59.736 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:13:59.778 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:59.777+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:59.792 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:59.792 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:59.792 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:13:59.792 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:59.801 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:13:59.839 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:59.838+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:13:59.904 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:13:59.904 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:13:59.904 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:13:59.905 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:13:59.905 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:13:59.913 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:13:59.940 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:13:59.939+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:00.017 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:00.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2185 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489524301000003, 2024-09-13T09:14:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063348994000000006, 2024-09-13T09:14:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2185 2024-09-13T09:14:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.018 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:00.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525765+0000", 2024-09-13T09:14:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2390000000000001e-06 2024-09-13T09:14:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525772+0000", 2024-09-13T09:14:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:14:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525777+0000", 2024-09-13T09:14:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0799999999999996e-06 2024-09-13T09:14:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589020+0000", 2024-09-13T09:14:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063242501000000007 2024-09-13T09:14:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589063+0000", 2024-09-13T09:14:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2673999999999998e-05 2024-09-13T09:14:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589112+0000", 2024-09-13T09:14:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9286000000000002e-05 2024-09-13T09:14:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2187 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489504754000002, 2024-09-13T09:14:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063500102000000003, 2024-09-13T09:14:00.022 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2187 2024-09-13T09:14:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525784+0000", 2024-09-13T09:14:00.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1629999999999999e-06 2024-09-13T09:14:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525792+0000", 2024-09-13T09:14:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:14:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525796+0000", 2024-09-13T09:14:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7389999999999999e-06 2024-09-13T09:14:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589132+0000", 2024-09-13T09:14:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063336119999999996 2024-09-13T09:14:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589187+0000", 2024-09-13T09:14:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4126000000000003e-05 2024-09-13T09:14:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589283+0000", 2024-09-13T09:14:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6032999999999993e-05 2024-09-13T09:14:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.026 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2188 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489486708000001, 2024-09-13T09:14:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063724617999999997, 2024-09-13T09:14:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2188 2024-09-13T09:14:00.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525802+0000", 2024-09-13T09:14:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0439999999999998e-06 2024-09-13T09:14:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525809+0000", 2024-09-13T09:14:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.238e-06 2024-09-13T09:14:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525814+0000", 2024-09-13T09:14:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-13T09:14:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589327+0000", 2024-09-13T09:14:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063513257000000004 2024-09-13T09:14:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589418+0000", 2024-09-13T09:14:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0796000000000003e-05 2024-09-13T09:14:00.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589525+0000", 2024-09-13T09:14:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010693500000000001 2024-09-13T09:14:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2190 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489468621999997, 2024-09-13T09:14:00.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063863981, 2024-09-13T09:14:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2190 2024-09-13T09:14:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525818+0000", 2024-09-13T09:14:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525820+0000", 2024-09-13T09:14:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:14:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525827+0000", 2024-09-13T09:14:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0589999999999999e-06 2024-09-13T09:14:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525831+0000", 2024-09-13T09:14:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7870000000000001e-06 2024-09-13T09:14:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589562+0000", 2024-09-13T09:14:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063731038000000004 2024-09-13T09:14:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589612+0000", 2024-09-13T09:14:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0087999999999999e-05 2024-09-13T09:14:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589683+0000", 2024-09-13T09:14:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0121000000000002e-05 2024-09-13T09:14:00.035 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2192 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489451662, 2024-09-13T09:14:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064050299000000005, 2024-09-13T09:14:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2192 2024-09-13T09:14:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525835+0000", 2024-09-13T09:14:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525837+0000", 2024-09-13T09:14:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:14:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525845+0000", 2024-09-13T09:14:00.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3660000000000003e-06 2024-09-13T09:14:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525850+0000", 2024-09-13T09:14:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0030000000000002e-06 2024-09-13T09:14:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589720+0000", 2024-09-13T09:14:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063870176000000001 2024-09-13T09:14:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589801+0000", 2024-09-13T09:14:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0889000000000004e-05 2024-09-13T09:14:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589886+0000", 2024-09-13T09:14:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5183000000000001e-05 2024-09-13T09:14:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.040 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489433558999998, 2024-09-13T09:14:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:14:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:14:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:00.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:14:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:14:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:14:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:14:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:14:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:14:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:14:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:14:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:14:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:14:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:14:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:14:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:14:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489415860999998, 2024-09-13T09:14:00.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:14:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:14:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:14:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:14:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:14:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:14:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:14:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:14:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:14:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:14:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:14:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:14:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:14:00.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:14:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489395246000001, 2024-09-13T09:14:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:14:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.050 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:14:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:00.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:00.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:14:00.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:14:00.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:14:00.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:14:00.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:14:00.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:14:00.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:14:00.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:14:00.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:14:00.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:14:00.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:14:00.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:14:00.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.054 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.055 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.055 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:00.055 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489376290000003, 2024-09-13T09:14:00.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:14:00.055 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.055 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.055 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:14:00.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.055 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:00.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:00.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:00.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:14:00.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:14:00.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:14:00.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:14:00.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:14:00.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:14:00.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:14:00.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:14:00.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:14:00.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:14:00.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:14:00.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:14:00.059 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.059 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.059 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.059 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.059 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:00.060 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489359032000003, 2024-09-13T09:14:00.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:14:00.060 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.060 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.060 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.060 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.060 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.060 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:14:00.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.060 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:00.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:00.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:00.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:14:00.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:14:00.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:14:00.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:14:00.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:14:00.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:14:00.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:14:00.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:14:00.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:14:00.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:14:00.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:14:00.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:14:00.064 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.064 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.064 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.064 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.064 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:00.064 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489339018000003, 2024-09-13T09:14:00.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:14:00.064 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.064 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.065 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.065 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.065 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.065 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:14:00.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.065 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:00.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:00.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:00.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:14:00.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:14:00.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:14:00.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:14:00.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:14:00.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:14:00.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:14:00.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:14:00.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:14:00.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:14:00.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:14:00.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:14:00.068 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.068 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.068 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.069 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.069 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:00.069 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489313436000003, 2024-09-13T09:14:00.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:14:00.069 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.069 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.069 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:14:00.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.069 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:00.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:00.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:14:00.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:14:00.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:14:00.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:14:00.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:14:00.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:14:00.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:14:00.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:14:00.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:14:00.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:14:00.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:14:00.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:14:00.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:14:00.073 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.073 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.073 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.074 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.074 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:00.074 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489294151999999, 2024-09-13T09:14:00.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:14:00.074 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.074 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.074 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.074 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.074 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.074 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:14:00.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.074 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:00.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:00.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:00.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:14:00.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:14:00.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:14:00.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:14:00.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:14:00.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:14:00.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:14:00.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:14:00.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:14:00.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:14:00.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:14:00.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:14:00.078 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.078 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.078 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.078 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.078 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:00.078 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489276896, 2024-09-13T09:14:00.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:14:00.078 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.078 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.079 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:14:00.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.079 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:00.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:00.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:00.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:14:00.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:14:00.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:14:00.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:14:00.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:14:00.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:14:00.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:14:00.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:14:00.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:14:00.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:14:00.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:14:00.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:14:00.082 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.082 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.082 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.083 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.083 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:00.083 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489257416000001, 2024-09-13T09:14:00.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:14:00.083 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.083 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.083 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.083 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.083 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.083 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:14:00.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.083 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:00.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:00.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:14:00.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:14:00.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:14:00.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:14:00.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:14:00.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:14:00.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:14:00.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:14:00.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:14:00.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:14:00.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:14:00.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:14:00.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:14:00.087 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.087 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.087 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.087 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.087 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:00.087 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489240487000004, 2024-09-13T09:14:00.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:14:00.088 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.088 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.088 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.088 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.088 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.088 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:14:00.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.088 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:00.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:00.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:14:00.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:14:00.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:14:00.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:14:00.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:14:00.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:14:00.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:14:00.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:14:00.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:14:00.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:14:00.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:14:00.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:14:00.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:14:00.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.091 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489221338, 2024-09-13T09:14:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:14:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:14:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.092 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:00.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:14:00.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:14:00.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:14:00.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:14:00.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:14:00.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:14:00.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:14:00.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:14:00.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:14:00.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:14:00.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:14:00.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:14:00.096 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.096 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.096 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.096 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.096 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:00.096 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.489203199000002, 2024-09-13T09:14:00.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:14:00.096 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.097 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.097 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.097 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.097 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.097 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:14:00.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.097 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:00.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:00.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:00.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:14:00.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:14:00.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:14:00.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:14:00.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:14:00.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:14:00.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:14:00.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:14:00.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:14:00.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:14:00.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:14:00.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:14:00.100 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.100 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.100 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.101 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.101 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:00.101 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.488792703999998, 2024-09-13T09:14:00.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:14:00.101 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.101 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.101 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.101 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.101 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.101 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:14:00.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.102 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:00.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:00.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:00.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:14:00.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:14:00.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:14:00.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:14:00.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:14:00.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:14:00.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:14:00.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:14:00.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:14:00.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:14:00.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:14:00.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:14:00.105 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.105 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.105 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.105 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:00.105 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:00.105 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.488769802999997, 2024-09-13T09:14:00.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:14:00.106 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.106 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.106 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.106 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:00.106 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:00.106 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:14:00.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.106 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:00.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:00.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:00.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:14:00.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:14:00.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:14:00.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:14:00.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:14:00.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:14:00.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:14:00.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:14:00.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:14:00.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:14:00.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:14:00.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:14:00.109 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.109 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.110 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.110 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.110 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.110 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:00.111 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:14:00.112 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:00.110+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:00.212 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:00.211+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:00.215 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:00.215 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:00.215 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:00.215 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:00.224 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:14:00.313 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:00.311+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:00.325 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:00.325 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:00.326 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:00.326 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:00.326 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:00.334 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:14:00.418 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:00.417+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:00.479 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:00.479 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:00.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:00.479 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:00.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.479 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:00.479 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:00.479 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.73128243, 2024-09-13T09:14:00.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:14:00.479 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.479 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:00.480 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:00.480 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:14:00.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.480 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:00.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:00.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:00.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:14:00.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:14:00.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:14:00.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:14:00.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:14:00.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:14:00.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:14:00.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:14:00.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:14:00.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:14:00.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:14:00.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:14:00.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.483 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.484 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:00.484 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:00.484 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.731273752, 2024-09-13T09:14:00.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:14:00.484 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.484 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:00.485 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:00.485 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:14:00.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.485 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:00.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:00.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:00.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:00.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:14:00.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:14:00.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:14:00.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:14:00.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:14:00.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:14:00.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:14:00.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:14:00.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:14:00.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:14:00.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:14:00.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:14:00.488 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.488 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.488 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.488 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:00.488 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:00.488 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.731264766000002, 2024-09-13T09:14:00.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:14:00.489 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.489 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.489 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.489 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:00.489 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:00.489 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:14:00.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.489 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:00.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:00.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:00.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:14:00.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:14:00.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:14:00.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:14:00.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:14:00.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:14:00.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:14:00.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:14:00.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:14:00.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:14:00.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:14:00.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:14:00.492 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.492 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.492 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.493 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:00.493 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:00.493 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.731256793999997, 2024-09-13T09:14:00.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:14:00.493 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.493 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.493 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.493 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:00.493 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:00.493 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:14:00.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.493 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:00.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:00.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:00.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:00.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:14:00.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:14:00.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:14:00.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:14:00.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:14:00.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:14:00.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:14:00.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:14:00.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:14:00.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:14:00.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:14:00.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:14:00.497 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.497 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.497 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.497 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:00.497 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:00.497 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.729381959999998, 2024-09-13T09:14:00.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:14:00.497 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.497 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.497 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.497 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:00.498 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:00.498 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:14:00.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.498 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:00.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:00.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:00.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:14:00.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:14:00.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:14:00.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:14:00.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:14:00.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:14:00.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:14:00.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:14:00.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:14:00.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:14:00.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:14:00.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:14:00.501 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.501 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.501 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.501 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:00.501 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:00.502 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.729371299, 2024-09-13T09:14:00.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:14:00.502 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.502 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.502 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.502 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:00.502 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:00.502 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:14:00.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.502 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:00.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:00.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:00.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:00.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:14:00.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:14:00.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:14:00.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:14:00.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:14:00.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:14:00.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:14:00.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:14:00.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:14:00.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:14:00.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:14:00.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:14:00.506 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.506 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.506 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.506 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:00.506 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:00.506 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.729349323000001, 2024-09-13T09:14:00.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:14:00.506 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.506 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.507 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.507 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:00.507 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:00.507 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:14:00.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.507 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:00.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:00.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:00.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:14:00.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:14:00.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:14:00.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:14:00.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:14:00.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:14:00.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:14:00.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:14:00.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:14:00.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:14:00.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:14:00.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:14:00.510 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.510 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.510 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.511 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:00.511 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:00.511 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.729335820999999, 2024-09-13T09:14:00.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:14:00.511 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.511 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.511 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.511 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:00.511 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:00.511 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:14:00.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.512 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:00.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:00.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:00.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:14:00.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:14:00.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:14:00.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:14:00.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:14:00.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:14:00.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:14:00.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:14:00.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:14:00.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:14:00.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:14:00.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:14:00.516 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.516 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.516 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.516 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:00.516 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:00.517 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.727326722999997, 2024-09-13T09:14:00.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:14:00.517 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.517 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.517 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.517 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:00.517 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:00.517 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:14:00.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.517 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:00.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:00.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:00.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:14:00.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:14:00.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:14:00.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:14:00.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:14:00.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:14:00.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:14:00.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:14:00.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:14:00.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:14:00.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:14:00.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:14:00.520 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.520 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.521 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.521 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:00.521 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:00.521 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.727302401000003, 2024-09-13T09:14:00.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:14:00.521 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.521 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.521 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.521 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:00.521 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:00.521 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:14:00.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.522 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:00.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:00.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:00.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:14:00.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:14:00.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:14:00.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:14:00.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:14:00.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:14:00.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:14:00.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:14:00.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:14:00.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:14:00.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:14:00.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:14:00.525 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.525 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.525 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.525 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:00.525 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:14:00.525 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.727291426999997, 2024-09-13T09:14:00.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:14:00.526 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.526 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:00.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:00.526 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:14:00.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.526 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:14:00.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:14:00.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:14:00.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:14:00.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:14:00.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:14:00.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:14:00.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:14:00.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:14:00.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:14:00.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:14:00.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:14:00.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:14:00.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:14:00.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:14:00.529 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.529 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.529 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.530 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:00.530 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:00.530 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.725373802, 2024-09-13T09:14:00.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:14:00.530 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.530 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:00.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:00.530 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:14:00.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.530 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:14:00.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:14:00.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:14:00.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:14:00.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:14:00.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:14:00.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:14:00.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:14:00.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:14:00.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:14:00.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:14:00.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:14:00.534 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.534 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.534 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.534 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:00.534 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:00.534 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.725364405000001, 2024-09-13T09:14:00.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:14:00.534 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.534 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.534 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.535 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:00.535 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:00.535 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:14:00.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.535 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:00.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:00.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:00.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:00.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:14:00.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:14:00.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:14:00.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:14:00.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:14:00.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:14:00.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:14:00.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:14:00.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:14:00.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:14:00.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:14:00.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:14:00.538 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.538 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.538 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.538 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3181 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T09:14:00.538 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:14:00.539 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 30.455000228999999, 2024-09-13T09:14:00.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.480957603, 2024-09-13T09:14:00.539 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.539 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:00.539 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.539 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:14:00.539 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:14:00.539 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3181 2024-09-13T09:14:00.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.539 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:14:00.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:14:00.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021894+0000", 2024-09-13T09:14:00.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:14:00.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021903+0000", 2024-09-13T09:14:00.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4109999999999996e-06 2024-09-13T09:14:00.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021919+0000", 2024-09-13T09:14:00.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6225000000000001e-05 2024-09-13T09:14:00.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021926+0000", 2024-09-13T09:14:00.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4120000000000004e-06 2024-09-13T09:14:00.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021975+0000", 2024-09-13T09:14:00.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8526000000000003e-05 2024-09-13T09:14:00.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022049+0000", 2024-09-13T09:14:00.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4018999999999997e-05 2024-09-13T09:14:00.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:14:00.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022230+0000", 2024-09-13T09:14:00.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018085599999999999 2024-09-13T09:14:00.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:00.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022796+0000", 2024-09-13T09:14:00.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056638999999999995 2024-09-13T09:14:00.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499540+0000", 2024-09-13T09:14:00.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.47674342800000002 2024-09-13T09:14:00.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:00.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499547+0000", 2024-09-13T09:14:00.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:14:00.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502794+0000", 2024-09-13T09:14:00.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.003246818 2024-09-13T09:14:00.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502854+0000", 2024-09-13T09:14:00.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9925e-05 2024-09-13T09:14:00.544 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.545 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.545 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.545 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3182 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 139818970156544 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:14:00.545 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:14:00.545 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.981765674000002, 2024-09-13T09:14:00.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0089058390000001, 2024-09-13T09:14:00.545 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.545 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:14:00.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:14:00.546 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3182 2024-09-13T09:14:00.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.546 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:14:00.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:14:00.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495128+0000", 2024-09-13T09:14:00.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:14:00.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495139+0000", 2024-09-13T09:14:00.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0599e-05 2024-09-13T09:14:00.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495157+0000", 2024-09-13T09:14:00.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7830999999999999e-05 2024-09-13T09:14:00.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495167+0000", 2024-09-13T09:14:00.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0689999999999999e-05 2024-09-13T09:14:00.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495229+0000", 2024-09-13T09:14:00.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1898999999999998e-05 2024-09-13T09:14:00.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:14:00.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495243+0000", 2024-09-13T09:14:00.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3349e-05 2024-09-13T09:14:00.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499575+0000", 2024-09-13T09:14:00.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043324410000000002 2024-09-13T09:14:00.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:00.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499582+0000", 2024-09-13T09:14:00.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0849999999999999e-06 2024-09-13T09:14:00.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502872+0000", 2024-09-13T09:14:00.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0032895150000001 2024-09-13T09:14:00.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503016+0000", 2024-09-13T09:14:00.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014394200000000001 2024-09-13T09:14:00.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504036+0000", 2024-09-13T09:14:00.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010208680000000001 2024-09-13T09:14:00.551 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.551 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.551 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.551 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:19151 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 140383089882784 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:14:00.551 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:14:00.551 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.981754214999999, 2024-09-13T09:14:00.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.009071287, 2024-09-13T09:14:00.551 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.551 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:00.551 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.551 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:00.552 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:00.552 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19151 2024-09-13T09:14:00.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.552 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:14:00.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:14:00.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495140+0000", 2024-09-13T09:14:00.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:14:00.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495149+0000", 2024-09-13T09:14:00.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0759999999999992e-06 2024-09-13T09:14:00.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495165+0000", 2024-09-13T09:14:00.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5322e-05 2024-09-13T09:14:00.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495173+0000", 2024-09-13T09:14:00.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0979999999999994e-06 2024-09-13T09:14:00.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495318+0000", 2024-09-13T09:14:00.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014553 2024-09-13T09:14:00.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:14:00.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495325+0000", 2024-09-13T09:14:00.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:14:00.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499598+0000", 2024-09-13T09:14:00.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004273064 2024-09-13T09:14:00.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:00.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499602+0000", 2024-09-13T09:14:00.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8539999999999999e-06 2024-09-13T09:14:00.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503507+0000", 2024-09-13T09:14:00.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0039045520000001 2024-09-13T09:14:00.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503583+0000", 2024-09-13T09:14:00.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6283000000000001e-05 2024-09-13T09:14:00.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504213+0000", 2024-09-13T09:14:00.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00063044700000000002 2024-09-13T09:14:00.557 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.557 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.557 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.557 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3908 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:00.557 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:14:00.557 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 12.384217324, 2024-09-13T09:14:00.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070530317999999995, 2024-09-13T09:14:00.557 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.557 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:00.557 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.557 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:00.557 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:00.558 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3908 2024-09-13T09:14:00.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.558 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:14:00.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:14:00.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092678+0000", 2024-09-13T09:14:00.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092681+0000", 2024-09-13T09:14:00.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.903e-06 2024-09-13T09:14:00.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092687+0000", 2024-09-13T09:14:00.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0859999999999998e-06 2024-09-13T09:14:00.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092692+0000", 2024-09-13T09:14:00.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-13T09:14:00.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.159711+0000", 2024-09-13T09:14:00.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067018922999999994 2024-09-13T09:14:00.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.159796+0000", 2024-09-13T09:14:00.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5247000000000006e-05 2024-09-13T09:14:00.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:00.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160111+0000", 2024-09-13T09:14:00.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00031449499999999998 2024-09-13T09:14:00.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:00.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162219+0000", 2024-09-13T09:14:00.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021085570000000001 2024-09-13T09:14:00.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163127+0000", 2024-09-13T09:14:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00090723400000000004 2024-09-13T09:14:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163172+0000", 2024-09-13T09:14:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5383000000000002e-05 2024-09-13T09:14:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163209+0000", 2024-09-13T09:14:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7132000000000003e-05 2024-09-13T09:14:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3909 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 12.384199833, 2024-09-13T09:14:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070616834000000003, 2024-09-13T09:14:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3909 2024-09-13T09:14:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092699+0000", 2024-09-13T09:14:00.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:14:00.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092704+0000", 2024-09-13T09:14:00.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5940000000000001e-06 2024-09-13T09:14:00.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092709+0000", 2024-09-13T09:14:00.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5109999999999997e-06 2024-09-13T09:14:00.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160604+0000", 2024-09-13T09:14:00.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067895633999999996 2024-09-13T09:14:00.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160658+0000", 2024-09-13T09:14:00.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3292999999999999e-05 2024-09-13T09:14:00.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:00.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160935+0000", 2024-09-13T09:14:00.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000276879 2024-09-13T09:14:00.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:00.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163223+0000", 2024-09-13T09:14:00.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022886320000000001 2024-09-13T09:14:00.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:00.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163258+0000", 2024-09-13T09:14:00.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5101000000000002e-05 2024-09-13T09:14:00.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:00.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163292+0000", 2024-09-13T09:14:00.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3237e-05 2024-09-13T09:14:00.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163313+0000", 2024-09-13T09:14:00.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1642e-05 2024-09-13T09:14:00.568 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.568 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.568 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.569 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3910 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:00.569 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:00.569 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 12.384182769000001, 2024-09-13T09:14:00.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733837999999993, 2024-09-13T09:14:00.569 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.569 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:00.569 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.569 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:00.569 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:00.569 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3910 2024-09-13T09:14:00.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.570 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:00.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:00.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:00.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092716+0000", 2024-09-13T09:14:00.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.757e-06 2024-09-13T09:14:00.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092722+0000", 2024-09-13T09:14:00.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7869999999999998e-06 2024-09-13T09:14:00.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092726+0000", 2024-09-13T09:14:00.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.622e-06 2024-09-13T09:14:00.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161487+0000", 2024-09-13T09:14:00.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068761166999999998 2024-09-13T09:14:00.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161569+0000", 2024-09-13T09:14:00.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1185000000000003e-05 2024-09-13T09:14:00.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:00.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161868+0000", 2024-09-13T09:14:00.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029979799999999999 2024-09-13T09:14:00.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:00.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163366+0000", 2024-09-13T09:14:00.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001497888 2024-09-13T09:14:00.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:00.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163373+0000", 2024-09-13T09:14:00.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1640000000000004e-06 2024-09-13T09:14:00.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:00.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163412+0000", 2024-09-13T09:14:00.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8423999999999997e-05 2024-09-13T09:14:00.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163447+0000", 2024-09-13T09:14:00.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5441000000000002e-05 2024-09-13T09:14:00.574 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.574 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.574 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.575 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3911 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:00.575 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:00.575 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 12.384162852999999, 2024-09-13T09:14:00.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.072284372999999999, 2024-09-13T09:14:00.575 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:00.575 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:00.575 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:00.575 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:00.575 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:00.575 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3911 2024-09-13T09:14:00.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.576 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:00.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:00.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:00.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:00.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:00.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:00.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:00.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:00.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:00.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092736+0000", 2024-09-13T09:14:00.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9510000000000002e-06 2024-09-13T09:14:00.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:00.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092743+0000", 2024-09-13T09:14:00.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9e-06 2024-09-13T09:14:00.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092748+0000", 2024-09-13T09:14:00.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999998e-06 2024-09-13T09:14:00.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:00.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162257+0000", 2024-09-13T09:14:00.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.069508890000000004 2024-09-13T09:14:00.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:00.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162342+0000", 2024-09-13T09:14:00.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5886999999999999e-05 2024-09-13T09:14:00.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:00.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162682+0000", 2024-09-13T09:14:00.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00033971700000000001 2024-09-13T09:14:00.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:00.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164897+0000", 2024-09-13T09:14:00.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022152370000000001 2024-09-13T09:14:00.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:00.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164945+0000", 2024-09-13T09:14:00.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7243999999999997e-05 2024-09-13T09:14:00.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:00.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164985+0000", 2024-09-13T09:14:00.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0599000000000002e-05 2024-09-13T09:14:00.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:00.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:00.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:00.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.165018+0000", 2024-09-13T09:14:00.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2549000000000003e-05 2024-09-13T09:14:00.580 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.580 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.580 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.580 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:00.581 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:00.581 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:00.582 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:14:00.583 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:00.581+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:00.683 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:00.681+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:00.707 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:00.707 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:00.707 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:00.707 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:00.717 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:14:00.784 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:00.783+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:00.826 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:00.826 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:00.826 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:00.826 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:00.826 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:00.834 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:14:00.885 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:00.884+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:00.984 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.453348719999994, 2024-09-13T09:14:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:14:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:00.985 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:14:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:00.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:14:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:14:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:14:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:14:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:14:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:14:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:14:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:14:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:14:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:14:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:14:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:14:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:14:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:14:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:14:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:14:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:14:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:14:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:14:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:14:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:14:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:00.990 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.453330684999997, 2024-09-13T09:14:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:14:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:14:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.991 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:00.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:14:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:14:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:14:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:14:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:14:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:14:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:14:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:14:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:14:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:14:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:14:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:14:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:14:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:14:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:14:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:14:00.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:14:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:14:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:14:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:14:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.453314015000004, 2024-09-13T09:14:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:14:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:14:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:14:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:00.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:14:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:14:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:14:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:14:00.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:14:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:14:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:14:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:14:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:14:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:14:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:14:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:14:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:14:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:14:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:14:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:14:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:14:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:14:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:14:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.002 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.453296382999994, 2024-09-13T09:14:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:14:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:14:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:14:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:14:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:14:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:14:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:14:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:14:01.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:14:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:14:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:14:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:14:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:14:01.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:14:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:14:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:14:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:14:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:14:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:14:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:14:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:14:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:14:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:01.008 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.453250077999996, 2024-09-13T09:14:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:14:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:14:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:14:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:14:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:14:01.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:14:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:14:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:14:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:14:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:14:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:14:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:14:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:14:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:14:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:14:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:14:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:14:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:14:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:14:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:14:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:14:01.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:14:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:14:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.453224777000003, 2024-09-13T09:14:01.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:14:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:14:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:14:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:14:01.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:14:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:14:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:14:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:14:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:14:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:14:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:14:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:14:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:14:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:14:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:14:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:14:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:14:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:14:01.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:14:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:14:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:14:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:14:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:14:01.024 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.453206547999997, 2024-09-13T09:14:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:14:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.025 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:14:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:01.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:14:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:14:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:14:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:14:01.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:14:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:14:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:14:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:14:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:14:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:14:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:14:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:14:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:14:01.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:14:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:14:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:14:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:14:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:14:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:14:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:14:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.032 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.453159228999994, 2024-09-13T09:14:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:14:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:14:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:01.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:14:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:14:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:14:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:14:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:14:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:14:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:14:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:14:01.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:14:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:14:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:14:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:14:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:14:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:14:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:14:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:14:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:14:01.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:14:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:14:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:14:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.039 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.452644991, 2024-09-13T09:14:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:14:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:14:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:14:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:14:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:14:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:14:01.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:14:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:14:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:14:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:14:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:14:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:14:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:14:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:14:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:14:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:14:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:14:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:14:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:14:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:14:01.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:14:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:14:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:14:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.046 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.452610461000006, 2024-09-13T09:14:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:14:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:14:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:14:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:14:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:14:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:14:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:14:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:14:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:14:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:14:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:14:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:14:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:14:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:14:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:14:01.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:14:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:14:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:14:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:14:01.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:14:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:14:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:14:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.053 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.452592554000006, 2024-09-13T09:14:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:14:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:14:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.054 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:14:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:14:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:14:01.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:14:01.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:14:01.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:14:01.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:14:01.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:14:01.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:14:01.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:14:01.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:14:01.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:14:01.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:14:01.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:14:01.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:14:01.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:14:01.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:14:01.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:14:01.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:14:01.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:14:01.060 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.060 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.060 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.060 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.060 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:01.061 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.443465705999998, 2024-09-13T09:14:01.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:14:01.061 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.061 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.061 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.061 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.061 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.061 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:14:01.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.061 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:01.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:01.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:01.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:14:01.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:14:01.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:14:01.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:14:01.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:14:01.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:14:01.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:14:01.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:14:01.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:14:01.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:14:01.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:14:01.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:14:01.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:14:01.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:14:01.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:14:01.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:14:01.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:14:01.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:14:01.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:14:01.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:14:01.067 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.067 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.067 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.068 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.068 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:01.068 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.443414125999993, 2024-09-13T09:14:01.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:14:01.068 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.068 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.068 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.068 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.068 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.068 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:14:01.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.069 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:01.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:01.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.070 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.070 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:14:01.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.070 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.070 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:14:01.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:14:01.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:14:01.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:14:01.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:14:01.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:14:01.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:14:01.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:14:01.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:14:01.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:14:01.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:14:01.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:14:01.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.073 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:14:01.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:14:01.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.073 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:14:01.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:14:01.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.074 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.074 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:14:01.074 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:14:01.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.074 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.074 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:14:01.074 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:14:01.074 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.074 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.074 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.075 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.075 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:01.075 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.443397849999997, 2024-09-13T09:14:01.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:14:01.075 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.075 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.075 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.076 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.076 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.076 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:14:01.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.076 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.076 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.076 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:01.076 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:01.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:14:01.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:14:01.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:14:01.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:14:01.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:14:01.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:14:01.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:14:01.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:14:01.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:14:01.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:14:01.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:14:01.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.080 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.080 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:14:01.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:14:01.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.080 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.080 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:14:01.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:14:01.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.080 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.080 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:14:01.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:14:01.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.081 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.081 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:14:01.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:14:01.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.081 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.081 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:14:01.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:14:01.082 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.082 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.082 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.082 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.082 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:01.082 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.443382076000006, 2024-09-13T09:14:01.082 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:14:01.082 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.082 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.082 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.083 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.083 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.083 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:14:01.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.083 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.083 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.083 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:01.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:01.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:14:01.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:14:01.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:14:01.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:14:01.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:14:01.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:14:01.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:14:01.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:14:01.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:14:01.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:14:01.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:14:01.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.087 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.087 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:14:01.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:14:01.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.087 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.087 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:14:01.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:14:01.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.087 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:14:01.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:14:01.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:14:01.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:14:01.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:14:01.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:14:01.089 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.089 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.089 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.089 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.089 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:01.089 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.443367201000001, 2024-09-13T09:14:01.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:14:01.089 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.089 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.090 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.090 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.090 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.090 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:14:01.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.090 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:01.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:01.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:01.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:14:01.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:14:01.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.092 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.092 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:14:01.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:14:01.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.092 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.092 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:14:01.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:14:01.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.093 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:14:01.093 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:14:01.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.093 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:14:01.093 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:14:01.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:14:01.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:14:01.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:14:01.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:14:01.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:14:01.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:14:01.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:14:01.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:14:01.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:14:01.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:14:01.096 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.096 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.096 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.096 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.096 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:01.096 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.443349738999999, 2024-09-13T09:14:01.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:14:01.096 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.096 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.096 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.097 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.097 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.097 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:14:01.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.097 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:01.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.098 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:01.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.098 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.098 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:14:01.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.098 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.098 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:14:01.099 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:14:01.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.099 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.099 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:14:01.099 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:14:01.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.099 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.099 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:14:01.099 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:14:01.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.100 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:14:01.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:14:01.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.100 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:14:01.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:14:01.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:14:01.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:14:01.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:14:01.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:14:01.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:14:01.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:14:01.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:14:01.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:14:01.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:14:01.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:14:01.102 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.102 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.103 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.103 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.103 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:01.103 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.441749107000007, 2024-09-13T09:14:01.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:14:01.103 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.103 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.103 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.103 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.103 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.104 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:14:01.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.104 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.104 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.104 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:01.104 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.104 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.104 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:01.104 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.105 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.105 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:14:01.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.105 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.105 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:14:01.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:14:01.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:14:01.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:14:01.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:14:01.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:14:01.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:14:01.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:14:01.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:14:01.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:14:01.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:14:01.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:14:01.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:14:01.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:14:01.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:14:01.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:14:01.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:14:01.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:14:01.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.109 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:14:01.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:14:01.109 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.109 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.109 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.110 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.110 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:01.110 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.441709055999993, 2024-09-13T09:14:01.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:14:01.110 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.110 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.110 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.110 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.110 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.110 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:14:01.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.110 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:01.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:01.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:14:01.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.112 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:14:01.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:14:01.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.112 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:14:01.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:14:01.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:14:01.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:14:01.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:14:01.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:14:01.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:14:01.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:14:01.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:14:01.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:14:01.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:14:01.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:14:01.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:14:01.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:14:01.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:14:01.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:14:01.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.116 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.116 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:14:01.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:14:01.116 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.116 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.116 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.116 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.116 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:01.117 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.441665293, 2024-09-13T09:14:01.117 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:14:01.117 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.117 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.117 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:14:01.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.117 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.117 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:01.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:01.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:01.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:14:01.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:14:01.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:14:01.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:14:01.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:14:01.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:14:01.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:14:01.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:14:01.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:14:01.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:14:01.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:14:01.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:14:01.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:14:01.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:14:01.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:14:01.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:14:01.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:14:01.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:14:01.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:14:01.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:14:01.123 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.123 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.123 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.123 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.123 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.123 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:01.125 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:14:01.125 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:01.124+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:01.225 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:01.224+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:01.272 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:01.272 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:01.273 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:01.273 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:01.281 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:14:01.326 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:01.325+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:01.387 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:01.387 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:01.387 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:01.387 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:01.387 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:01.395 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:14:01.427 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:01.426+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:01.528 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:01.526+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:01.545 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:01.545 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:01.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:01.545 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:01.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.545 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.545 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:01.545 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.039424535999999, 2024-09-13T09:14:01.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:14:01.545 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:14:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:14:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:14:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:14:01.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:14:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:14:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:14:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:14:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:14:01.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:14:01.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:14:01.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:14:01.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:14:01.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:14:01.549 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.549 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.549 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.550 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.550 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:01.550 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.039396851999996, 2024-09-13T09:14:01.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:14:01.550 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.550 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.550 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.550 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.550 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.550 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:14:01.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.551 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:01.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:01.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:01.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:14:01.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:14:01.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:14:01.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:14:01.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:14:01.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:14:01.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:14:01.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:14:01.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:14:01.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:14:01.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:14:01.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:14:01.554 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.554 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.554 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.555 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.555 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:01.555 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.039372510999996, 2024-09-13T09:14:01.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:14:01.555 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.555 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.555 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.555 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.555 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.556 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:14:01.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.556 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:01.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:01.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:14:01.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:14:01.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:14:01.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:14:01.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:14:01.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:14:01.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:14:01.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:14:01.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:14:01.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:14:01.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:14:01.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:14:01.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:14:01.559 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.559 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.559 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.559 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.559 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:01.559 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.039356190999996, 2024-09-13T09:14:01.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:14:01.560 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.560 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.560 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.560 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.560 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.560 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:14:01.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.560 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:01.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:01.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:01.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:14:01.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:01.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:14:01.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:14:01.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:14:01.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:14:01.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:14:01.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:14:01.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:14:01.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:14:01.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:14:01.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:14:01.564 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.564 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.564 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.564 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.039339776000006, 2024-09-13T09:14:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:14:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:14:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:14:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:14:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:14:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:14:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:14:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:14:01.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:14:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:14:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:14:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:14:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:14:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:14:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.039323667999994, 2024-09-13T09:14:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:14:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.569 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:14:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:01.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:14:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:14:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:14:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:14:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:14:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:14:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:14:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:14:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:14:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:14:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:14:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:14:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.574 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.574 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:01.574 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.039308106999997, 2024-09-13T09:14:01.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:14:01.574 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.574 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.574 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.574 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.574 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.574 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:14:01.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.575 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:01.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:14:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:14:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:14:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:14:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:14:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:14:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:14:01.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:14:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:14:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:14:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:14:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:14:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.039291269000003, 2024-09-13T09:14:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:14:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:14:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:14:01.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:14:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:14:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:14:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:14:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:14:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:14:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:14:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:14:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:14:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:14:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:14:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:14:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.039275277000002, 2024-09-13T09:14:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:14:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.583 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:14:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:14:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:14:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:14:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:14:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:14:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:14:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:14:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:14:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:14:01.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:14:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:14:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:14:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:14:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.039247778999993, 2024-09-13T09:14:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:14:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:14:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:01.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:01.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:01.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:14:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:14:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:14:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:14:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:14:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:14:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:14:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:14:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:14:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:14:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:14:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:14:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.039221920000003, 2024-09-13T09:14:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:14:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.592 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:14:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:14:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:14:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:14:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:14:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:14:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:14:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:14:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:14:01.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:14:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:14:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:14:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:14:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.039180430000002, 2024-09-13T09:14:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:14:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:14:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:14:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:14:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:14:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:14:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:14:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:14:01.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:14:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:14:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:14:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:14:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:14:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:14:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:14:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.039159192, 2024-09-13T09:14:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:14:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.601 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:14:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:14:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:14:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:14:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:14:01.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:14:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:14:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:14:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:14:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:14:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:14:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:14:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:14:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.605 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.039136392000003, 2024-09-13T09:14:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:14:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:14:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:14:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:14:01.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:14:01.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:14:01.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:14:01.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:14:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:14:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:14:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:14:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:14:01.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:14:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:14:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:14:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.039119936000006, 2024-09-13T09:14:01.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:14:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:14:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:01.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:01.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:14:01.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:14:01.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:14:01.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:14:01.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:14:01.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:14:01.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:14:01.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:14:01.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:14:01.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:14:01.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:14:01.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:14:01.614 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.615 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.615 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.615 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.615 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:01.615 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.038874798999998, 2024-09-13T09:14:01.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:14:01.615 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.615 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.615 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.615 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.615 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.615 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:14:01.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.616 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:01.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:01.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:14:01.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:14:01.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:14:01.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:14:01.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:14:01.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:14:01.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:14:01.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:14:01.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:14:01.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:14:01.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:14:01.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:14:01.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:14:01.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:14:01.619 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.619 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.619 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.619 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.620 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:01.620 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.036989239999997, 2024-09-13T09:14:01.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:14:01.620 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.620 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.620 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.620 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.620 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.620 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:14:01.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.620 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:01.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:01.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:14:01.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:14:01.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:14:01.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:14:01.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:14:01.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:14:01.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:14:01.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:14:01.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:14:01.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:14:01.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:14:01.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:14:01.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:14:01.624 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.624 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.624 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.624 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.624 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:01.624 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.036967873999998, 2024-09-13T09:14:01.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:14:01.624 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.625 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.625 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.625 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.625 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.625 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:14:01.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.625 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:01.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:14:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:14:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:14:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:14:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:14:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:14:01.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:14:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:14:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:14:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:14:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:14:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:14:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 79.036762506000002, 2024-09-13T09:14:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:14:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:01.631 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:14:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:14:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:14:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:14:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:14:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:14:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:14:01.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:14:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:14:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:14:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:14:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:14:01.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:14:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:14:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:14:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5272.0:1775 3.0 3:02a42fca:::rbd_data.11235782ee8d.0000000000000030:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2244608~40448] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:14:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 51.295220665000002, 2024-09-13T09:14:01.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.666843732, 2024-09-13T09:14:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5272", 2024-09-13T09:14:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4089902097", 2024-09-13T09:14:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1775 2024-09-13T09:14:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:01.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247780+0000", 2024-09-13T09:14:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247865+0000", 2024-09-13T09:14:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4359999999999999e-05 2024-09-13T09:14:01.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247876+0000", 2024-09-13T09:14:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1456e-05 2024-09-13T09:14:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247883+0000", 2024-09-13T09:14:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:14:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247926+0000", 2024-09-13T09:14:01.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3203e-05 2024-09-13T09:14:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247932+0000", 2024-09-13T09:14:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:14:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909639+0000", 2024-09-13T09:14:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6617069099999999 2024-09-13T09:14:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909828+0000", 2024-09-13T09:14:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000189075 2024-09-13T09:14:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911676+0000", 2024-09-13T09:14:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001847235 2024-09-13T09:14:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911765+0000", 2024-09-13T09:14:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9530000000000005e-05 2024-09-13T09:14:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:14:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911901+0000", 2024-09-13T09:14:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013603899999999999 2024-09-13T09:14:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.912889+0000", 2024-09-13T09:14:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098807400000000003 2024-09-13T09:14:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914438+0000", 2024-09-13T09:14:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015491070000000001 2024-09-13T09:14:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914505+0000", 2024-09-13T09:14:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:14:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914625+0000", 2024-09-13T09:14:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120108 2024-09-13T09:14:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:01.643 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:01.643 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:01.645 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:14:01.645 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:01.644+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:01.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:14:01.738 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:14:01.766 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:01.766 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:01.766 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:01.766 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:01.777 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:14:01.782 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:01.781+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:01.846 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:01.844+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:01.880 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:01.880 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:01.880 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:01.880 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:01.881 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:01.887 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:14:01.887 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:14:01.889 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:14:01.907 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:14:01.908 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:14:01.908 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:14:01.924 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:14:01.924 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:14:01.940 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:14:01.940 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:14:01.946 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:01.946+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:01.957 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:14:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:14:01.960 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:14:01.961 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:14:02.032 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:02.032 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:02.040 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:14:02.048 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:02.047+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:02.143 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:02.143 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:02.149 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:02.148+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:02.152 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:14:02.250 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:02.249+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:02.295 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:02.296 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:02.304 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:14:02.352 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:02.351+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:02.447 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:02.447 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:02.453 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:02.452+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:02.456 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:14:02.554 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:02.553+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:02.558 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:02.559 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:02.567 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:14:02.655 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:02.653+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:02.669 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:02.669 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:02.673 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-09-13T09:14:02.680 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:14:02.765 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:02.763+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:02.826 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:02.826 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:02.834 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:14:02.876 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:02.874+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:02.948 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:02.948 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:02.957 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:14:02.958 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:02.956+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:03.058 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:03.057+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:03.064 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:03.064 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:03.073 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:14:03.160 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:03.158+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:03.179 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:03.179 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:03.188 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:14:03.261 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:03.260+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:03.295 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:03.295 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:03.303 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:14:03.361 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:03.360+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:03.450 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:03.450 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:03.459 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:14:03.464 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:03.462+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:03.565 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:03.564+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:03.565 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:03.565 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:03.574 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:14:03.665 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:03.664+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:03.682 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:03.683 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:03.696 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:14:03.766 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:03.765+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:03.849 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:03.849 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:03.857 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:14:03.902 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:03.900+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:03.959 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:03.959 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:03.962 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:14:03.968 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:14:04.006 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:04.005+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:04.068 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:04.066+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:04.084 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:14:04.085 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:14:04.101 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:14:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:14:04.102 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:14:04.111 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:04.111 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:04.119 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:14:04.119 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:14:04.121 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:14:04.135 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:14:04.135 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:14:04.152 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:14:04.152 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:14:04.155 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:14:04.155 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:14:04.197 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:04.196+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:04.263 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:04.263 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:04.263 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:04.264 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:04.264 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:04.271 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:14:04.272 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:04.269+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:04.372 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:04.371+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:04.416 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:04.416 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:04.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:04.416 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:04.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.416 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2185 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.416 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:04.416 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.888129286999998, 2024-09-13T09:14:04.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063348994000000006, 2024-09-13T09:14:04.417 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.417 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.417 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.417 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.417 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.417 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2185 2024-09-13T09:14:04.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.417 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:04.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:04.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:04.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525765+0000", 2024-09-13T09:14:04.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2390000000000001e-06 2024-09-13T09:14:04.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525772+0000", 2024-09-13T09:14:04.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:14:04.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525777+0000", 2024-09-13T09:14:04.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0799999999999996e-06 2024-09-13T09:14:04.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589020+0000", 2024-09-13T09:14:04.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063242501000000007 2024-09-13T09:14:04.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589063+0000", 2024-09-13T09:14:04.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2673999999999998e-05 2024-09-13T09:14:04.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589112+0000", 2024-09-13T09:14:04.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9286000000000002e-05 2024-09-13T09:14:04.421 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.421 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.421 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.421 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2187 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.421 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:04.421 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.888109739999997, 2024-09-13T09:14:04.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063500102000000003, 2024-09-13T09:14:04.421 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.421 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.422 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.422 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2187 2024-09-13T09:14:04.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.422 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:04.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:04.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:04.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525784+0000", 2024-09-13T09:14:04.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1629999999999999e-06 2024-09-13T09:14:04.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525792+0000", 2024-09-13T09:14:04.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:14:04.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525796+0000", 2024-09-13T09:14:04.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7389999999999999e-06 2024-09-13T09:14:04.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589132+0000", 2024-09-13T09:14:04.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063336119999999996 2024-09-13T09:14:04.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589187+0000", 2024-09-13T09:14:04.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4126000000000003e-05 2024-09-13T09:14:04.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589283+0000", 2024-09-13T09:14:04.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6032999999999993e-05 2024-09-13T09:14:04.426 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.426 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.426 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.426 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2188 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.426 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:04.426 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.888091694000003, 2024-09-13T09:14:04.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063724617999999997, 2024-09-13T09:14:04.426 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.427 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.427 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2188 2024-09-13T09:14:04.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.427 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:04.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:04.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:04.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525802+0000", 2024-09-13T09:14:04.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0439999999999998e-06 2024-09-13T09:14:04.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525809+0000", 2024-09-13T09:14:04.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.238e-06 2024-09-13T09:14:04.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525814+0000", 2024-09-13T09:14:04.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-13T09:14:04.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589327+0000", 2024-09-13T09:14:04.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063513257000000004 2024-09-13T09:14:04.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589418+0000", 2024-09-13T09:14:04.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0796000000000003e-05 2024-09-13T09:14:04.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589525+0000", 2024-09-13T09:14:04.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010693500000000001 2024-09-13T09:14:04.431 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.431 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.431 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.431 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2190 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.431 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:04.431 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.888073607999999, 2024-09-13T09:14:04.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063863981, 2024-09-13T09:14:04.431 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.431 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.431 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.432 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.432 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2190 2024-09-13T09:14:04.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.432 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:04.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:04.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525818+0000", 2024-09-13T09:14:04.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525820+0000", 2024-09-13T09:14:04.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:14:04.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525827+0000", 2024-09-13T09:14:04.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0589999999999999e-06 2024-09-13T09:14:04.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525831+0000", 2024-09-13T09:14:04.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7870000000000001e-06 2024-09-13T09:14:04.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589562+0000", 2024-09-13T09:14:04.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063731038000000004 2024-09-13T09:14:04.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589612+0000", 2024-09-13T09:14:04.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0087999999999999e-05 2024-09-13T09:14:04.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589683+0000", 2024-09-13T09:14:04.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0121000000000002e-05 2024-09-13T09:14:04.435 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.435 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.436 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.436 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2192 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.436 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:04.436 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.888056648000003, 2024-09-13T09:14:04.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064050299000000005, 2024-09-13T09:14:04.436 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.436 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.436 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.436 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.436 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.436 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2192 2024-09-13T09:14:04.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.437 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:04.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:04.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525835+0000", 2024-09-13T09:14:04.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525837+0000", 2024-09-13T09:14:04.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:14:04.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525845+0000", 2024-09-13T09:14:04.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3660000000000003e-06 2024-09-13T09:14:04.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525850+0000", 2024-09-13T09:14:04.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0030000000000002e-06 2024-09-13T09:14:04.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589720+0000", 2024-09-13T09:14:04.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063870176000000001 2024-09-13T09:14:04.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589801+0000", 2024-09-13T09:14:04.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0889000000000004e-05 2024-09-13T09:14:04.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589886+0000", 2024-09-13T09:14:04.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5183000000000001e-05 2024-09-13T09:14:04.440 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.440 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.440 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.440 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.440 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:04.440 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.888038545000001, 2024-09-13T09:14:04.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:14:04.441 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.441 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.441 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.441 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.441 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.441 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:14:04.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.441 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:04.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:04.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:14:04.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:14:04.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:14:04.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:14:04.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:14:04.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:14:04.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:14:04.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:14:04.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:14:04.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:14:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:14:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:14:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:14:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.888020847, 2024-09-13T09:14:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:14:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:14:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:14:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:14:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:14:04.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:14:04.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:14:04.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:14:04.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:14:04.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:14:04.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:14:04.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:14:04.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:14:04.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:14:04.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.451 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.452 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.452 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:04.452 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.888000232000003, 2024-09-13T09:14:04.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:14:04.452 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.452 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.452 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:14:04.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.452 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:04.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:04.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:04.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:14:04.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:14:04.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:14:04.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:14:04.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:14:04.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:14:04.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:14:04.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:14:04.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:14:04.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:14:04.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:14:04.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:14:04.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.456 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.456 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.456 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:04.456 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.887981275999998, 2024-09-13T09:14:04.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:14:04.456 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.456 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.456 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.457 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:14:04.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.457 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:04.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:04.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:04.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:14:04.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:14:04.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:14:04.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:14:04.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:14:04.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:14:04.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:14:04.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:14:04.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:14:04.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:14:04.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:14:04.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:14:04.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.460 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.460 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.461 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:04.461 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.887964017999998, 2024-09-13T09:14:04.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:14:04.461 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.461 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.461 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:14:04.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.461 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:04.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:04.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:04.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:14:04.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:14:04.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:14:04.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:14:04.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:14:04.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:14:04.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:14:04.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:14:04.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:14:04.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:14:04.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:14:04.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:14:04.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.465 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.465 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.465 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:04.465 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.887944003999998, 2024-09-13T09:14:04.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:14:04.465 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.465 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.465 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.465 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.466 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:14:04.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.466 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:04.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:04.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:04.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:14:04.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:14:04.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:14:04.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:14:04.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:14:04.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:14:04.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:14:04.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:14:04.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:14:04.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:14:04.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:14:04.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:14:04.469 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.469 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.469 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.469 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.470 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:04.470 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.887918421999998, 2024-09-13T09:14:04.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:14:04.470 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.470 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.470 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:14:04.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.470 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:14:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:14:04.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:14:04.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:14:04.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:14:04.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:14:04.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:14:04.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:14:04.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:14:04.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:14:04.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:14:04.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:14:04.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:14:04.473 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.474 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.474 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.474 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.474 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:04.474 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.887899138000002, 2024-09-13T09:14:04.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:14:04.474 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.474 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.475 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:14:04.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.475 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:04.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:04.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:04.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:14:04.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:14:04.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:14:04.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:14:04.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:14:04.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:14:04.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:14:04.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:14:04.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:14:04.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:14:04.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:14:04.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:14:04.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.478 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.478 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.479 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:04.479 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.887881882000002, 2024-09-13T09:14:04.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:14:04.479 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.479 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.479 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:14:04.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.479 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:04.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:04.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:04.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:14:04.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:14:04.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:14:04.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:14:04.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:14:04.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:14:04.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:14:04.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:14:04.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:14:04.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:14:04.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:14:04.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:14:04.486 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.486 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.486 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.486 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.487 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:04.487 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.887862402000003, 2024-09-13T09:14:04.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:14:04.488 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.488 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.488 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:14:04.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.488 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:04.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:04.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:14:04.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:14:04.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:14:04.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:14:04.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:14:04.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:14:04.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:14:04.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:14:04.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:14:04.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:14:04.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:14:04.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:14:04.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:14:04.492 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.492 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.492 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.492 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.493 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:04.493 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.887845472999999, 2024-09-13T09:14:04.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:14:04.493 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.493 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.493 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.493 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.493 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.493 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:14:04.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.493 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:04.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:04.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:14:04.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:14:04.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:14:04.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:14:04.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:14:04.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:14:04.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:14:04.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:14:04.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:14:04.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:14:04.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:14:04.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:14:04.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:14:04.497 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.497 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.497 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.497 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.497 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:04.497 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.887826324000002, 2024-09-13T09:14:04.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:14:04.497 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.497 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.497 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.497 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.498 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.498 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:14:04.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.498 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:04.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:04.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:04.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:14:04.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:14:04.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:14:04.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:14:04.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:14:04.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:14:04.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:14:04.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:14:04.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:14:04.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:14:04.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:14:04.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:14:04.501 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.501 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.501 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.501 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.502 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:04.502 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.887808184999997, 2024-09-13T09:14:04.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:14:04.502 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.502 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.502 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.502 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.502 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.502 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:14:04.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.502 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:04.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:04.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:04.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:14:04.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:14:04.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:14:04.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:14:04.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:14:04.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:14:04.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:14:04.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:14:04.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:14:04.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:14:04.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:14:04.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:14:04.506 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.506 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.506 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.506 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.506 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:04.506 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.88739769, 2024-09-13T09:14:04.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:14:04.506 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.506 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.506 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.506 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:14:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:04.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:14:04.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:14:04.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:14:04.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:14:04.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:14:04.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:14:04.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:14:04.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:14:04.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:14:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:14:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:14:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:14:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:04.510 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.887374788999999, 2024-09-13T09:14:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:14:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:14:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:04.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:04.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:04.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:14:04.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:14:04.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:14:04.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:14:04.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:14:04.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:14:04.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:14:04.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:14:04.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:14:04.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:14:04.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:14:04.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:14:04.515 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.515 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.515 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.515 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.515 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.515 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:04.516 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:04.515+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:04.517 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:14:04.617 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:04.616+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:04.620 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:04.620 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:04.620 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:04.620 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:04.629 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:14:04.681 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5] 2024-09-13T09:14:04.681 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:14:04.681 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-13T09:14:04.681 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:14:04.681 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:14:04.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-13T09:14:04.693 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:14:04.718 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:04.717+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:04.791 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:04.791 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:04.791 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:04.791 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:04.791 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:04.804 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:14:04.819 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:04.819 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "" 2024-09-13T09:14:04.819 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:04.845 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:04.843+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:04.920 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:04.918+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:04.948 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:04.948 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:04.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:04.948 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:04.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.948 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:04.948 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:04.948 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 67.200420894000004, 2024-09-13T09:14:04.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:14:04.949 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.949 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:04.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:04.949 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:14:04.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.949 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:04.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:04.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:04.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:14:04.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:14:04.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:14:04.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:14:04.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:14:04.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:14:04.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:14:04.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:14:04.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:14:04.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:14:04.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:14:04.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:14:04.953 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.953 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.953 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.953 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:04.953 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:04.953 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 67.200412216000004, 2024-09-13T09:14:04.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:14:04.953 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.954 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:04.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:04.954 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:14:04.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.954 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:04.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:04.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:04.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:04.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:14:04.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:14:04.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:14:04.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:14:04.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:14:04.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:14:04.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:14:04.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:14:04.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:14:04.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:14:04.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:14:04.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:14:04.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.958 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.958 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:04.958 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:04.958 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 67.200403230000006, 2024-09-13T09:14:04.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:14:04.958 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.958 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.958 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.958 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:04.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:04.959 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:14:04.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.959 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:04.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:04.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:04.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:14:04.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:14:04.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:14:04.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:14:04.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:14:04.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:14:04.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:14:04.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:14:04.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:14:04.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:14:04.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:14:04.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:14:04.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.963 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.963 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:04.963 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:04.963 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 67.200395258, 2024-09-13T09:14:04.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:14:04.964 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.964 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:04.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:04.964 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:14:04.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.964 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:04.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:04.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:04.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:04.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:14:04.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:14:04.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:14:04.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:14:04.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:14:04.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:14:04.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:14:04.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:14:04.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:14:04.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:14:04.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:14:04.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:14:04.968 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.968 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.968 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.968 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:04.968 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:04.968 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 67.198520423999994, 2024-09-13T09:14:04.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:14:04.969 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.969 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:04.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:04.969 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:14:04.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.969 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:04.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:04.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:04.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:14:04.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:14:04.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:14:04.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:14:04.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:14:04.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:14:04.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:14:04.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:14:04.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:14:04.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:14:04.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:14:04.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:14:04.973 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.973 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.973 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.973 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:04.973 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:04.973 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 67.198509763000004, 2024-09-13T09:14:04.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:14:04.973 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.973 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.974 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.974 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:04.974 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:04.974 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:14:04.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.974 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:04.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:04.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:04.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:04.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:14:04.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:14:04.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:14:04.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:14:04.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:14:04.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:14:04.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:14:04.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:14:04.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:14:04.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:14:04.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:14:04.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:14:04.977 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.977 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.978 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.978 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:04.978 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:04.978 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 67.198487787000005, 2024-09-13T09:14:04.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:14:04.978 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.978 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.978 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.978 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:04.979 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:04.979 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:14:04.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.979 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:04.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:04.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:04.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:14:04.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:14:04.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:14:04.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:14:04.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:14:04.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:14:04.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:14:04.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:14:04.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:14:04.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:14:04.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:14:04.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:14:04.982 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.982 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.982 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.983 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:04.983 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:04.983 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 67.198474285000003, 2024-09-13T09:14:04.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:14:04.983 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.983 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.983 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.983 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:04.983 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:04.983 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:14:04.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.983 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:04.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:04.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:04.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:14:04.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:14:04.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:14:04.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:14:04.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:14:04.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:14:04.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:14:04.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:14:04.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:14:04.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:14:04.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:14:04.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:14:04.988 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.988 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.988 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.988 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:04.988 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:04.989 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 67.196465187000001, 2024-09-13T09:14:04.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:14:04.989 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.989 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.989 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.989 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:04.989 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:04.989 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:14:04.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.990 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:04.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:04.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:04.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:14:04.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:14:04.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:14:04.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:14:04.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:14:04.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:14:04.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:14:04.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:14:04.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:14:04.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:14:04.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:14:04.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:14:04.993 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.993 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.993 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.993 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:04.993 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:04.994 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 67.196440865, 2024-09-13T09:14:04.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:14:04.994 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:04.994 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:04.994 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:04.994 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:04.994 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:04.994 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:14:04.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.994 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:04.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:04.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:04.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:04.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:04.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:04.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:04.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:04.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:04.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:04.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:14:04.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:14:04.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:04.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:14:04.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:14:04.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:04.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:14:04.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:14:04.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:04.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:14:04.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:14:04.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:04.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:14:04.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:14:04.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:04.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:14:04.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:14:04.998 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.998 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:04.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:04.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:04.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:04.999 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:04.999 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:14:04.999 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 67.196429890999994, 2024-09-13T09:14:04.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:14:05.000 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:05.000 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:05.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:05.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:05.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:05.000 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:14:05.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.000 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:05.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:05.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:14:05.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:05.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:14:05.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:05.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:14:05.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:05.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:14:05.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:14:05.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:05.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:14:05.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:14:05.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:14:05.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:14:05.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:14:05.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:14:05.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:05.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:14:05.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:14:05.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:05.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:14:05.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:14:05.004 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.004 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:05.004 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.005 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:05.005 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:05.005 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 67.194512266000004, 2024-09-13T09:14:05.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:14:05.005 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:05.005 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:05.005 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:05.005 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:05.005 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:05.005 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:14:05.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.005 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:05.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:05.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:05.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:05.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:05.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:05.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:05.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:05.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:14:05.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:14:05.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:05.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:14:05.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:14:05.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:14:05.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:14:05.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:14:05.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:14:05.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:05.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:14:05.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:14:05.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:05.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:14:05.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:14:05.009 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.009 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:05.009 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.009 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:05.009 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:05.009 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 67.194502869000004, 2024-09-13T09:14:05.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:14:05.009 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:05.010 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:05.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:05.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:05.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:05.010 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:14:05.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.010 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:05.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:05.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:05.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:05.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:05.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:05.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:05.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:05.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:05.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:14:05.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:14:05.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:05.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:14:05.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:14:05.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:14:05.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:14:05.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:14:05.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:14:05.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:05.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:14:05.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:14:05.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:05.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:14:05.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:14:05.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.014 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:05.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.014 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3181 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T09:14:05.014 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:14:05.014 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.924138693000003, 2024-09-13T09:14:05.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.480957603, 2024-09-13T09:14:05.014 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:05.014 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:05.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:05.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:14:05.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:14:05.015 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3181 2024-09-13T09:14:05.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.015 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:05.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:05.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:14:05.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:05.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:14:05.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:05.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021894+0000", 2024-09-13T09:14:05.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:14:05.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:05.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021903+0000", 2024-09-13T09:14:05.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4109999999999996e-06 2024-09-13T09:14:05.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:05.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021919+0000", 2024-09-13T09:14:05.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6225000000000001e-05 2024-09-13T09:14:05.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021926+0000", 2024-09-13T09:14:05.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4120000000000004e-06 2024-09-13T09:14:05.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021975+0000", 2024-09-13T09:14:05.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8526000000000003e-05 2024-09-13T09:14:05.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:05.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022049+0000", 2024-09-13T09:14:05.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4018999999999997e-05 2024-09-13T09:14:05.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:14:05.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022230+0000", 2024-09-13T09:14:05.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018085599999999999 2024-09-13T09:14:05.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:05.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022796+0000", 2024-09-13T09:14:05.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056638999999999995 2024-09-13T09:14:05.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499540+0000", 2024-09-13T09:14:05.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.47674342800000002 2024-09-13T09:14:05.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499547+0000", 2024-09-13T09:14:05.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:14:05.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502794+0000", 2024-09-13T09:14:05.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.003246818 2024-09-13T09:14:05.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:05.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502854+0000", 2024-09-13T09:14:05.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9925e-05 2024-09-13T09:14:05.021 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.021 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:05.021 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.021 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3182 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 139818970156544 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:14:05.021 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:14:05.021 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.450904137999999, 2024-09-13T09:14:05.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0089058390000001, 2024-09-13T09:14:05.022 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:05.022 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:05.022 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:05.022 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:14:05.022 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:14:05.022 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3182 2024-09-13T09:14:05.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.022 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:05.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:05.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:14:05.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:05.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:14:05.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:05.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495128+0000", 2024-09-13T09:14:05.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:14:05.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:05.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495139+0000", 2024-09-13T09:14:05.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0599e-05 2024-09-13T09:14:05.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:05.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495157+0000", 2024-09-13T09:14:05.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7830999999999999e-05 2024-09-13T09:14:05.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495167+0000", 2024-09-13T09:14:05.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0689999999999999e-05 2024-09-13T09:14:05.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495229+0000", 2024-09-13T09:14:05.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1898999999999998e-05 2024-09-13T09:14:05.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:14:05.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495243+0000", 2024-09-13T09:14:05.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3349e-05 2024-09-13T09:14:05.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499575+0000", 2024-09-13T09:14:05.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043324410000000002 2024-09-13T09:14:05.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499582+0000", 2024-09-13T09:14:05.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0849999999999999e-06 2024-09-13T09:14:05.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502872+0000", 2024-09-13T09:14:05.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0032895150000001 2024-09-13T09:14:05.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:05.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503016+0000", 2024-09-13T09:14:05.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014394200000000001 2024-09-13T09:14:05.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:05.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504036+0000", 2024-09-13T09:14:05.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010208680000000001 2024-09-13T09:14:05.027 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.027 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:05.027 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.027 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:19151 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 140383089882784 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:14:05.027 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:14:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 34.450892678999999, 2024-09-13T09:14:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.009071287, 2024-09-13T09:14:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19151 2024-09-13T09:14:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:14:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:14:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:05.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495140+0000", 2024-09-13T09:14:05.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:14:05.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:05.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495149+0000", 2024-09-13T09:14:05.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0759999999999992e-06 2024-09-13T09:14:05.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:05.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495165+0000", 2024-09-13T09:14:05.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5322e-05 2024-09-13T09:14:05.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495173+0000", 2024-09-13T09:14:05.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0979999999999994e-06 2024-09-13T09:14:05.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495318+0000", 2024-09-13T09:14:05.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014553 2024-09-13T09:14:05.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:14:05.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495325+0000", 2024-09-13T09:14:05.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:14:05.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499598+0000", 2024-09-13T09:14:05.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004273064 2024-09-13T09:14:05.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499602+0000", 2024-09-13T09:14:05.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8539999999999999e-06 2024-09-13T09:14:05.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503507+0000", 2024-09-13T09:14:05.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0039045520000001 2024-09-13T09:14:05.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:05.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503583+0000", 2024-09-13T09:14:05.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6283000000000001e-05 2024-09-13T09:14:05.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:05.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504213+0000", 2024-09-13T09:14:05.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00063044700000000002 2024-09-13T09:14:05.034 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.034 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:05.034 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.034 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3908 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:05.035 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:14:05.035 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 16.853355788000002, 2024-09-13T09:14:05.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070530317999999995, 2024-09-13T09:14:05.035 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:05.035 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:05.035 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:05.035 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:05.035 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:05.035 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3908 2024-09-13T09:14:05.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.035 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:05.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:05.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:14:05.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:05.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:14:05.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:05.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092678+0000", 2024-09-13T09:14:05.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:05.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092681+0000", 2024-09-13T09:14:05.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.903e-06 2024-09-13T09:14:05.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:05.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092687+0000", 2024-09-13T09:14:05.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0859999999999998e-06 2024-09-13T09:14:05.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092692+0000", 2024-09-13T09:14:05.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-13T09:14:05.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.159711+0000", 2024-09-13T09:14:05.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067018922999999994 2024-09-13T09:14:05.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:05.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.159796+0000", 2024-09-13T09:14:05.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5247000000000006e-05 2024-09-13T09:14:05.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:05.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160111+0000", 2024-09-13T09:14:05.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00031449499999999998 2024-09-13T09:14:05.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:05.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162219+0000", 2024-09-13T09:14:05.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021085570000000001 2024-09-13T09:14:05.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:05.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163127+0000", 2024-09-13T09:14:05.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00090723400000000004 2024-09-13T09:14:05.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:05.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163172+0000", 2024-09-13T09:14:05.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5383000000000002e-05 2024-09-13T09:14:05.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:05.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163209+0000", 2024-09-13T09:14:05.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7132000000000003e-05 2024-09-13T09:14:05.040 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.040 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:05.040 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.040 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3909 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:05.040 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:05.041 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 16.853338297000001, 2024-09-13T09:14:05.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070616834000000003, 2024-09-13T09:14:05.041 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:05.041 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:05.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:05.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:05.041 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:05.041 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3909 2024-09-13T09:14:05.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.041 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:05.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:05.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:05.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:05.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:05.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:05.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:05.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:05.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092699+0000", 2024-09-13T09:14:05.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:14:05.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:05.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092704+0000", 2024-09-13T09:14:05.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5940000000000001e-06 2024-09-13T09:14:05.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092709+0000", 2024-09-13T09:14:05.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5109999999999997e-06 2024-09-13T09:14:05.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160604+0000", 2024-09-13T09:14:05.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067895633999999996 2024-09-13T09:14:05.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:05.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160658+0000", 2024-09-13T09:14:05.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3292999999999999e-05 2024-09-13T09:14:05.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:05.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160935+0000", 2024-09-13T09:14:05.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000276879 2024-09-13T09:14:05.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:05.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163223+0000", 2024-09-13T09:14:05.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022886320000000001 2024-09-13T09:14:05.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:05.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163258+0000", 2024-09-13T09:14:05.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5101000000000002e-05 2024-09-13T09:14:05.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:05.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163292+0000", 2024-09-13T09:14:05.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3237e-05 2024-09-13T09:14:05.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:05.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163313+0000", 2024-09-13T09:14:05.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1642e-05 2024-09-13T09:14:05.047 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.047 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:05.047 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.047 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3910 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:05.047 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:05.047 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 16.853321232999999, 2024-09-13T09:14:05.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733837999999993, 2024-09-13T09:14:05.048 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:05.048 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:05.048 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:05.048 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:05.048 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:05.048 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3910 2024-09-13T09:14:05.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.048 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:05.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:05.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:05.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:05.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:05.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:05.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:05.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:05.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092716+0000", 2024-09-13T09:14:05.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.757e-06 2024-09-13T09:14:05.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:05.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092722+0000", 2024-09-13T09:14:05.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7869999999999998e-06 2024-09-13T09:14:05.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092726+0000", 2024-09-13T09:14:05.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.622e-06 2024-09-13T09:14:05.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161487+0000", 2024-09-13T09:14:05.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068761166999999998 2024-09-13T09:14:05.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:05.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161569+0000", 2024-09-13T09:14:05.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1185000000000003e-05 2024-09-13T09:14:05.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:05.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161868+0000", 2024-09-13T09:14:05.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029979799999999999 2024-09-13T09:14:05.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:05.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163366+0000", 2024-09-13T09:14:05.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001497888 2024-09-13T09:14:05.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:05.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163373+0000", 2024-09-13T09:14:05.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1640000000000004e-06 2024-09-13T09:14:05.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:05.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163412+0000", 2024-09-13T09:14:05.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8423999999999997e-05 2024-09-13T09:14:05.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:05.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163447+0000", 2024-09-13T09:14:05.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5441000000000002e-05 2024-09-13T09:14:05.053 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.053 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:05.053 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.053 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3911 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:05.053 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:05.053 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 16.853301317, 2024-09-13T09:14:05.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.072284372999999999, 2024-09-13T09:14:05.054 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:05.054 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:05.054 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:05.054 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:05.054 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:05.056 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3911 2024-09-13T09:14:05.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.056 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:05.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:05.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:05.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:05.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:05.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:05.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:05.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:05.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:05.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092736+0000", 2024-09-13T09:14:05.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9510000000000002e-06 2024-09-13T09:14:05.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:05.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092743+0000", 2024-09-13T09:14:05.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9e-06 2024-09-13T09:14:05.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092748+0000", 2024-09-13T09:14:05.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999998e-06 2024-09-13T09:14:05.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:05.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162257+0000", 2024-09-13T09:14:05.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.069508890000000004 2024-09-13T09:14:05.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:05.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162342+0000", 2024-09-13T09:14:05.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5886999999999999e-05 2024-09-13T09:14:05.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:05.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162682+0000", 2024-09-13T09:14:05.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00033971700000000001 2024-09-13T09:14:05.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:05.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164897+0000", 2024-09-13T09:14:05.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022152370000000001 2024-09-13T09:14:05.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:05.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164945+0000", 2024-09-13T09:14:05.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7243999999999997e-05 2024-09-13T09:14:05.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:05.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164985+0000", 2024-09-13T09:14:05.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0599000000000002e-05 2024-09-13T09:14:05.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:05.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:05.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:05.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.165018+0000", 2024-09-13T09:14:05.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2549000000000003e-05 2024-09-13T09:14:05.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.061 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:05.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:05.061 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:05.061 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:05.063 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:05.062+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:05.063 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:14:05.164 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:05.163+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:05.170 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:05.170 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:05.170 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:05.170 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:05.179 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:14:05.265 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:05.264+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:05.286 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:05.286 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:05.286 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:05.286 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:05.286 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:05.295 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:14:05.367 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:05.365+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:05.446 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:05.446 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:05.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:05.446 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:05.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.446 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.446 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:05.446 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.914752386999993, 2024-09-13T09:14:05.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:14:05.447 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.447 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.447 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.447 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.447 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.447 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:14:05.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.447 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:05.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:05.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:14:05.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:14:05.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:14:05.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:14:05.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:14:05.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.450 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:14:05.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:14:05.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.450 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:14:05.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:14:05.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.450 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:14:05.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:14:05.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:14:05.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:14:05.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:14:05.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:14:05.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.452 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:14:05.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:14:05.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.452 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:14:05.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:14:05.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:14:05.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:14:05.453 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.453 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.453 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.453 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.453 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:05.453 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.914734351999996, 2024-09-13T09:14:05.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:14:05.453 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.453 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.454 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.454 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.454 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.454 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:14:05.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.454 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.456 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:14:05.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:14:05.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.456 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:14:05.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:14:05.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.456 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:14:05.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:14:05.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:14:05.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:14:05.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:14:05.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:14:05.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:14:05.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:14:05.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:14:05.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:14:05.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:14:05.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:14:05.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:14:05.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:14:05.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:14:05.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:14:05.459 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.459 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.459 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.459 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.460 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:05.460 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.914717682000003, 2024-09-13T09:14:05.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:14:05.460 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.460 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.460 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.460 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.460 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.460 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:14:05.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.460 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.460 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:05.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:05.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:14:05.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:14:05.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:05.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:14:05.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:14:05.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:14:05.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:14:05.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.463 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:14:05.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:14:05.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.463 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:14:05.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:14:05.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.464 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:14:05.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:14:05.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.464 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:14:05.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:14:05.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.464 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:14:05.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:14:05.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:14:05.465 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:14:05.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:14:05.465 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:14:05.465 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.465 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.465 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.466 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.466 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:05.466 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.914700049999993, 2024-09-13T09:14:05.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:14:05.466 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.466 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.466 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.466 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.466 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.466 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:14:05.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.466 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.467 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:05.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.467 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:05.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.467 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:05.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.468 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.468 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:14:05.468 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:14:05.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.468 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.468 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:14:05.468 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:14:05.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.468 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.469 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:14:05.469 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:14:05.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.469 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.469 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:14:05.469 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:14:05.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.469 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.469 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:14:05.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:14:05.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:14:05.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:14:05.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:14:05.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:14:05.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:14:05.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:14:05.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:14:05.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:14:05.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.472 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.472 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.472 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:14:05.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:14:05.472 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.472 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.472 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.472 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.472 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.472 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:05.472 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.914653744999995, 2024-09-13T09:14:05.473 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:14:05.473 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.473 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.473 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.473 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.473 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.473 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:14:05.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.473 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.474 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:05.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.474 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:05.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:14:05.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:14:05.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:14:05.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:14:05.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:14:05.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:14:05.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:14:05.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:14:05.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:14:05.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:14:05.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:14:05.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:14:05.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:14:05.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:14:05.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:14:05.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.478 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:14:05.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:14:05.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.478 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:14:05.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:14:05.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:14:05.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:14:05.479 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.479 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.479 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.479 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.479 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:05.480 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.914628444000002, 2024-09-13T09:14:05.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:14:05.480 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.480 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.480 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.480 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.480 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.480 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:14:05.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.480 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:05.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:05.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:14:05.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:14:05.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:14:05.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:14:05.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:14:05.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:14:05.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:14:05.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:14:05.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:14:05.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:14:05.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:14:05.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:14:05.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:14:05.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:14:05.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:14:05.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:14:05.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:14:05.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:14:05.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:14:05.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:14:05.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:14:05.485 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.485 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.485 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.486 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.486 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:05.486 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.914610214999996, 2024-09-13T09:14:05.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:14:05.486 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.486 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.486 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.486 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.486 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.486 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:14:05.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.487 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:05.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:05.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:05.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:14:05.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:14:05.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:14:05.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:14:05.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:14:05.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:14:05.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:14:05.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:14:05.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:14:05.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:14:05.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:14:05.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:14:05.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:14:05.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:14:05.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:14:05.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:14:05.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:14:05.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:14:05.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:14:05.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:14:05.491 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.491 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.491 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.492 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.492 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:05.492 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.914562896000007, 2024-09-13T09:14:05.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:14:05.492 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.492 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.492 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.492 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.492 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.492 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:14:05.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.492 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:05.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:05.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:14:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:14:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:14:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:14:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:14:05.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:14:05.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:14:05.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:14:05.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:14:05.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:14:05.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:14:05.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:14:05.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:14:05.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:14:05.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:14:05.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:14:05.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:14:05.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:14:05.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:14:05.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:14:05.497 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.497 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.497 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.498 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.498 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:05.498 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.914048657999999, 2024-09-13T09:14:05.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:14:05.499 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.499 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.499 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.499 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.499 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.499 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:14:05.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.499 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:05.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:05.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:14:05.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:05.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:14:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:14:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:14:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:14:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:14:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:14:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:14:05.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:14:05.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:14:05.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:14:05.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:14:05.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:14:05.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:14:05.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:14:05.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:14:05.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:14:05.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:14:05.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:14:05.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:14:05.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:14:05.504 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.504 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.504 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.505 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.505 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:05.505 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.914014128000005, 2024-09-13T09:14:05.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:14:05.505 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.505 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.505 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.505 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.505 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.505 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:14:05.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.505 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:05.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:05.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:05.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:14:05.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:14:05.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:14:05.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:14:05.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:14:05.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:14:05.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:14:05.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:14:05.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:14:05.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:14:05.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:14:05.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:14:05.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:14:05.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:14:05.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:14:05.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:14:05.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:14:05.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:14:05.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:14:05.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:14:05.510 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.510 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.510 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.511 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.511 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:05.511 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.913996221000005, 2024-09-13T09:14:05.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:14:05.511 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.511 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.511 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.511 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.511 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.511 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:14:05.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.511 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:05.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:05.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:05.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:14:05.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:14:05.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:14:05.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:14:05.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:14:05.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:14:05.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:14:05.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:14:05.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:14:05.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:14:05.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:14:05.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:14:05.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:14:05.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:14:05.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:14:05.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:14:05.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:14:05.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:14:05.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:14:05.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:14:05.519 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.519 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.519 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.520 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.520 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:05.520 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.904869372999997, 2024-09-13T09:14:05.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:14:05.520 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.520 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.520 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.520 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.520 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.520 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:14:05.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.521 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:05.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:05.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:05.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:14:05.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:14:05.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:14:05.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:14:05.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:14:05.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:14:05.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:14:05.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:14:05.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:14:05.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:14:05.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:14:05.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:14:05.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:14:05.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:14:05.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:14:05.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:14:05.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:14:05.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:14:05.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:14:05.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:14:05.526 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.526 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.526 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.526 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.526 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:05.526 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.904817793000007, 2024-09-13T09:14:05.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:14:05.527 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.527 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.527 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.527 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.527 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.527 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:14:05.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.527 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:05.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:05.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:14:05.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:14:05.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:14:05.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:14:05.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:14:05.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:14:05.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:14:05.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:14:05.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:14:05.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:14:05.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:14:05.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:14:05.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:14:05.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:14:05.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:14:05.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:14:05.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:14:05.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:14:05.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:14:05.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.532 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:14:05.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:14:05.532 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.532 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.532 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.532 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.532 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:05.532 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.904801516999996, 2024-09-13T09:14:05.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:14:05.533 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.533 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.533 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.533 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.533 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.533 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:14:05.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.533 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:05.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:05.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:14:05.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:14:05.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:14:05.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:14:05.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:14:05.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:14:05.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:14:05.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:14:05.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:14:05.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:14:05.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:14:05.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:14:05.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:14:05.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:14:05.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:14:05.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:14:05.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:14:05.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:14:05.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:14:05.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:14:05.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:14:05.538 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.538 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.538 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.538 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.538 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:05.538 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.904785743000005, 2024-09-13T09:14:05.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:14:05.539 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.539 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.539 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.539 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.539 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.539 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:14:05.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.539 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:05.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:05.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:14:05.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:14:05.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:14:05.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.541 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:14:05.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:14:05.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.541 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:14:05.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:14:05.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:14:05.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:14:05.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:14:05.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:14:05.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:14:05.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:14:05.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.543 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.543 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:14:05.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:14:05.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.543 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.543 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:14:05.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:14:05.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.543 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.543 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:14:05.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:14:05.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:14:05.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:14:05.544 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.544 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.544 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.544 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.544 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:05.545 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.904770868, 2024-09-13T09:14:05.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:14:05.545 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.545 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.545 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.545 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.545 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.545 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:14:05.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.545 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:05.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:05.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:05.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:14:05.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:14:05.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:14:05.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:14:05.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:14:05.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:14:05.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:14:05.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:14:05.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:14:05.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:14:05.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:14:05.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:14:05.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:14:05.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:14:05.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:14:05.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:14:05.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:14:05.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:14:05.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:14:05.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:14:05.550 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.550 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.550 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.551 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.551 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:05.551 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.904753405999998, 2024-09-13T09:14:05.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:14:05.551 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.551 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.551 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.551 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.551 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.551 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:14:05.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.551 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:05.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:05.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:14:05.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:14:05.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:14:05.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:14:05.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:14:05.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:14:05.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:14:05.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:14:05.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:14:05.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:14:05.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:14:05.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:14:05.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:14:05.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:14:05.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:14:05.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:14:05.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:14:05.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:14:05.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:14:05.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:14:05.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:14:05.557 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.557 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.557 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.557 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.557 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:05.557 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.903152774000006, 2024-09-13T09:14:05.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:14:05.557 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.557 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.557 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.558 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.558 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.558 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:14:05.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.558 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:05.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:05.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:14:05.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:14:05.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:14:05.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:14:05.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:14:05.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:14:05.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:14:05.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:14:05.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:14:05.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:14:05.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:14:05.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:14:05.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:14:05.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:14:05.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:14:05.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:14:05.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:14:05.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:14:05.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:14:05.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:14:05.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:14:05.563 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.563 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.563 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.563 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.563 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:05.563 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.903112723000007, 2024-09-13T09:14:05.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:14:05.563 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.563 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.563 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.564 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.564 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.564 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:14:05.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.564 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:05.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:05.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:14:05.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:14:05.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:14:05.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:14:05.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:14:05.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:14:05.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:14:05.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:14:05.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:14:05.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:14:05.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:14:05.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:14:05.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:14:05.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:14:05.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:14:05.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:14:05.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:14:05.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:14:05.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:14:05.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:14:05.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:14:05.569 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.569 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.569 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.569 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.569 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:05.569 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 82.903068959999999, 2024-09-13T09:14:05.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:14:05.569 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.569 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.569 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.569 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.570 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.570 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:14:05.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.570 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:05.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:05.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:05.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:14:05.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:14:05.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:14:05.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:14:05.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:14:05.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:14:05.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:14:05.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:14:05.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:14:05.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:14:05.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:14:05.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:14:05.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:05.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:14:05.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:14:05.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:14:05.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:14:05.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:14:05.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:14:05.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:14:05.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:14:05.575 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.576 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.576 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.576 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.576 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.576 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:05.577 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:14:05.578 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:05.576+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:05.678 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:05.677+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:05.687 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:05.687 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:05.688 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:05.688 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:05.688 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-09-13T09:14:05.700 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:14:05.779 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:05.778+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:05.810 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:05.810 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:05.810 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:05.810 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:05.810 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:05.819 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:14:05.881 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:05.880+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:05.967 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:05.967 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:05.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:05.967 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:05.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.967 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.967 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.461508451, 2024-09-13T09:14:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:14:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:14:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:05.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:14:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:14:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:14:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:14:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:14:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:14:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:14:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:14:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:14:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:14:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:14:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:14:05.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:14:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.461480766999998, 2024-09-13T09:14:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:14:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.972 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:14:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:14:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:14:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:14:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:14:05.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:14:05.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:14:05.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:14:05.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:14:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:14:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:14:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:14:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:14:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.976 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.461456425999998, 2024-09-13T09:14:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:14:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:14:05.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:14:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:14:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:14:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:14:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:14:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:14:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:14:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:14:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:14:05.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:14:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:14:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:14:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:14:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.461440105999998, 2024-09-13T09:14:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:14:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:14:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:14:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:14:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:14:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:14:05.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:14:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:14:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:14:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:14:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:14:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:14:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:14:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.461423690999993, 2024-09-13T09:14:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:14:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.986 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.987 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.987 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.987 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.987 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:14:05.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.987 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:14:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:14:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:14:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:14:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:14:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:14:05.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:14:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:14:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:14:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:14:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:14:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:14:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.461407582999996, 2024-09-13T09:14:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:14:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.991 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:14:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:14:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:14:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:14:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:14:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:14:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:14:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:14:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:14:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:14:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:14:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:14:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:14:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.461392021999998, 2024-09-13T09:14:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:14:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:14:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:05.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:14:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:14:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:14:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:14:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:14:05.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:14:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:14:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:14:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:14:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:14:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:05.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:14:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:14:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.461375184000005, 2024-09-13T09:14:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:14:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:06.000 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:14:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:06.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:14:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:14:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:14:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:14:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:14:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:14:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:14:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:14:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:14:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:14:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:14:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:14:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:14:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.004 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.461359192000003, 2024-09-13T09:14:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:14:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:14:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:14:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:14:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:14:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:14:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:14:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:14:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:14:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:14:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:14:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:14:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:14:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:14:06.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:14:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.461331693999995, 2024-09-13T09:14:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:14:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:06.009 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:14:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:14:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:14:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:14:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:14:06.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:14:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:14:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:14:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:14:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:06.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:14:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:14:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:14:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:14:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:06.013 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.461305835000005, 2024-09-13T09:14:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:14:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:14:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:06.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:14:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:14:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:14:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:14:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:14:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:14:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:14:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:14:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:14:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:14:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:14:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:14:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.461264345000004, 2024-09-13T09:14:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:14:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:06.018 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:14:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:14:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:14:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:14:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:14:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:14:06.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:14:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:14:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:14:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:14:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:14:06.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:14:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:14:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:14:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.461243107000001, 2024-09-13T09:14:06.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:14:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:14:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:14:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:14:06.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:14:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:14:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:14:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:14:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:06.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:14:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:14:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:14:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:14:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:14:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:14:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:06.026 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.461220307000005, 2024-09-13T09:14:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:14:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:14:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.027 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:14:06.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:14:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:14:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:14:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:14:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:06.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:14:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:14:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:14:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:14:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:14:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:14:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:14:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:14:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.461203850999993, 2024-09-13T09:14:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:14:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:06.031 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:14:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:06.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:14:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:14:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:14:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:14:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:14:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:14:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:14:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:14:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:14:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:14:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:14:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:14:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.035 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.460958714, 2024-09-13T09:14:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:14:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:14:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.036 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:14:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:14:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:14:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:14:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:14:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:14:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:06.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:14:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:14:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:14:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:14:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:14:06.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:14:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:14:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:14:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:06.040 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.459073154999999, 2024-09-13T09:14:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:14:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:14:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:14:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:14:06.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:14:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:14:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:14:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:14:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:14:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:06.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:14:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:14:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:14:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:14:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:14:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:14:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.044 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.459051789, 2024-09-13T09:14:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:14:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:06.045 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:14:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:14:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:14:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:14:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:14:06.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:14:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:14:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:14:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:14:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:06.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:14:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:14:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:14:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:14:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:06.049 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 83.458846421000004, 2024-09-13T09:14:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:14:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:14:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:06.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:14:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:14:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:14:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:14:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:14:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:14:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:14:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:14:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:14:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:14:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:14:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:14:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:14:06.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:14:06.054 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.054 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5272.0:1775 3.0 3:02a42fca:::rbd_data.11235782ee8d.0000000000000030:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2244608~40448] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:14:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 55.717304579999997, 2024-09-13T09:14:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.666843732, 2024-09-13T09:14:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5272", 2024-09-13T09:14:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4089902097", 2024-09-13T09:14:06.055 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1775 2024-09-13T09:14:06.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.056 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:06.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:06.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:06.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247780+0000", 2024-09-13T09:14:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247865+0000", 2024-09-13T09:14:06.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4359999999999999e-05 2024-09-13T09:14:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247876+0000", 2024-09-13T09:14:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1456e-05 2024-09-13T09:14:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247883+0000", 2024-09-13T09:14:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:14:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247926+0000", 2024-09-13T09:14:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3203e-05 2024-09-13T09:14:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247932+0000", 2024-09-13T09:14:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:14:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909639+0000", 2024-09-13T09:14:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6617069099999999 2024-09-13T09:14:06.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909828+0000", 2024-09-13T09:14:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000189075 2024-09-13T09:14:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911676+0000", 2024-09-13T09:14:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001847235 2024-09-13T09:14:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:06.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911765+0000", 2024-09-13T09:14:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9530000000000005e-05 2024-09-13T09:14:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:14:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911901+0000", 2024-09-13T09:14:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013603899999999999 2024-09-13T09:14:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.912889+0000", 2024-09-13T09:14:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098807400000000003 2024-09-13T09:14:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914438+0000", 2024-09-13T09:14:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015491070000000001 2024-09-13T09:14:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914505+0000", 2024-09-13T09:14:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:14:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914625+0000", 2024-09-13T09:14:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120108 2024-09-13T09:14:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.062 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:06.063 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.063 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:06.063 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:06.063 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:06.064 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:14:06.065 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:06.063+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:06.157 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:14:06.165 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:06.163+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:06.168 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:06.168 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:06.168 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:06.168 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:06.178 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:14:06.266 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:06.265+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:06.278 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:14:06.278 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:14:06.295 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:14:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:14:06.295 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:14:06.313 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:14:06.313 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:14:06.323 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:06.323 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:06.329 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:14:06.329 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:14:06.332 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:14:06.345 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:14:06.346 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:14:06.349 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:14:06.349 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:14:06.374 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:06.373+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:06.467 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:06.466+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:06.474 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:06.475 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:06.483 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:14:06.568 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:06.567+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:06.627 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:06.627 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:06.636 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:14:06.680 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:06.678+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:06.750 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:06.750 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:06.759 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:14:06.801 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:06.800+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:06.870 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:06.869+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:06.905 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:06.905 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:06.914 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:14:06.971 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:06.970+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:07.063 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:07.063 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:07.073 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:14:07.074 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:07.072+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:07.174 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:07.173+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:07.176 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:07.176 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:07.185 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:14:07.275 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:07.274+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:07.327 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:07.327 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:07.336 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:14:07.378 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:07.376+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:07.456 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:07.456 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:07.465 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:14:07.477 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:07.476+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:07.573 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:07.573 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:07.577 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:07.576+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:07.582 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:14:07.678 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:07.677+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:07.704 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:14:07.737 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:07.737 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:07.746 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:14:07.779 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:07.779+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:07.881 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:07.880+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:07.896 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:07.897 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:07.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:14:07.982 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:07.981+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:08.013 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:08.013 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:08.021 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:14:08.083 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:08.082+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:08.167 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:08.168 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:08.177 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:14:08.184 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:08.183+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:08.281 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:08.281 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:08.284 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:08.283+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:08.289 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:14:08.351 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:14:08.385 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:08.384+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:08.392 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:08.392 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:08.402 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:14:08.473 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:14:08.473 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:14:08.485 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:08.485+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:08.489 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:14:08.490 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:14:08.490 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:14:08.509 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:14:08.509 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:14:08.526 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:14:08.526 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:14:08.542 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:14:08.542 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:14:08.545 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:14:08.546 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:14:08.549 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:08.549 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:08.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:08.549 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:08.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.549 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.549 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:08.549 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.025469408999996, 2024-09-13T09:14:08.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:14:08.549 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.550 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.550 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:14:08.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.550 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:08.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:08.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:08.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:08.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:14:08.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:14:08.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:14:08.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:14:08.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:14:08.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:14:08.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:14:08.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:14:08.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:14:08.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:14:08.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:14:08.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:14:08.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:14:08.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:14:08.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.554 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.554 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.555 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:08.555 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.025420353000001, 2024-09-13T09:14:08.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:14:08.555 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.555 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.555 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:14:08.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.555 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:08.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:08.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:08.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:08.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:14:08.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:14:08.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:14:08.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:14:08.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:14:08.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:14:08.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:14:08.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:14:08.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:14:08.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:14:08.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:14:08.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:14:08.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:14:08.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:14:08.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.560 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.560 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.560 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:08.560 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.025380278, 2024-09-13T09:14:08.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:14:08.560 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.560 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.560 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.561 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.561 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.561 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:14:08.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.561 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:08.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:08.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:08.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:08.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:14:08.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:14:08.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:14:08.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:14:08.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:14:08.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:14:08.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:14:08.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:14:08.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:14:08.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:14:08.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:14:08.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:14:08.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:14:08.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:14:08.565 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.565 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.565 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.565 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.565 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:08.565 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.025350462999995, 2024-09-13T09:14:08.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:14:08.565 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.565 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.565 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.566 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.566 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.566 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:14:08.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.566 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:08.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:08.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:08.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:08.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:14:08.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:14:08.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:14:08.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:14:08.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:14:08.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:14:08.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:14:08.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:14:08.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:14:08.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:14:08.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:14:08.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:14:08.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:14:08.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:14:08.569 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.569 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.570 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.570 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.570 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:08.570 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.025320801999996, 2024-09-13T09:14:08.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:14:08.570 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.570 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.570 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.570 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.570 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.570 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:14:08.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.571 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:08.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:08.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:08.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:08.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:14:08.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:14:08.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:14:08.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:14:08.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:14:08.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:14:08.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:14:08.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:14:08.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:14:08.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:14:08.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:14:08.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:14:08.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:14:08.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:14:08.574 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.574 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.574 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.575 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.575 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:08.575 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.025291594999999, 2024-09-13T09:14:08.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:14:08.575 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.575 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.575 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.575 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.575 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.575 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:14:08.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.575 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:08.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:08.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:08.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:08.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:14:08.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:14:08.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:14:08.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:14:08.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:14:08.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:14:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:14:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:14:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:14:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:14:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:14:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:14:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:14:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:14:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.025235413999994, 2024-09-13T09:14:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:14:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:14:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:14:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:14:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:14:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:14:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:14:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:14:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:14:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:14:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:14:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:14:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:14:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:14:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:14:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:14:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.025181953000001, 2024-09-13T09:14:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:14:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:14:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:14:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:14:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:14:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:14:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:14:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:14:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:14:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:14:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:14:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:14:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:14:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:14:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:14:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:14:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:14:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.025125122999995, 2024-09-13T09:14:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:14:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:14:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:14:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:14:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:14:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:14:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:14:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:14:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:14:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:14:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:14:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:14:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:14:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:14:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:14:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:14:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:14:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.025074095999997, 2024-09-13T09:14:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:14:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:14:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:14:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:14:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:14:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:14:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:14:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:14:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:14:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:14:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:14:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:14:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:14:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:14:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:14:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:14:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:14:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.025032906000007, 2024-09-13T09:14:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:14:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:14:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:14:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:14:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:14:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:14:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:14:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:14:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:14:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:14:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:14:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:14:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:14:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:14:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:14:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:14:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.024990790000004, 2024-09-13T09:14:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:14:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:14:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:14:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:14:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:14:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:14:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:14:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:14:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:14:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:14:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:14:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:14:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:14:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:14:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:14:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:14:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.024946911000001, 2024-09-13T09:14:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:14:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:14:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:14:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:14:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:14:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:14:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:14:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:14:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:14:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:14:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:14:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:14:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:14:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:14:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:14:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:14:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.024900994000006, 2024-09-13T09:14:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:14:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:14:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:14:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:14:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:14:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:14:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:14:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:14:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:14:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:14:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:14:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:14:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:14:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:14:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:14:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:14:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:14:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.024851222999999, 2024-09-13T09:14:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:14:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:14:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:14:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:14:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:14:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:14:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:14:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:14:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:14:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:14:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:14:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:14:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:14:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:14:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:14:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:14:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.024805818999994, 2024-09-13T09:14:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:14:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:14:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:14:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:14:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:14:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:14:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:14:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:14:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:14:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:14:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:14:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:14:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:14:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:14:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:14:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:14:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.024755181000003, 2024-09-13T09:14:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:14:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:14:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:14:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:14:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:14:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:14:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:14:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:14:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:14:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:14:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:14:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:14:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:14:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:14:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:14:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:14:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:14:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.024699338000005, 2024-09-13T09:14:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:14:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:14:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:14:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:14:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:14:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:14:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:14:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:14:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:14:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:14:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:14:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:14:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:14:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:14:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:14:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:14:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.024642416000006, 2024-09-13T09:14:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:14:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:14:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:14:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:14:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:14:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:14:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:14:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:14:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:14:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:14:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:14:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:14:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:14:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:14:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:14:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:14:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 86.024588852999997, 2024-09-13T09:14:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:14:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:14:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:14:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:14:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:14:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:14:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:14:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:14:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:14:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:14:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:14:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:14:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:14:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:14:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:14:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:14:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:08.652 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:08.654 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:14:08.654 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:08.652+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:08.701 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-09-13T09:14:08.754 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:08.753+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:08.805 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:08.805 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:08.805 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:08.805 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:08.814 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:14:08.857 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:08.857+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:08.919 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:08.919 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:08.919 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:08.919 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:08.919 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:08.929 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:14:08.970 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:08.969+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:09.058 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:09.057+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:09.073 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:09.073 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:09.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:09.073 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:09.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.074 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2185 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.074 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:09.074 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545927581999997, 2024-09-13T09:14:09.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063348994000000006, 2024-09-13T09:14:09.074 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.074 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.074 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.074 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.074 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.074 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2185 2024-09-13T09:14:09.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.074 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:09.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:09.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:09.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525765+0000", 2024-09-13T09:14:09.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2390000000000001e-06 2024-09-13T09:14:09.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525772+0000", 2024-09-13T09:14:09.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:14:09.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525777+0000", 2024-09-13T09:14:09.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0799999999999996e-06 2024-09-13T09:14:09.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589020+0000", 2024-09-13T09:14:09.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063242501000000007 2024-09-13T09:14:09.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589063+0000", 2024-09-13T09:14:09.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2673999999999998e-05 2024-09-13T09:14:09.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589112+0000", 2024-09-13T09:14:09.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9286000000000002e-05 2024-09-13T09:14:09.078 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.078 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.078 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.079 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2187 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.079 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:09.079 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545908034999997, 2024-09-13T09:14:09.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063500102000000003, 2024-09-13T09:14:09.079 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.079 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.080 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2187 2024-09-13T09:14:09.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.080 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:09.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:09.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:09.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525784+0000", 2024-09-13T09:14:09.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1629999999999999e-06 2024-09-13T09:14:09.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525792+0000", 2024-09-13T09:14:09.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:14:09.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525796+0000", 2024-09-13T09:14:09.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7389999999999999e-06 2024-09-13T09:14:09.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589132+0000", 2024-09-13T09:14:09.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063336119999999996 2024-09-13T09:14:09.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589187+0000", 2024-09-13T09:14:09.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4126000000000003e-05 2024-09-13T09:14:09.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589283+0000", 2024-09-13T09:14:09.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6032999999999993e-05 2024-09-13T09:14:09.083 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.083 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.083 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.084 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2188 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.084 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:09.084 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545889989000003, 2024-09-13T09:14:09.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063724617999999997, 2024-09-13T09:14:09.084 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.084 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.084 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.084 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.084 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.084 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2188 2024-09-13T09:14:09.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.084 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:09.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:09.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:09.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525802+0000", 2024-09-13T09:14:09.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0439999999999998e-06 2024-09-13T09:14:09.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525809+0000", 2024-09-13T09:14:09.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.238e-06 2024-09-13T09:14:09.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525814+0000", 2024-09-13T09:14:09.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-13T09:14:09.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589327+0000", 2024-09-13T09:14:09.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063513257000000004 2024-09-13T09:14:09.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589418+0000", 2024-09-13T09:14:09.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0796000000000003e-05 2024-09-13T09:14:09.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589525+0000", 2024-09-13T09:14:09.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010693500000000001 2024-09-13T09:14:09.088 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.088 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.088 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.088 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2190 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.089 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:09.089 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545871902999998, 2024-09-13T09:14:09.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063863981, 2024-09-13T09:14:09.089 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.089 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.089 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.089 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.089 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.089 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2190 2024-09-13T09:14:09.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.089 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:09.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:09.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525818+0000", 2024-09-13T09:14:09.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525820+0000", 2024-09-13T09:14:09.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:14:09.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525827+0000", 2024-09-13T09:14:09.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0589999999999999e-06 2024-09-13T09:14:09.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525831+0000", 2024-09-13T09:14:09.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7870000000000001e-06 2024-09-13T09:14:09.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589562+0000", 2024-09-13T09:14:09.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063731038000000004 2024-09-13T09:14:09.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589612+0000", 2024-09-13T09:14:09.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0087999999999999e-05 2024-09-13T09:14:09.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589683+0000", 2024-09-13T09:14:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0121000000000002e-05 2024-09-13T09:14:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2192 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545854943000002, 2024-09-13T09:14:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064050299000000005, 2024-09-13T09:14:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.094 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.094 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.094 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2192 2024-09-13T09:14:09.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.094 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:09.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:09.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525835+0000", 2024-09-13T09:14:09.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525837+0000", 2024-09-13T09:14:09.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:14:09.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525845+0000", 2024-09-13T09:14:09.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3660000000000003e-06 2024-09-13T09:14:09.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525850+0000", 2024-09-13T09:14:09.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0030000000000002e-06 2024-09-13T09:14:09.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589720+0000", 2024-09-13T09:14:09.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063870176000000001 2024-09-13T09:14:09.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589801+0000", 2024-09-13T09:14:09.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0889000000000004e-05 2024-09-13T09:14:09.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589886+0000", 2024-09-13T09:14:09.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5183000000000001e-05 2024-09-13T09:14:09.097 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.097 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.097 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.097 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.098 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:09.098 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.54583684, 2024-09-13T09:14:09.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:14:09.098 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.098 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.098 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.098 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.098 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.098 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:14:09.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.098 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:09.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:09.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:14:09.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:14:09.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:14:09.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:14:09.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:14:09.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:14:09.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:14:09.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:14:09.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:14:09.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:14:09.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:14:09.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:14:09.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:14:09.101 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.102 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.102 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.102 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.102 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:09.102 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545819141999999, 2024-09-13T09:14:09.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:14:09.102 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.102 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.102 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:14:09.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.103 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:09.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:09.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:09.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:14:09.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:14:09.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:14:09.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:14:09.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:14:09.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:14:09.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:14:09.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:14:09.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:14:09.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:14:09.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:14:09.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:14:09.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.107 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.107 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.107 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:09.107 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545798527000002, 2024-09-13T09:14:09.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:14:09.107 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.108 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.108 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:14:09.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.108 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:09.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:09.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:09.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:14:09.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:14:09.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:14:09.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:14:09.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:14:09.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:14:09.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:14:09.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:14:09.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:14:09.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:14:09.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:14:09.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:14:09.111 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.111 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.112 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.112 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.112 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:09.112 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545779570999997, 2024-09-13T09:14:09.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:14:09.112 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.112 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.112 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.112 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.112 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.112 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:14:09.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.113 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:09.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:09.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:09.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:14:09.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:14:09.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:14:09.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:14:09.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:14:09.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:14:09.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:14:09.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:14:09.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:14:09.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:14:09.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:14:09.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:14:09.116 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.116 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.116 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.116 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.116 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:09.116 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545762312999997, 2024-09-13T09:14:09.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:14:09.117 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.117 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.117 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.117 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.117 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.117 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:14:09.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.117 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:09.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:09.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:09.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:14:09.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:14:09.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:14:09.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:14:09.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:14:09.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:14:09.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:14:09.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:14:09.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:14:09.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:14:09.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:14:09.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:14:09.120 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.121 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.121 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.121 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.121 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:09.121 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545742298999997, 2024-09-13T09:14:09.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:14:09.121 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.121 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.122 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:14:09.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.122 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:09.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:09.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:09.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:14:09.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:14:09.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:14:09.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:14:09.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:14:09.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:14:09.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:14:09.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:14:09.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:14:09.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:14:09.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:14:09.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:14:09.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.125 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.125 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.126 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:09.126 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545716716999998, 2024-09-13T09:14:09.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:14:09.126 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.126 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.126 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:14:09.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.126 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:09.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:09.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:14:09.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:14:09.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:14:09.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:14:09.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:14:09.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:14:09.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:14:09.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:14:09.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:14:09.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:14:09.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:14:09.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.130 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.130 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:14:09.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:14:09.130 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.130 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.130 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.130 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.130 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:09.130 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545697433000001, 2024-09-13T09:14:09.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:14:09.130 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.131 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.131 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.131 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.131 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.131 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:14:09.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.131 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:09.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:09.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:09.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:14:09.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:14:09.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:14:09.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:14:09.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:14:09.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:14:09.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:14:09.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:14:09.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:14:09.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:14:09.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:14:09.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:14:09.134 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.134 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.134 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.135 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.135 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:09.135 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545680177000001, 2024-09-13T09:14:09.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:14:09.135 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.135 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.135 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.135 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.135 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.135 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:14:09.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.135 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:09.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:09.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:09.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:14:09.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:14:09.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:14:09.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:14:09.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:14:09.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:14:09.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:14:09.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:14:09.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:14:09.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:14:09.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:14:09.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:14:09.139 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.139 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.139 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.139 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.139 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:09.139 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545660697000002, 2024-09-13T09:14:09.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:14:09.140 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.140 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.140 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.140 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.140 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.140 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:14:09.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.140 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:09.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:09.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:14:09.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:14:09.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:14:09.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:14:09.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:14:09.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:14:09.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:14:09.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:14:09.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:14:09.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:14:09.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:14:09.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:14:09.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:14:09.143 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.143 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.143 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.144 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.144 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:09.144 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545643767999998, 2024-09-13T09:14:09.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:14:09.144 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.144 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.144 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:14:09.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.144 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:09.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:09.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:14:09.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:14:09.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:14:09.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:14:09.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:14:09.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:14:09.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:14:09.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:14:09.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:14:09.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:14:09.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:14:09.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:14:09.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:14:09.148 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.148 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.148 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.148 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.149 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:09.149 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545624619000002, 2024-09-13T09:14:09.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:14:09.149 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.149 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.149 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.149 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.149 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.149 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:14:09.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.149 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:09.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:09.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:09.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:14:09.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:14:09.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:14:09.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:14:09.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:14:09.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:14:09.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:14:09.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:14:09.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:14:09.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:14:09.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:14:09.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:14:09.153 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.153 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.153 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.153 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.153 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:09.153 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545606480000004, 2024-09-13T09:14:09.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:14:09.153 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.153 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.154 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.154 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.154 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.154 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:14:09.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.154 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:09.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:09.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:09.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:14:09.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:14:09.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:14:09.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:14:09.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:14:09.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:14:09.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:14:09.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:14:09.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:14:09.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:14:09.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:14:09.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:14:09.157 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.157 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.157 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.158 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.158 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:09.158 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545195984999999, 2024-09-13T09:14:09.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:14:09.158 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.158 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.158 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.158 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.158 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.158 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:14:09.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.158 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:09.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:09.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:09.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:14:09.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:14:09.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:14:09.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:14:09.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:14:09.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:14:09.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:14:09.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:14:09.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:14:09.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:14:09.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:14:09.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:14:09.162 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.162 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.162 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.162 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:09.162 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:09.162 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.545173083999998, 2024-09-13T09:14:09.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:14:09.162 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.163 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.163 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.163 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:09.163 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:09.163 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:14:09.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.163 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:09.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:09.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:09.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:14:09.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:14:09.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:14:09.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:14:09.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:14:09.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:14:09.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:14:09.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:14:09.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:14:09.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:14:09.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:14:09.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:14:09.166 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.166 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.166 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.167 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.167 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.167 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:09.168 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:14:09.169 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:09.167+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:09.268 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:09.268+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:09.314 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:09.314 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:09.314 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:09.314 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:09.323 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:14:09.369 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:09.369+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:09.466 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:09.466 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:09.466 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:09.466 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:09.466 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:09.470 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:09.469+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:09.474 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:14:09.572 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:09.571+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:09.579 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:09.579 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:09.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:09.579 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:09.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.579 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:09.579 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:09.580 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.831463141, 2024-09-13T09:14:09.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:14:09.580 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.580 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:09.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:09.580 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:14:09.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.580 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:09.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:09.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:09.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:14:09.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:14:09.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:14:09.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:14:09.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:14:09.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:14:09.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:14:09.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:14:09.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:14:09.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:14:09.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:14:09.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:14:09.584 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.584 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.584 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.584 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:09.584 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:09.584 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.831454463, 2024-09-13T09:14:09.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:14:09.584 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.584 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.584 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.585 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:09.585 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:09.585 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:14:09.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.585 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:09.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:09.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:09.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:09.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:14:09.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:14:09.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:14:09.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:14:09.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:14:09.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:14:09.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:14:09.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:14:09.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:14:09.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:14:09.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:14:09.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:14:09.588 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.588 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.589 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.589 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:09.589 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:09.589 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.831445477000003, 2024-09-13T09:14:09.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:14:09.589 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.589 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:09.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:09.589 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:14:09.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.590 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:09.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:09.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:09.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:14:09.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:14:09.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:14:09.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:14:09.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:14:09.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:14:09.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:14:09.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:14:09.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:14:09.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:14:09.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:14:09.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:14:09.593 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.593 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.593 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.594 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:09.594 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:09.594 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.831437504999997, 2024-09-13T09:14:09.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:14:09.594 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.594 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.594 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.594 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:09.594 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:09.594 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:14:09.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.595 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:09.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:09.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:09.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:09.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:14:09.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:14:09.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:14:09.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:14:09.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:14:09.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:14:09.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:14:09.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:14:09.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:14:09.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:14:09.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:14:09.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:14:09.598 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.598 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.598 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.598 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:09.598 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:09.598 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.829562671000005, 2024-09-13T09:14:09.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:14:09.599 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.599 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.599 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.599 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:09.599 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:09.599 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:14:09.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.599 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:09.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:09.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:09.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:14:09.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:14:09.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:14:09.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:14:09.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:14:09.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:14:09.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:14:09.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:14:09.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:14:09.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:14:09.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:14:09.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:14:09.602 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.603 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.603 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.603 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:09.603 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:09.603 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.82955201, 2024-09-13T09:14:09.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:14:09.603 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.603 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.603 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.603 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:09.603 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:09.604 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:14:09.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.604 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:09.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:09.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:09.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:09.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:14:09.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:14:09.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:14:09.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:14:09.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:14:09.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:14:09.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:14:09.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:14:09.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:14:09.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:14:09.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:14:09.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:14:09.607 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.607 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.607 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.607 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:09.608 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:09.608 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.829530034000001, 2024-09-13T09:14:09.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:14:09.608 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.608 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.608 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.608 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:09.608 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:09.608 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:14:09.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.608 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:09.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:09.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:09.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:14:09.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:14:09.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:14:09.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:14:09.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:14:09.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:14:09.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:14:09.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:14:09.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:14:09.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:14:09.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:14:09.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:14:09.612 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.612 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.612 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.612 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:09.612 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:09.612 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.829516532, 2024-09-13T09:14:09.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:14:09.612 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.612 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.613 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.613 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:09.613 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:09.613 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:14:09.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.613 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:09.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:09.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:09.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:14:09.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:14:09.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:14:09.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:14:09.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:14:09.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:14:09.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:14:09.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:14:09.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:14:09.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:14:09.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:14:09.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:14:09.618 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.618 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.618 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.618 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:09.619 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:09.619 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.827507433999997, 2024-09-13T09:14:09.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:14:09.619 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.619 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:09.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:09.619 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:14:09.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.619 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:09.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:09.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:09.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:14:09.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:14:09.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:14:09.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:14:09.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:14:09.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:14:09.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:14:09.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:14:09.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:14:09.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:14:09.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:14:09.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:14:09.623 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.623 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.623 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.623 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:09.623 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:09.623 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.827483111999996, 2024-09-13T09:14:09.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:14:09.623 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.623 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.623 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.623 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:09.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:09.624 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:14:09.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.624 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:09.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:09.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:09.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:14:09.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:14:09.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:14:09.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:14:09.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:14:09.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:14:09.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:14:09.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:14:09.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:14:09.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:14:09.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:14:09.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:14:09.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.627 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.627 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:09.628 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:14:09.628 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.827472138000005, 2024-09-13T09:14:09.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:14:09.628 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.628 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:09.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:09.628 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:14:09.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.628 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:14:09.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:14:09.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:14:09.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:14:09.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:14:09.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:14:09.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:14:09.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:14:09.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:14:09.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:14:09.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:14:09.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:14:09.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:14:09.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:14:09.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:14:09.632 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.632 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.632 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.632 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:09.632 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:09.632 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.825554513, 2024-09-13T09:14:09.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:14:09.632 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.632 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:09.633 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:09.633 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:14:09.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.633 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:09.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:09.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:09.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:14:09.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:14:09.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:14:09.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:14:09.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:14:09.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:14:09.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:14:09.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:14:09.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:14:09.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:14:09.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:14:09.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:14:09.636 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.636 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.636 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.636 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:09.637 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:09.637 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.825545116000001, 2024-09-13T09:14:09.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:14:09.637 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.637 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:09.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:09.637 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:14:09.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.637 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:09.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:09.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:09.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:09.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:14:09.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:14:09.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:14:09.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:14:09.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:14:09.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:14:09.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:14:09.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:14:09.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:14:09.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:14:09.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:14:09.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:14:09.641 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.641 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.641 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.641 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3181 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T09:14:09.641 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:14:09.641 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.55518094, 2024-09-13T09:14:09.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.480957603, 2024-09-13T09:14:09.641 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.641 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:09.641 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.642 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:14:09.642 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:14:09.642 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3181 2024-09-13T09:14:09.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.642 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:14:09.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:14:09.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021894+0000", 2024-09-13T09:14:09.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:14:09.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021903+0000", 2024-09-13T09:14:09.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4109999999999996e-06 2024-09-13T09:14:09.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021919+0000", 2024-09-13T09:14:09.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6225000000000001e-05 2024-09-13T09:14:09.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021926+0000", 2024-09-13T09:14:09.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4120000000000004e-06 2024-09-13T09:14:09.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021975+0000", 2024-09-13T09:14:09.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8526000000000003e-05 2024-09-13T09:14:09.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022049+0000", 2024-09-13T09:14:09.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4018999999999997e-05 2024-09-13T09:14:09.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:14:09.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022230+0000", 2024-09-13T09:14:09.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018085599999999999 2024-09-13T09:14:09.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:09.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022796+0000", 2024-09-13T09:14:09.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056638999999999995 2024-09-13T09:14:09.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499540+0000", 2024-09-13T09:14:09.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.47674342800000002 2024-09-13T09:14:09.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:09.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499547+0000", 2024-09-13T09:14:09.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:14:09.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502794+0000", 2024-09-13T09:14:09.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.003246818 2024-09-13T09:14:09.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502854+0000", 2024-09-13T09:14:09.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9925e-05 2024-09-13T09:14:09.647 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.647 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.647 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.647 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3182 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 139818970156544 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:14:09.647 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:14:09.647 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.081946385000002, 2024-09-13T09:14:09.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0089058390000001, 2024-09-13T09:14:09.648 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.648 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:14:09.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:14:09.648 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3182 2024-09-13T09:14:09.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.648 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:14:09.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:14:09.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495128+0000", 2024-09-13T09:14:09.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:14:09.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495139+0000", 2024-09-13T09:14:09.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0599e-05 2024-09-13T09:14:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495157+0000", 2024-09-13T09:14:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7830999999999999e-05 2024-09-13T09:14:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495167+0000", 2024-09-13T09:14:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0689999999999999e-05 2024-09-13T09:14:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495229+0000", 2024-09-13T09:14:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1898999999999998e-05 2024-09-13T09:14:09.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:14:09.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495243+0000", 2024-09-13T09:14:09.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3349e-05 2024-09-13T09:14:09.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499575+0000", 2024-09-13T09:14:09.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043324410000000002 2024-09-13T09:14:09.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:09.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499582+0000", 2024-09-13T09:14:09.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0849999999999999e-06 2024-09-13T09:14:09.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502872+0000", 2024-09-13T09:14:09.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0032895150000001 2024-09-13T09:14:09.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503016+0000", 2024-09-13T09:14:09.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014394200000000001 2024-09-13T09:14:09.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504036+0000", 2024-09-13T09:14:09.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010208680000000001 2024-09-13T09:14:09.653 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.653 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.653 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.653 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:19151 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 140383089882784 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:14:09.653 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:14:09.653 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.081934926000002, 2024-09-13T09:14:09.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.009071287, 2024-09-13T09:14:09.653 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.653 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:09.654 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.654 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:09.654 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:09.654 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19151 2024-09-13T09:14:09.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.654 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:14:09.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:14:09.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495140+0000", 2024-09-13T09:14:09.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:14:09.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495149+0000", 2024-09-13T09:14:09.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0759999999999992e-06 2024-09-13T09:14:09.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495165+0000", 2024-09-13T09:14:09.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5322e-05 2024-09-13T09:14:09.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495173+0000", 2024-09-13T09:14:09.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0979999999999994e-06 2024-09-13T09:14:09.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495318+0000", 2024-09-13T09:14:09.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014553 2024-09-13T09:14:09.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:14:09.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495325+0000", 2024-09-13T09:14:09.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:14:09.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499598+0000", 2024-09-13T09:14:09.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004273064 2024-09-13T09:14:09.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:09.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499602+0000", 2024-09-13T09:14:09.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8539999999999999e-06 2024-09-13T09:14:09.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503507+0000", 2024-09-13T09:14:09.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0039045520000001 2024-09-13T09:14:09.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503583+0000", 2024-09-13T09:14:09.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6283000000000001e-05 2024-09-13T09:14:09.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504213+0000", 2024-09-13T09:14:09.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00063044700000000002 2024-09-13T09:14:09.660 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.660 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.660 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.660 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3908 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:09.660 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:14:09.660 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 21.484398035000002, 2024-09-13T09:14:09.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070530317999999995, 2024-09-13T09:14:09.660 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.661 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:09.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:09.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:09.661 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3908 2024-09-13T09:14:09.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.661 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:14:09.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:14:09.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092678+0000", 2024-09-13T09:14:09.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092681+0000", 2024-09-13T09:14:09.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.903e-06 2024-09-13T09:14:09.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092687+0000", 2024-09-13T09:14:09.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0859999999999998e-06 2024-09-13T09:14:09.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092692+0000", 2024-09-13T09:14:09.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-13T09:14:09.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.159711+0000", 2024-09-13T09:14:09.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067018922999999994 2024-09-13T09:14:09.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.159796+0000", 2024-09-13T09:14:09.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5247000000000006e-05 2024-09-13T09:14:09.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:09.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160111+0000", 2024-09-13T09:14:09.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00031449499999999998 2024-09-13T09:14:09.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:09.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162219+0000", 2024-09-13T09:14:09.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021085570000000001 2024-09-13T09:14:09.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:09.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163127+0000", 2024-09-13T09:14:09.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00090723400000000004 2024-09-13T09:14:09.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163172+0000", 2024-09-13T09:14:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5383000000000002e-05 2024-09-13T09:14:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163209+0000", 2024-09-13T09:14:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7132000000000003e-05 2024-09-13T09:14:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3909 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:09.667 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:09.667 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 21.484380544, 2024-09-13T09:14:09.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070616834000000003, 2024-09-13T09:14:09.667 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.667 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:09.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:09.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:09.667 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3909 2024-09-13T09:14:09.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.667 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:09.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:09.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:09.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092699+0000", 2024-09-13T09:14:09.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:14:09.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092704+0000", 2024-09-13T09:14:09.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5940000000000001e-06 2024-09-13T09:14:09.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092709+0000", 2024-09-13T09:14:09.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5109999999999997e-06 2024-09-13T09:14:09.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160604+0000", 2024-09-13T09:14:09.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067895633999999996 2024-09-13T09:14:09.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160658+0000", 2024-09-13T09:14:09.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3292999999999999e-05 2024-09-13T09:14:09.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:09.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160935+0000", 2024-09-13T09:14:09.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000276879 2024-09-13T09:14:09.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:09.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163223+0000", 2024-09-13T09:14:09.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022886320000000001 2024-09-13T09:14:09.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:09.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163258+0000", 2024-09-13T09:14:09.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5101000000000002e-05 2024-09-13T09:14:09.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:09.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163292+0000", 2024-09-13T09:14:09.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3237e-05 2024-09-13T09:14:09.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163313+0000", 2024-09-13T09:14:09.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1642e-05 2024-09-13T09:14:09.672 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.672 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.672 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.673 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3910 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:09.673 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:09.673 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 21.484363479999999, 2024-09-13T09:14:09.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733837999999993, 2024-09-13T09:14:09.673 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.673 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:09.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:09.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:09.673 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3910 2024-09-13T09:14:09.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.673 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:09.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:09.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:09.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092716+0000", 2024-09-13T09:14:09.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.757e-06 2024-09-13T09:14:09.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092722+0000", 2024-09-13T09:14:09.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7869999999999998e-06 2024-09-13T09:14:09.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092726+0000", 2024-09-13T09:14:09.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.622e-06 2024-09-13T09:14:09.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161487+0000", 2024-09-13T09:14:09.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068761166999999998 2024-09-13T09:14:09.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161569+0000", 2024-09-13T09:14:09.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1185000000000003e-05 2024-09-13T09:14:09.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:09.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161868+0000", 2024-09-13T09:14:09.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029979799999999999 2024-09-13T09:14:09.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:09.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163366+0000", 2024-09-13T09:14:09.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001497888 2024-09-13T09:14:09.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:09.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163373+0000", 2024-09-13T09:14:09.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1640000000000004e-06 2024-09-13T09:14:09.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:09.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163412+0000", 2024-09-13T09:14:09.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8423999999999997e-05 2024-09-13T09:14:09.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163447+0000", 2024-09-13T09:14:09.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5441000000000002e-05 2024-09-13T09:14:09.678 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.678 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.678 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.678 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3911 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:09.678 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:09.678 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 21.484343564, 2024-09-13T09:14:09.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.072284372999999999, 2024-09-13T09:14:09.679 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:09.679 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:09.679 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:09.679 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:09.679 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:09.679 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3911 2024-09-13T09:14:09.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.679 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:09.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:09.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:09.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:09.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:09.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:09.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:09.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:09.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:09.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:09.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092736+0000", 2024-09-13T09:14:09.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9510000000000002e-06 2024-09-13T09:14:09.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:09.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092743+0000", 2024-09-13T09:14:09.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9e-06 2024-09-13T09:14:09.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092748+0000", 2024-09-13T09:14:09.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999998e-06 2024-09-13T09:14:09.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:09.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162257+0000", 2024-09-13T09:14:09.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.069508890000000004 2024-09-13T09:14:09.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:09.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162342+0000", 2024-09-13T09:14:09.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5886999999999999e-05 2024-09-13T09:14:09.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:09.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162682+0000", 2024-09-13T09:14:09.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00033971700000000001 2024-09-13T09:14:09.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:09.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164897+0000", 2024-09-13T09:14:09.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022152370000000001 2024-09-13T09:14:09.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:09.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164945+0000", 2024-09-13T09:14:09.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7243999999999997e-05 2024-09-13T09:14:09.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:09.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164985+0000", 2024-09-13T09:14:09.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0599000000000002e-05 2024-09-13T09:14:09.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:09.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:09.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:09.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.165018+0000", 2024-09-13T09:14:09.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2549000000000003e-05 2024-09-13T09:14:09.684 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.684 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.684 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.684 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:09.684 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:09.684 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:09.686 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:09.685+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:09.686 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:14:09.787 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:09.786+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:09.811 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:09.811 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:09.811 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:09.811 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:28433 3.12 3.8d738e12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-13T09:14:09.811 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:09.810451+0000", 2024-09-13T09:14:09.811 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.4517000000000001e-05, 2024-09-13T09:14:09.811 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0207000000000001e-05, 2024-09-13T09:14:09.811 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:09.811 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "queued for pg", 2024-09-13T09:14:09.811 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:09.812 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:09.812 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:09.812 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 28433 2024-09-13T09:14:09.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:09.812 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:09.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:09.812 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:09.812 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:09.810451+0000", 2024-09-13T09:14:09.812 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:09.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:09.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:09.813 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:09.813 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:09.810451+0000", 2024-09-13T09:14:09.813 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:09.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:09.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:09.813 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:09.813 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:09.810450+0000", 2024-09-13T09:14:09.813 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:09.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:09.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:09.814 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:09.814 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:09.810455+0000", 2024-09-13T09:14:09.814 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7559999999999999e-06 2024-09-13T09:14:09.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:09.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:09.814 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:09.814 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:09.810467+0000", 2024-09-13T09:14:09.814 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1270999999999999e-05 2024-09-13T09:14:09.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:09.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:09.814 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:09.815 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:09.810472+0000", 2024-09-13T09:14:09.815 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.062e-06 2024-09-13T09:14:09.815 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:09.815 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:09.815 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:09.815 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:09.815 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:14:09.816 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 1 2024-09-13T09:14:09.816 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:09.821 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:14:09.831 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5] 2024-09-13T09:14:09.831 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:14:09.831 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-13T09:14:09.831 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T09:14:09.831 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T09:14:09.866 INFO:tasks.ceph.osd.2:Started 2024-09-13T09:14:09.866 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:14:09.910 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:09.909+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:09.935 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:09.935 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:09.935 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:09.935 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:09.935 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:09.945 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:14:09.975 INFO:teuthology.orchestra.run.smithi044.stderr:no valid command found; 10 closest matches: 2024-09-13T09:14:09.975 INFO:teuthology.orchestra.run.smithi044.stderr:0 2024-09-13T09:14:09.975 INFO:teuthology.orchestra.run.smithi044.stderr:1 2024-09-13T09:14:09.975 INFO:teuthology.orchestra.run.smithi044.stderr:2 2024-09-13T09:14:09.976 INFO:teuthology.orchestra.run.smithi044.stderr:abort 2024-09-13T09:14:09.976 INFO:teuthology.orchestra.run.smithi044.stderr:assert 2024-09-13T09:14:09.976 INFO:teuthology.orchestra.run.smithi044.stderr:config diff 2024-09-13T09:14:09.976 INFO:teuthology.orchestra.run.smithi044.stderr:config diff get 2024-09-13T09:14:09.976 INFO:teuthology.orchestra.run.smithi044.stderr:config get 2024-09-13T09:14:09.976 INFO:teuthology.orchestra.run.smithi044.stderr:config help [] 2024-09-13T09:14:09.976 INFO:teuthology.orchestra.run.smithi044.stderr:config set ... 2024-09-13T09:14:09.976 INFO:teuthology.orchestra.run.smithi044.stderr:admin_socket: invalid command 2024-09-13T09:14:09.978 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T09:14:09.979 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-13T09:14:10.020 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:10.019+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:10.084 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:10.084 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:10.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:10.085 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:10.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.085 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.085 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:10.085 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.553056765999997, 2024-09-13T09:14:10.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:14:10.085 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.085 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.085 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.085 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.085 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.085 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:14:10.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.086 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:10.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:10.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:14:10.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.087 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.087 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:14:10.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:14:10.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:14:10.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:14:10.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:14:10.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:14:10.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:14:10.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:14:10.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:14:10.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:14:10.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:14:10.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:14:10.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:14:10.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:14:10.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:14:10.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:14:10.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:14:10.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:14:10.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:14:10.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:14:10.091 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.091 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.091 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.091 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.091 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:10.092 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.553038731000001, 2024-09-13T09:14:10.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:14:10.092 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.092 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.092 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.092 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.092 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.092 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:14:10.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.093 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.093 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:10.093 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.093 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:10.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:10.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:14:10.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:14:10.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:14:10.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:14:10.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:14:10.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:14:10.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:14:10.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:14:10.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:14:10.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:14:10.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:14:10.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:14:10.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:14:10.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:14:10.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:14:10.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:14:10.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.098 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.098 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:14:10.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:14:10.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.098 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.098 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:14:10.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:14:10.098 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.098 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.098 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.099 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.099 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:10.099 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.553022060999993, 2024-09-13T09:14:10.099 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:14:10.099 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.099 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.099 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.099 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.099 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.100 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:14:10.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.100 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.100 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:10.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.100 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:10.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:14:10.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:14:10.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:10.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:14:10.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:14:10.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:14:10.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:14:10.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:14:10.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:14:10.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.103 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:14:10.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:14:10.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.104 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.104 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:14:10.104 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:14:10.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.104 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.104 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:14:10.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:14:10.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.105 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.105 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:14:10.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:14:10.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.105 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.105 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:14:10.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:14:10.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:14:10.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:14:10.106 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.106 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.106 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.106 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.106 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:10.107 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.553004428999998, 2024-09-13T09:14:10.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:14:10.107 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.107 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.107 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.107 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.107 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.107 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:14:10.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.107 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:10.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:10.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:10.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.109 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:14:10.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:14:10.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.109 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:14:10.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:14:10.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.109 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:14:10.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:14:10.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.110 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.110 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:14:10.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:14:10.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.110 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.110 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:14:10.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:14:10.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.110 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:14:10.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:14:10.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:14:10.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:14:10.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:14:10.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:14:10.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.112 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:14:10.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:14:10.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.112 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:14:10.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:14:10.112 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.112 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.112 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.113 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.113 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:10.113 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.552958124, 2024-09-13T09:14:10.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:14:10.114 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.114 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.114 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.114 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.114 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.114 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:14:10.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.114 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:10.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:10.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:14:10.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:14:10.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:14:10.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.116 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.116 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:14:10.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:14:10.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.116 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.116 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:14:10.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:14:10.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.116 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.117 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:14:10.117 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:14:10.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.117 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.117 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:14:10.117 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:14:10.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.117 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.117 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:14:10.117 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:14:10.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:14:10.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:14:10.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:14:10.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:14:10.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:14:10.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:14:10.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:14:10.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:14:10.119 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.119 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.119 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.119 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.119 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:10.119 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.552932823000006, 2024-09-13T09:14:10.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:14:10.120 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.120 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.120 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.120 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.120 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.120 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:14:10.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.120 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:10.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:10.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:14:10.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:14:10.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:14:10.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:14:10.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:14:10.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:14:10.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:14:10.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:14:10.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:14:10.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:14:10.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:14:10.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.124 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:14:10.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:14:10.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.124 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.124 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:14:10.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:14:10.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:14:10.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:14:10.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:14:10.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:14:10.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:14:10.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:14:10.126 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.126 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.126 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.126 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.126 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:10.126 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.552914594000001, 2024-09-13T09:14:10.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:14:10.126 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.126 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.127 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.127 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.127 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.127 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:14:10.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.127 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:10.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:10.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:10.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:14:10.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:14:10.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:14:10.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:14:10.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:14:10.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:14:10.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:14:10.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:14:10.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.130 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:14:10.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:14:10.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.130 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:14:10.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:14:10.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:14:10.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:14:10.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:14:10.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:14:10.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:14:10.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:14:10.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:14:10.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:14:10.132 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.132 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.132 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.132 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.132 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:10.132 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.552867274999997, 2024-09-13T09:14:10.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:14:10.133 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.133 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.133 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.133 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.133 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.133 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:14:10.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.133 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:10.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.134 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:10.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.134 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:10.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.134 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:14:10.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:14:10.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.135 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:14:10.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:14:10.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.135 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:14:10.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:14:10.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:14:10.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:14:10.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:14:10.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:14:10.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:14:10.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:14:10.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:14:10.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:14:10.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:14:10.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:14:10.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.138 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:14:10.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:14:10.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.138 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.138 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:14:10.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:14:10.138 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.138 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.138 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.138 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.139 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:10.139 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.552353037000003, 2024-09-13T09:14:10.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:14:10.139 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.139 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.139 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.139 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.139 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.139 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:14:10.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.139 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:14:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:14:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:14:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:14:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:14:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:14:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:14:10.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:14:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:14:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:14:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:14:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:14:10.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:14:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:14:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:14:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:14:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:14:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:14:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:14:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:14:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:14:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.144 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.552318506999995, 2024-09-13T09:14:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:14:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:14:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:10.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:14:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:14:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:14:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:14:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:14:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:14:10.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:14:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:14:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:14:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:14:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:14:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:14:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:14:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:14:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:14:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:14:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:14:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:14:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:14:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:14:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.150 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.552300599999995, 2024-09-13T09:14:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:14:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:14:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:14:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:14:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:14:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:14:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:14:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:14:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:14:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:14:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:14:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:14:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:14:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:14:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:14:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:14:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:14:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:14:10.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:14:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:14:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:14:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:14:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.543173752000001, 2024-09-13T09:14:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:14:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:14:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:14:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:14:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:14:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:14:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:14:10.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:14:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:14:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:14:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:14:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:14:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:14:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:14:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:14:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:14:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:14:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:14:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:14:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:14:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:14:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:14:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.543122171999997, 2024-09-13T09:14:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:14:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:14:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:14:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:14:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:14:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:14:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:14:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:14:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:14:10.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:14:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:14:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:14:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:14:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:14:10.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:14:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:14:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:14:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:14:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:14:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:14:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:14:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:14:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:14:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:10.168 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.543105896, 2024-09-13T09:14:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:14:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:14:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:14:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:14:10.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:14:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:14:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:14:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:14:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:14:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:14:10.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:14:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:14:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:14:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:14:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:14:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:14:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:14:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:14:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:14:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:14:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:14:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:14:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:14:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:10.174 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.543090121999995, 2024-09-13T09:14:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:14:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:14:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:10.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:14:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:14:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:14:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:14:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:14:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:14:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:14:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:14:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:14:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:14:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:14:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:14:10.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:14:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:14:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:14:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:14:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:14:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:14:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:14:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:14:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:14:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.543075247000004, 2024-09-13T09:14:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:14:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.181 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:14:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:14:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:14:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:14:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:14:10.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:14:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:14:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:14:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:14:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:14:10.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:14:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:14:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:14:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:14:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:14:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:14:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:14:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:14:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:14:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:14:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:14:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.186 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.543057785000002, 2024-09-13T09:14:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:14:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:14:10.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:14:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:14:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:14:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:14:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:14:10.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:14:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:14:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:14:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:14:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:14:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:14:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:14:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:14:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:14:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:14:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:14:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:14:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:14:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:14:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:14:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:14:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.192 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.541457152999996, 2024-09-13T09:14:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:14:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.193 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:14:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:14:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:14:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:14:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:14:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:14:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:14:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:14:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:14:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:14:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:14:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:14:10.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:14:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:14:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:14:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:14:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:14:10.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:14:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:14:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:14:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:14:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:14:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.198 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.541417101999997, 2024-09-13T09:14:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:14:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:14:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.199 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:14:10.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:14:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:14:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:14:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:14:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:14:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:14:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:14:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:14:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:14:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:14:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:14:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:14:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:14:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:14:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:14:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:14:10.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:14:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:14:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:14:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:14:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 87.541373339000003, 2024-09-13T09:14:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:14:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:14:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.205 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:10.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:14:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:14:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:14:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:14:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:14:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:14:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:14:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:14:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:14:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:14:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:14:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:14:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:14:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:14:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:14:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:14:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:14:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:14:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:14:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:14:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.211 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.211 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:10.213 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:14:10.213 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:10.212+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:10.313 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:10.312+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:10.361 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:10.361 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:10.362 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:10.362 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:10.370 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:14:10.415 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:10.414+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:10.470 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:10.469+0000 7fa389c24640 -1 Falling back to public interface 2024-09-13T09:14:10.477 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:10.477 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:10.477 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:10.487 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:14:10.515 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:10.514+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:10.546 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:14:10.615 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:10.615+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:10.638 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.132188313, 2024-09-13T09:14:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:14:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:14:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:14:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:14:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:14:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:14:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:14:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:14:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:14:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:14:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:14:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:14:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:14:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:14:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:14:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.132160628999998, 2024-09-13T09:14:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:14:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:14:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:14:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:14:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:14:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:14:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:14:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:14:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:14:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:14:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:14:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:14:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:14:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:14:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.132136287999998, 2024-09-13T09:14:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:14:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:14:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:14:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:14:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:14:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:14:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:14:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:14:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:14:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:14:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:14:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:14:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:14:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:14:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:14:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.132119967999998, 2024-09-13T09:14:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:14:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:14:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:14:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:14:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:14:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:14:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:14:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:14:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:14:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:14:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:14:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:14:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:14:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.132103552999993, 2024-09-13T09:14:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:14:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:14:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:14:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:14:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:14:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:14:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:14:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:14:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:14:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:14:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:14:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:14:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:14:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:14:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.132087444999996, 2024-09-13T09:14:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:14:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:14:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:14:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:14:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:14:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:14:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:14:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:14:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:14:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:14:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:14:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:14:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:14:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:14:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.132071883999998, 2024-09-13T09:14:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:14:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:14:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:14:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:14:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:14:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:14:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:14:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:14:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:14:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:14:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:14:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:14:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:14:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:14:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.132055046000005, 2024-09-13T09:14:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:14:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:14:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:14:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:14:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:14:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:14:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:14:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:14:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:14:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:14:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:14:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:14:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:14:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:14:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:14:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.132039054000003, 2024-09-13T09:14:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:14:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:14:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:14:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:14:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:14:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:14:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:14:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:14:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:14:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:14:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:14:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:14:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:14:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:14:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:14:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.132011555999995, 2024-09-13T09:14:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:14:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:14:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:14:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:14:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:14:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:14:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:14:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:14:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:14:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:14:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:14:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:14:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:14:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:14:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.131985697000005, 2024-09-13T09:14:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:14:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:14:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:14:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:14:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:14:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:14:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:14:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:14:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:14:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:14:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:14:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:14:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:14:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:14:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.131944207000004, 2024-09-13T09:14:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:14:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:14:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:14:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:14:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:14:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:14:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:14:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:14:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:14:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:14:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:14:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:14:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:14:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:14:10.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:14:10.717 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.717 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.717 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.717 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.717 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:10.718 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.131922969000001, 2024-09-13T09:14:10.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:14:10.718 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.718 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.718 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.718 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.718 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.718 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:14:10.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.719 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:10.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:10.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:10.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:14:10.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:14:10.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.721 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:14:10.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:14:10.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.721 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:14:10.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:14:10.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:14:10.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:14:10.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:14:10.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:14:10.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:14:10.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:14:10.723 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.723 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.723 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.723 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.723 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:10.723 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.131900169000005, 2024-09-13T09:14:10.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:14:10.724 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.724 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.724 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.724 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.724 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.724 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:14:10.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.724 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:10.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:10.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:14:10.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:14:10.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:14:10.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:14:10.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:14:10.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:14:10.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:14:10.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:14:10.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:14:10.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:14:10.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:14:10.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:14:10.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:14:10.727 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.727 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.727 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.728 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.728 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:10.728 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.131883712999993, 2024-09-13T09:14:10.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:14:10.728 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.728 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.728 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.728 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.728 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.728 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:14:10.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.728 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:14:10.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:14:10.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:14:10.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:14:10.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:14:10.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:14:10.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:14:10.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:14:10.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.732 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.732 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:14:10.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:14:10.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.732 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.732 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:14:10.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:14:10.732 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.732 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.732 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.733 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.733 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:10.733 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.131638576, 2024-09-13T09:14:10.733 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:14:10.733 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.733 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.733 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.733 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.733 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.733 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:14:10.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.733 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.734 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:10.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.734 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:10.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.734 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:14:10.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:14:10.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:14:10.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:14:10.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:14:10.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:14:10.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:14:10.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:14:10.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:14:10.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:14:10.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:14:10.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:14:10.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.739 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:14:10.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:14:10.739 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.739 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.739 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.739 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.739 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:10.739 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.129753016999999, 2024-09-13T09:14:10.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:14:10.739 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.739 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.739 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.740 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.740 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.740 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:14:10.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.740 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.740 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:10.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.740 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:10.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:14:10.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:14:10.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:14:10.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:14:10.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:14:10.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:14:10.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:14:10.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:14:10.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:14:10.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:14:10.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:14:10.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:14:10.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:14:10.743 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.744 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.744 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.744 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.744 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:10.744 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.129731651, 2024-09-13T09:14:10.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:14:10.744 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.744 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.744 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.744 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.744 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.745 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:14:10.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.745 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:10.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:10.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:10.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:10.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:14:10.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:14:10.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:14:10.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:14:10.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:14:10.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:14:10.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:14:10.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:14:10.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:14:10.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:14:10.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.748 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.748 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:14:10.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:14:10.748 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.748 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.748 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.748 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:10.748 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:10.749 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 88.129526283000004, 2024-09-13T09:14:10.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:14:10.749 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.749 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:10.749 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.749 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:10.749 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:10.749 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:14:10.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.749 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.749 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.749 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:10.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:10.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:14:10.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:14:10.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:14:10.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:14:10.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:14:10.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:14:10.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:14:10.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:14:10.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:14:10.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:14:10.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.752 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:14:10.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:14:10.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.752 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:14:10.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:14:10.752 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.753 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.753 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.753 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5272.0:1775 3.0 3:02a42fca:::rbd_data.11235782ee8d.0000000000000030:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2244608~40448] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:14:10.753 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:10.753 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 60.387984441999997, 2024-09-13T09:14:10.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.666843732, 2024-09-13T09:14:10.753 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:10.753 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:10.753 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:10.753 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5272", 2024-09-13T09:14:10.753 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4089902097", 2024-09-13T09:14:10.754 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1775 2024-09-13T09:14:10.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.754 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:10.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.754 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:10.754 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:10.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.754 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:10.754 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:10.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:10.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:10.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247780+0000", 2024-09-13T09:14:10.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:10.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:10.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247865+0000", 2024-09-13T09:14:10.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4359999999999999e-05 2024-09-13T09:14:10.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:10.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247876+0000", 2024-09-13T09:14:10.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1456e-05 2024-09-13T09:14:10.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:10.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247883+0000", 2024-09-13T09:14:10.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:14:10.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247926+0000", 2024-09-13T09:14:10.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3203e-05 2024-09-13T09:14:10.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:10.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247932+0000", 2024-09-13T09:14:10.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:14:10.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909639+0000", 2024-09-13T09:14:10.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6617069099999999 2024-09-13T09:14:10.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:10.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909828+0000", 2024-09-13T09:14:10.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000189075 2024-09-13T09:14:10.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:10.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911676+0000", 2024-09-13T09:14:10.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001847235 2024-09-13T09:14:10.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:10.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911765+0000", 2024-09-13T09:14:10.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9530000000000005e-05 2024-09-13T09:14:10.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:14:10.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911901+0000", 2024-09-13T09:14:10.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013603899999999999 2024-09-13T09:14:10.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:10.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.912889+0000", 2024-09-13T09:14:10.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098807400000000003 2024-09-13T09:14:10.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:10.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914438+0000", 2024-09-13T09:14:10.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015491070000000001 2024-09-13T09:14:10.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:10.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914505+0000", 2024-09-13T09:14:10.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:14:10.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:10.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:10.760 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:10.760 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914625+0000", 2024-09-13T09:14:10.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120108 2024-09-13T09:14:10.760 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.760 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.760 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.760 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:10.760 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:10.760 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:10.761 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:14:10.762 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:10.761+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:10.763 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:14:10.763 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:14:10.763 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:14:10.764 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:14:10.764 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:14:10.764 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:14:10.764 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:14:10.765 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:14:10.765 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:14:10.765 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:14:10.772 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:14:10.772 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:14:10.775 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:14:10.776 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:14:10.862 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:10.861+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:10.879 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:10.879 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:10.888 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:14:10.963 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:10.962+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:11.030 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:11.030 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:11.039 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:14:11.084 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:11.083+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:11.141 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:11.142 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:11.150 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:14:11.166 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:11.165+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:11.266 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:11.265+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:11.294 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:11.294 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:11.303 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:14:11.368 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:11.367+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:11.405 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:11.405 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:11.414 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:14:11.469 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:11.468+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:11.557 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:11.558 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:11.566 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:14:11.569 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:11.568+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:11.669 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:11.669 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:11.670 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:11.669+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:11.678 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:14:11.721 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-09-13T09:14:11.780 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:11.779+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:11.826 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:11.827 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:11.835 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:14:11.878 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:11.878+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:11.940 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:11.940 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:11.949 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:14:11.993 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:11.992+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:12.058 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:12.058 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:12.067 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:14:12.073 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:12.072+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:12.174 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:12.173+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:12.214 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:12.214 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:12.222 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:14:12.307 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:12.306+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:12.369 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:12.370 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:12.376 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:12.375+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:12.378 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:14:12.477 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:12.477+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:12.525 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:12.525 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:12.534 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:14:12.578 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:12.577+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:12.641 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:12.641 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:12.649 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:14:12.679 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:12.678+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:12.777 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:14:12.780 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:12.779+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:12.811 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:12.811 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:12.822 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:14:12.883 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:12.881+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:12.907 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:14:12.907 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:14:12.923 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:14:12.924 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:14:12.924 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:14:12.927 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:12.927 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:12.936 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:14:12.940 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:14:12.940 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:14:12.956 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:14:12.956 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:14:12.973 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:14:12.973 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:14:12.976 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:14:12.976 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:14:12.983 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:12.983+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:13.039 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:13.040 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:13.040 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:13.040 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:13.040 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:13.048 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:14:13.090 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:13.089+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:13.186 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:13.185+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:13.192 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:13.193 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:13.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:13.193 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:13.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.193 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.193 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:13.193 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.669654894999994, 2024-09-13T09:14:13.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:14:13.193 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.193 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.193 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.193 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.193 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.194 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:14:13.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.194 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:13.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:13.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:13.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:14:13.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:14:13.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:14:13.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:14:13.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:14:13.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:14:13.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:14:13.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:14:13.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:14:13.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:14:13.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:14:13.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:14:13.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:14:13.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:14:13.198 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.198 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.198 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.198 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.199 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:13.199 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.669605838999999, 2024-09-13T09:14:13.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:14:13.199 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.199 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.199 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.199 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.199 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.199 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:14:13.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.199 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:13.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:13.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:13.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:14:13.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:14:13.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:14:13.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:14:13.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:14:13.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:14:13.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:14:13.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:14:13.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:14:13.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:14:13.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.205 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.205 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:14:13.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:14:13.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.205 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.205 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:14:13.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:14:13.205 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.205 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.205 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.206 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.206 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:13.206 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.669565763999998, 2024-09-13T09:14:13.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:14:13.206 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.206 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.206 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:14:13.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.206 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:13.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:13.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:13.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:14:13.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:14:13.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:14:13.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:14:13.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:14:13.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:14:13.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:14:13.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:14:13.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:14:13.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:14:13.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:14:13.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:14:13.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:14:13.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:14:13.210 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.210 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.210 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.210 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.211 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:13.211 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.669535948999993, 2024-09-13T09:14:13.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:14:13.211 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.211 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.211 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.211 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.211 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.211 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:14:13.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.211 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:13.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:13.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:13.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:14:13.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:14:13.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:14:13.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:14:13.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:14:13.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:14:13.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:14:13.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:14:13.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:14:13.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:14:13.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:14:13.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:14:13.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:14:13.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:14:13.215 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.215 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.215 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.215 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.215 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:13.216 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.669506287999994, 2024-09-13T09:14:13.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:14:13.216 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.216 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.216 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.216 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.216 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.216 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:14:13.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.216 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:13.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:13.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:13.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:14:13.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:14:13.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:14:13.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:14:13.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:14:13.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:14:13.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:14:13.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:14:13.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:14:13.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:14:13.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:14:13.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:14:13.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:14:13.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:14:13.220 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.220 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.220 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.220 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.221 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:13.221 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.669477080999997, 2024-09-13T09:14:13.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:14:13.221 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.221 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.221 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.221 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.221 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.221 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:14:13.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.221 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:13.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:13.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:13.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:14:13.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:14:13.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:14:13.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:14:13.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:14:13.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:14:13.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:14:13.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:14:13.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:14:13.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:14:13.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:14:13.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:14:13.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:14:13.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:14:13.225 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.225 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.225 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.225 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.225 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:13.225 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.669420900000006, 2024-09-13T09:14:13.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:14:13.225 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.225 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.226 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.226 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.226 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.226 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:14:13.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.226 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:13.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:13.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:13.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:14:13.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:14:13.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:14:13.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:14:13.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:14:13.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:14:13.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:14:13.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:14:13.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:14:13.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:14:13.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:14:13.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:14:13.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:14:13.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:14:13.230 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.230 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.230 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.230 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.230 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:13.230 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.669367438999998, 2024-09-13T09:14:13.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:14:13.230 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.231 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.231 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.231 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.231 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.231 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:14:13.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.231 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:13.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:13.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:14:13.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:14:13.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:14:13.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:14:13.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:14:13.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:14:13.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:14:13.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:14:13.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:14:13.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:14:13.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:14:13.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:14:13.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:14:13.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:14:13.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:14:13.235 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.235 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.235 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.235 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.235 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:13.235 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.669310608999993, 2024-09-13T09:14:13.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:14:13.235 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.235 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.236 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.236 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.236 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:14:13.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.236 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:13.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:13.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:14:13.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:14:13.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:14:13.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:14:13.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:14:13.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:14:13.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:14:13.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:14:13.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:14:13.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:14:13.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:14:13.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:14:13.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:14:13.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:14:13.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:14:13.240 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.240 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.240 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.240 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.240 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:13.240 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.669259581999995, 2024-09-13T09:14:13.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:14:13.240 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.240 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.240 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.240 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.241 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.241 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:14:13.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.241 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:13.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:13.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:14:13.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:14:13.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:14:13.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:14:13.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:14:13.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:14:13.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:14:13.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:14:13.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:14:13.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:14:13.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:14:13.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:14:13.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:14:13.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:14:13.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:14:13.244 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.244 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.245 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.245 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.245 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:13.245 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.669218392000005, 2024-09-13T09:14:13.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:14:13.245 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.245 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.245 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.245 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.245 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.245 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:14:13.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.246 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:13.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:13.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:13.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:14:13.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:14:13.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:14:13.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:14:13.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:14:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:14:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:14:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:14:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:14:13.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:14:13.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:14:13.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:14:13.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:14:13.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:14:13.249 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.249 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.249 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.250 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.250 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:13.250 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.669176276000002, 2024-09-13T09:14:13.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:14:13.250 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.250 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.250 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.250 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.250 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.250 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:14:13.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.250 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:13.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:13.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:13.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:13.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:14:13.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:14:13.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:14:13.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:14:13.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:14:13.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:14:13.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:14:13.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:14:13.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:14:13.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:14:13.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:14:13.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:14:13.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:14:13.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:14:13.254 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.254 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.255 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.255 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.255 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:13.255 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.669132396999998, 2024-09-13T09:14:13.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:14:13.255 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.255 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.256 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.256 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.256 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.256 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:14:13.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.256 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:13.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:13.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:13.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:13.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:14:13.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:14:13.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:14:13.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:14:13.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:14:13.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:14:13.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:14:13.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:14:13.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:14:13.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:14:13.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:14:13.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:14:13.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:14:13.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:14:13.260 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.260 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.260 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.260 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.260 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:13.261 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.669086480000004, 2024-09-13T09:14:13.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:14:13.261 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.261 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.261 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.261 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.261 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.261 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:14:13.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.261 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:13.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:13.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:14:13.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:14:13.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:14:13.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:14:13.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:14:13.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:14:13.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:14:13.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:14:13.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:14:13.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:14:13.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:14:13.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:14:13.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:14:13.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:14:13.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:14:13.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.265 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.265 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.265 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:13.266 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.669036708999997, 2024-09-13T09:14:13.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:14:13.266 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.266 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.266 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.266 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.266 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.266 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:14:13.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.266 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:13.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:13.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:13.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:13.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:14:13.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:14:13.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:14:13.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:14:13.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:14:13.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:14:13.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:14:13.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:14:13.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:14:13.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:14:13.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:14:13.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:14:13.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:14:13.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:14:13.270 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.270 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.271 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.271 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.271 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:13.271 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.668991305000006, 2024-09-13T09:14:13.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:14:13.271 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.271 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.271 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.271 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.271 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.272 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:14:13.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.272 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:13.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:13.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:13.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:13.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:14:13.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:14:13.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:14:13.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:14:13.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:14:13.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:14:13.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:14:13.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:14:13.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:14:13.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:14:13.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:14:13.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:14:13.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:14:13.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:14:13.275 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.276 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.276 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.276 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.276 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:13.276 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.668940667000001, 2024-09-13T09:14:13.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:14:13.276 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.276 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.276 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.276 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.276 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.277 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:14:13.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.277 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:13.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:13.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:14:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:14:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:14:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:14:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:14:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:14:13.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:14:13.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:14:13.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:14:13.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:14:13.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:14:13.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:14:13.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:14:13.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:14:13.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:14:13.280 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.280 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.280 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.281 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.281 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:13.281 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.668884824000003, 2024-09-13T09:14:13.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:14:13.281 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.281 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.281 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.281 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.281 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.281 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:14:13.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.282 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:13.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:13.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:13.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:14:13.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:14:13.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:14:13.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:14:13.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:14:13.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:14:13.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:14:13.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:14:13.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:14:13.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:14:13.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:14:13.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:14:13.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:14:13.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:14:13.285 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.285 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.285 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.286 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.286 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:13.286 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.668827902000004, 2024-09-13T09:14:13.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:14:13.286 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.286 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.286 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.286 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.286 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.286 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:14:13.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.286 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:13.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:13.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:13.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:13.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:14:13.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:14:13.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:14:13.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:14:13.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:14:13.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:14:13.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:14:13.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:14:13.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:14:13.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:14:13.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:14:13.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:14:13.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:14:13.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:14:13.293 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.293 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.293 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.294 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:13.294 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:13.294 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 90.668774338999995, 2024-09-13T09:14:13.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:14:13.294 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.294 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:13.294 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.294 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:13.294 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:13.294 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:14:13.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.295 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:13.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:13.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:13.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:14:13.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:14:13.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:14:13.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:14:13.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:14:13.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:14:13.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:14:13.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:14:13.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:13.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:14:13.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:14:13.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:14:13.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:14:13.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:14:13.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:14:13.299 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.299 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.299 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.299 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.299 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.299 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:13.300 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:14:13.301 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:13.299+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:13.401 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:13.401+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:13.444 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:13.444 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:13.444 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:13.444 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:13.452 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:14:13.537 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:13.536+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:13.595 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:13.595 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:13.595 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:13.604 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:14:13.604 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:13.603+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:13.705 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:13.704+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:13.739 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:14:13.756 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:13.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2185 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228561169000002, 2024-09-13T09:14:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063348994000000006, 2024-09-13T09:14:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2185 2024-09-13T09:14:13.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525765+0000", 2024-09-13T09:14:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2390000000000001e-06 2024-09-13T09:14:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525772+0000", 2024-09-13T09:14:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:14:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525777+0000", 2024-09-13T09:14:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0799999999999996e-06 2024-09-13T09:14:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589020+0000", 2024-09-13T09:14:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063242501000000007 2024-09-13T09:14:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589063+0000", 2024-09-13T09:14:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2673999999999998e-05 2024-09-13T09:14:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589112+0000", 2024-09-13T09:14:13.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9286000000000002e-05 2024-09-13T09:14:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2187 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228541622000002, 2024-09-13T09:14:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063500102000000003, 2024-09-13T09:14:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.762 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2187 2024-09-13T09:14:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:13.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525784+0000", 2024-09-13T09:14:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1629999999999999e-06 2024-09-13T09:14:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525792+0000", 2024-09-13T09:14:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:14:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525796+0000", 2024-09-13T09:14:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7389999999999999e-06 2024-09-13T09:14:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589132+0000", 2024-09-13T09:14:13.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063336119999999996 2024-09-13T09:14:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589187+0000", 2024-09-13T09:14:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4126000000000003e-05 2024-09-13T09:14:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589283+0000", 2024-09-13T09:14:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6032999999999993e-05 2024-09-13T09:14:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.766 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2188 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228523576000001, 2024-09-13T09:14:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063724617999999997, 2024-09-13T09:14:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2188 2024-09-13T09:14:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.767 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525802+0000", 2024-09-13T09:14:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0439999999999998e-06 2024-09-13T09:14:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525809+0000", 2024-09-13T09:14:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.238e-06 2024-09-13T09:14:13.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525814+0000", 2024-09-13T09:14:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-13T09:14:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589327+0000", 2024-09-13T09:14:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063513257000000004 2024-09-13T09:14:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589418+0000", 2024-09-13T09:14:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0796000000000003e-05 2024-09-13T09:14:13.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589525+0000", 2024-09-13T09:14:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010693500000000001 2024-09-13T09:14:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2190 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228505490000003, 2024-09-13T09:14:13.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063863981, 2024-09-13T09:14:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2190 2024-09-13T09:14:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525818+0000", 2024-09-13T09:14:13.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525820+0000", 2024-09-13T09:14:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:14:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525827+0000", 2024-09-13T09:14:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0589999999999999e-06 2024-09-13T09:14:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525831+0000", 2024-09-13T09:14:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7870000000000001e-06 2024-09-13T09:14:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589562+0000", 2024-09-13T09:14:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063731038000000004 2024-09-13T09:14:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589612+0000", 2024-09-13T09:14:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0087999999999999e-05 2024-09-13T09:14:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589683+0000", 2024-09-13T09:14:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0121000000000002e-05 2024-09-13T09:14:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2192 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.22848853, 2024-09-13T09:14:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064050299000000005, 2024-09-13T09:14:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.776 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2192 2024-09-13T09:14:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525835+0000", 2024-09-13T09:14:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525837+0000", 2024-09-13T09:14:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:14:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525845+0000", 2024-09-13T09:14:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3660000000000003e-06 2024-09-13T09:14:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525850+0000", 2024-09-13T09:14:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0030000000000002e-06 2024-09-13T09:14:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589720+0000", 2024-09-13T09:14:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063870176000000001 2024-09-13T09:14:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589801+0000", 2024-09-13T09:14:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0889000000000004e-05 2024-09-13T09:14:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589886+0000", 2024-09-13T09:14:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5183000000000001e-05 2024-09-13T09:14:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.780 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228470426999998, 2024-09-13T09:14:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:14:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:14:13.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:14:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:14:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:14:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:14:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:14:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:14:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:14:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:14:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:14:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:14:13.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:14:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:14:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:14:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:13.785 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228452728999997, 2024-09-13T09:14:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:14:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:14:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:13.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:14:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:14:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:14:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:14:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:14:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:14:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:14:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:14:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:14:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:14:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:14:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:14:13.789 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228432114, 2024-09-13T09:14:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:14:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.790 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.791 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:14:13.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.791 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:13.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:13.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:13.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:14:13.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:14:13.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:14:13.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:14:13.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:14:13.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:14:13.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:14:13.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:14:13.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:14:13.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:14:13.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:14:13.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:14:13.794 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.794 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.794 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.795 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.795 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:13.795 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228413158000002, 2024-09-13T09:14:13.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:14:13.795 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.795 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.795 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.795 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.795 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.795 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:14:13.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.795 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:13.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:13.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:13.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:14:13.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:14:13.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:14:13.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:14:13.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:14:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:14:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:14:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:14:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:14:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:14:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:14:13.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:14:13.799 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.799 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.799 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.799 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.799 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:13.799 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228395900000002, 2024-09-13T09:14:13.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:14:13.799 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.799 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.799 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.799 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.799 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.800 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:14:13.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.800 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:13.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:13.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:13.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:14:13.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:14:13.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:14:13.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:14:13.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:14:13.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:14:13.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:14:13.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:14:13.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:14:13.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:14:13.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:14:13.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:14:13.803 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.803 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.803 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.804 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.804 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:13.804 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228375886000002, 2024-09-13T09:14:13.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:14:13.804 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.804 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.804 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.804 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.804 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.804 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:14:13.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.804 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:13.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:13.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:13.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:14:13.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:14:13.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:14:13.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:14:13.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:14:13.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:14:13.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:14:13.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:14:13.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:14:13.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:14:13.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:14:13.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:14:13.808 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.808 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.808 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.808 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.808 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:13.808 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228350304000003, 2024-09-13T09:14:13.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:14:13.808 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.808 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.809 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.809 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.809 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.809 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:14:13.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.809 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:13.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:13.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:14:13.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:14:13.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:14:13.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:14:13.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:14:13.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:14:13.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:14:13.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:14:13.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:14:13.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:14:13.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:14:13.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:14:13.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:14:13.812 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.812 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.812 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.812 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.812 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228331019999999, 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:13.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:13.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:14:13.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:14:13.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:14:13.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:14:13.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:14:13.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:14:13.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:14:13.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:14:13.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:14:13.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:14:13.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:14:13.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:14:13.816 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.816 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.816 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.817 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.817 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:13.817 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228313763999999, 2024-09-13T09:14:13.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:14:13.817 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.817 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.817 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.817 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.817 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.817 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:14:13.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.818 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:13.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:13.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:13.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:14:13.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:14:13.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:14:13.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:14:13.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:14:13.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:14:13.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:14:13.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:14:13.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:14:13.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:14:13.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:14:13.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:14:13.821 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.821 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.821 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.821 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.821 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:13.822 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228294284, 2024-09-13T09:14:13.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:14:13.822 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.822 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.822 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.822 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.822 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.822 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:14:13.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.822 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:13.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:13.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:14:13.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:14:13.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:14:13.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:14:13.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:14:13.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:14:13.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:14:13.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:14:13.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:14:13.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:14:13.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:14:13.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:14:13.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:14:13.826 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.826 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.826 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.826 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.826 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:13.826 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228277355000003, 2024-09-13T09:14:13.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:14:13.827 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.827 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.827 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:14:13.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.827 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:13.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:13.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:14:13.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:14:13.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:14:13.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:14:13.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:14:13.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:14:13.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:14:13.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:14:13.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:14:13.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:14:13.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:14:13.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:14:13.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:14:13.831 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.831 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.831 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.831 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.831 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:13.831 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228258206, 2024-09-13T09:14:13.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:14:13.831 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.831 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.832 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:14:13.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.832 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:13.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:13.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:13.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:14:13.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:14:13.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:14:13.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:14:13.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:14:13.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:14:13.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:14:13.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:14:13.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:14:13.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:14:13.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:14:13.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:14:13.835 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.835 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.835 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.836 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.836 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:13.836 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.228240067000002, 2024-09-13T09:14:13.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:14:13.836 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.836 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.836 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.836 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.836 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.836 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:14:13.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.837 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:13.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:13.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:13.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:14:13.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:14:13.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:14:13.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:14:13.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:14:13.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:14:13.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:14:13.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:14:13.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:14:13.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:14:13.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:14:13.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:14:13.840 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.840 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.840 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.840 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.840 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:13.840 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.227829571999997, 2024-09-13T09:14:13.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:14:13.840 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.841 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.841 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.841 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.841 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.841 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:14:13.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.841 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:13.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:13.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:13.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:14:13.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:14:13.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:14:13.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:14:13.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:14:13.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:14:13.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:14:13.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:14:13.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:14:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:14:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:14:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:14:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.845 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:13.845 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:13.845 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.227806671000003, 2024-09-13T09:14:13.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:14:13.845 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:13.845 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:13.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:13.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:13.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:13.845 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:14:13.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.846 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:13.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:13.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:13.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:13.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:13.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:13.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:13.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:13.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:13.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:13.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:14:13.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:14:13.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:13.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:14:13.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:14:13.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:13.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:14:13.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:14:13.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:13.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:14:13.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:14:13.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:13.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:14:13.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:14:13.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:13.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:13.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:13.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:14:13.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:14:13.849 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.849 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.849 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.849 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:13.849 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:13.849 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:13.851 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:14:13.851 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:13.850+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:13.951 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:13.951+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:13.996 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:13.996 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:13.996 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:13.996 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:14.005 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:14:14.005 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:14.004+0000 7fa389c24640 -1 osd.2 84 log_to_monitors true 2024-09-13T09:14:14.089 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:14.088+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:14.147 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:14.147 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:14.147 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:14.147 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:14.147 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:14.152 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:14.152+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:14.156 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:14:14.254 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:14.253+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:14.299 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:14.300 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:14.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:14.300 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:14.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.300 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:14.300 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:14.300 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.552019518999998, 2024-09-13T09:14:14.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:14:14.300 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.300 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:14.300 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.301 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:14.301 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:14.301 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:14:14.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.301 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:14.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:14.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:14.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:14:14.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:14:14.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:14:14.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:14:14.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:14:14.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:14:14.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:14:14.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:14:14.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:14:14.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:14:14.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:14:14.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:14:14.304 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.304 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.304 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.305 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:14.305 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:14.305 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.552010840999998, 2024-09-13T09:14:14.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:14:14.305 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.305 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:14.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:14.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:14.305 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:14:14.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.306 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:14.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:14.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:14.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:14.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:14:14.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:14:14.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:14:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:14:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:14:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:14:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:14:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:14:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:14:14.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:14:14.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:14:14.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:14:14.309 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.309 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.309 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.310 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:14.310 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:14.310 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.552001855, 2024-09-13T09:14:14.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:14:14.310 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.310 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:14.310 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.310 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:14.310 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:14.310 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:14:14.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.310 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:14.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:14.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:14.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:14:14.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:14:14.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:14:14.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:14:14.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:14:14.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:14:14.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:14:14.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:14:14.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:14:14.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:14:14.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:14:14.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:14:14.316 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.316 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.317 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.317 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:14.317 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:14.317 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.551993882999994, 2024-09-13T09:14:14.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:14:14.317 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.317 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:14.317 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.318 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:14.318 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:14.318 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:14:14.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.318 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:14.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:14.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:14.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:14.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:14:14.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:14:14.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:14:14.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:14:14.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:14:14.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:14:14.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:14:14.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:14:14.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:14:14.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:14:14.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:14:14.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:14:14.321 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.321 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.321 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.321 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:14.322 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:14.322 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.550119049000003, 2024-09-13T09:14:14.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:14:14.322 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.322 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:14.322 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.322 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:14.322 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:14.322 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:14:14.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.322 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:14.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:14.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:14.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:14:14.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:14:14.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:14:14.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:14:14.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:14:14.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:14:14.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:14:14.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:14:14.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:14:14.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:14:14.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:14:14.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:14:14.326 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.326 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.326 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.326 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:14.326 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:14.326 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.550108387999998, 2024-09-13T09:14:14.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:14:14.326 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.326 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:14.326 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.326 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:14.327 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:14.327 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:14:14.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.327 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:14.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:14.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:14.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:14.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:14:14.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:14:14.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:14:14.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:14:14.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:14:14.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:14:14.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:14:14.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:14:14.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:14:14.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:14:14.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:14:14.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:14:14.330 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.330 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.330 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.331 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:14.331 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:14.331 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.550086411999999, 2024-09-13T09:14:14.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:14:14.331 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.331 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:14.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:14.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:14.331 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:14:14.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.331 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:14.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:14.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:14.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:14:14.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:14:14.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:14:14.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:14:14.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:14:14.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:14:14.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:14:14.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:14:14.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:14:14.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:14:14.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:14:14.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:14:14.335 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.335 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.335 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.335 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:14.335 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:14.335 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.550072909999997, 2024-09-13T09:14:14.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:14:14.336 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.336 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:14.336 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.336 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:14.336 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:14.336 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:14:14.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.336 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:14.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:14.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:14.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:14:14.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:14:14.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:14:14.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:14:14.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:14:14.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:14:14.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:14:14.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:14:14.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:14:14.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:14:14.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:14:14.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:14:14.340 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.340 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.340 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.340 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:14.340 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:14.340 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.548063811999995, 2024-09-13T09:14:14.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:14:14.340 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.341 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:14.341 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.341 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:14.341 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:14.341 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:14:14.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.341 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:14.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:14.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:14.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:14:14.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:14:14.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:14:14.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:14:14.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:14:14.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:14:14.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:14:14.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:14:14.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:14:14.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:14:14.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:14:14.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:14:14.344 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.344 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.345 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:14.345 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:14.345 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.548039489999994, 2024-09-13T09:14:14.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:14:14.345 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.345 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:14.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:14.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:14.345 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:14:14.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.346 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:14.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:14.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:14.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:14:14.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:14:14.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:14:14.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:14:14.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:14:14.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:14:14.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:14:14.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:14:14.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:14:14.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:14:14.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:14:14.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:14:14.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.350 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.350 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.350 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:14.350 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:14:14.350 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.548028516000002, 2024-09-13T09:14:14.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:14:14.350 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.350 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:14.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:14.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:14.350 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:14:14.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.351 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:14:14.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:14:14.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:14:14.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:14:14.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:14:14.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:14:14.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:14:14.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:14:14.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:14:14.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:14:14.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:14:14.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:14:14.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:14:14.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:14:14.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:14:14.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.354 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.354 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:14.354 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:14.354 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.546110890999998, 2024-09-13T09:14:14.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:14:14.355 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.355 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:14.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:14.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:14.355 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:14:14.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.355 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:14.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:14.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:14.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:14:14.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:14:14.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:14:14.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:14:14.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:14:14.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:14:14.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:14:14.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:14:14.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:14:14.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:14:14.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:14:14.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:14:14.359 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.359 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.359 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.359 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:14.359 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:14.359 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.546101493999998, 2024-09-13T09:14:14.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:14:14.359 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.359 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:14.359 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.359 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:14.360 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:14.360 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:14:14.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.360 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:14.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:14.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:14.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:14.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:14:14.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:14:14.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:14:14.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:14:14.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:14:14.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:14:14.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:14:14.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:14:14.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:14:14.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:14:14.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:14:14.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:14:14.363 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.363 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.364 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.364 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3181 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T09:14:14.364 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:14:14.364 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 44.275737317999997, 2024-09-13T09:14:14.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.480957603, 2024-09-13T09:14:14.364 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.364 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:14.364 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.364 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:14:14.364 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:14:14.364 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3181 2024-09-13T09:14:14.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.365 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:14:14.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:14:14.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021894+0000", 2024-09-13T09:14:14.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:14:14.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021903+0000", 2024-09-13T09:14:14.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4109999999999996e-06 2024-09-13T09:14:14.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021919+0000", 2024-09-13T09:14:14.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6225000000000001e-05 2024-09-13T09:14:14.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021926+0000", 2024-09-13T09:14:14.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4120000000000004e-06 2024-09-13T09:14:14.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021975+0000", 2024-09-13T09:14:14.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8526000000000003e-05 2024-09-13T09:14:14.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022049+0000", 2024-09-13T09:14:14.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4018999999999997e-05 2024-09-13T09:14:14.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:14:14.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022230+0000", 2024-09-13T09:14:14.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018085599999999999 2024-09-13T09:14:14.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:14.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022796+0000", 2024-09-13T09:14:14.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056638999999999995 2024-09-13T09:14:14.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499540+0000", 2024-09-13T09:14:14.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.47674342800000002 2024-09-13T09:14:14.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499547+0000", 2024-09-13T09:14:14.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:14:14.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502794+0000", 2024-09-13T09:14:14.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.003246818 2024-09-13T09:14:14.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502854+0000", 2024-09-13T09:14:14.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9925e-05 2024-09-13T09:14:14.370 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.370 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.370 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.370 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3182 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 139818970156544 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:14:14.370 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:14:14.370 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.802502763, 2024-09-13T09:14:14.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0089058390000001, 2024-09-13T09:14:14.371 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.371 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:14.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:14:14.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:14:14.371 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3182 2024-09-13T09:14:14.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.371 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:14:14.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:14:14.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495128+0000", 2024-09-13T09:14:14.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:14:14.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495139+0000", 2024-09-13T09:14:14.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0599e-05 2024-09-13T09:14:14.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495157+0000", 2024-09-13T09:14:14.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7830999999999999e-05 2024-09-13T09:14:14.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495167+0000", 2024-09-13T09:14:14.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0689999999999999e-05 2024-09-13T09:14:14.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495229+0000", 2024-09-13T09:14:14.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1898999999999998e-05 2024-09-13T09:14:14.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:14:14.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495243+0000", 2024-09-13T09:14:14.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3349e-05 2024-09-13T09:14:14.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499575+0000", 2024-09-13T09:14:14.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043324410000000002 2024-09-13T09:14:14.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499582+0000", 2024-09-13T09:14:14.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0849999999999999e-06 2024-09-13T09:14:14.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502872+0000", 2024-09-13T09:14:14.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0032895150000001 2024-09-13T09:14:14.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503016+0000", 2024-09-13T09:14:14.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014394200000000001 2024-09-13T09:14:14.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504036+0000", 2024-09-13T09:14:14.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010208680000000001 2024-09-13T09:14:14.376 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.376 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.376 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.377 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:19151 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 140383089882784 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:14:14.377 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:14:14.377 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 43.802491304, 2024-09-13T09:14:14.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.009071287, 2024-09-13T09:14:14.377 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.377 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:14.377 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.377 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:14.377 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:14.377 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19151 2024-09-13T09:14:14.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.377 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:14:14.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:14:14.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495140+0000", 2024-09-13T09:14:14.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:14:14.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495149+0000", 2024-09-13T09:14:14.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0759999999999992e-06 2024-09-13T09:14:14.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495165+0000", 2024-09-13T09:14:14.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5322e-05 2024-09-13T09:14:14.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495173+0000", 2024-09-13T09:14:14.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0979999999999994e-06 2024-09-13T09:14:14.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495318+0000", 2024-09-13T09:14:14.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014553 2024-09-13T09:14:14.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:14:14.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495325+0000", 2024-09-13T09:14:14.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:14:14.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499598+0000", 2024-09-13T09:14:14.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004273064 2024-09-13T09:14:14.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499602+0000", 2024-09-13T09:14:14.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8539999999999999e-06 2024-09-13T09:14:14.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503507+0000", 2024-09-13T09:14:14.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0039045520000001 2024-09-13T09:14:14.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503583+0000", 2024-09-13T09:14:14.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6283000000000001e-05 2024-09-13T09:14:14.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504213+0000", 2024-09-13T09:14:14.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00063044700000000002 2024-09-13T09:14:14.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.382 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.383 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3908 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:14.383 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:14:14.383 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.204954412999999, 2024-09-13T09:14:14.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070530317999999995, 2024-09-13T09:14:14.383 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.383 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:14.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:14.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:14.383 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3908 2024-09-13T09:14:14.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.383 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:14:14.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:14:14.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092678+0000", 2024-09-13T09:14:14.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092681+0000", 2024-09-13T09:14:14.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.903e-06 2024-09-13T09:14:14.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092687+0000", 2024-09-13T09:14:14.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0859999999999998e-06 2024-09-13T09:14:14.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092692+0000", 2024-09-13T09:14:14.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-13T09:14:14.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.159711+0000", 2024-09-13T09:14:14.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067018922999999994 2024-09-13T09:14:14.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.159796+0000", 2024-09-13T09:14:14.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5247000000000006e-05 2024-09-13T09:14:14.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:14.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160111+0000", 2024-09-13T09:14:14.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00031449499999999998 2024-09-13T09:14:14.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:14.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162219+0000", 2024-09-13T09:14:14.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021085570000000001 2024-09-13T09:14:14.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:14.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163127+0000", 2024-09-13T09:14:14.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00090723400000000004 2024-09-13T09:14:14.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:14.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163172+0000", 2024-09-13T09:14:14.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5383000000000002e-05 2024-09-13T09:14:14.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163209+0000", 2024-09-13T09:14:14.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7132000000000003e-05 2024-09-13T09:14:14.389 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.389 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.389 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.389 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3909 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:14.389 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:14.389 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.204936922000002, 2024-09-13T09:14:14.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070616834000000003, 2024-09-13T09:14:14.389 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.389 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3909 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:14.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092699+0000", 2024-09-13T09:14:14.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:14:14.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092704+0000", 2024-09-13T09:14:14.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5940000000000001e-06 2024-09-13T09:14:14.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092709+0000", 2024-09-13T09:14:14.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5109999999999997e-06 2024-09-13T09:14:14.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160604+0000", 2024-09-13T09:14:14.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067895633999999996 2024-09-13T09:14:14.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160658+0000", 2024-09-13T09:14:14.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3292999999999999e-05 2024-09-13T09:14:14.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:14.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160935+0000", 2024-09-13T09:14:14.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000276879 2024-09-13T09:14:14.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:14.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163223+0000", 2024-09-13T09:14:14.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022886320000000001 2024-09-13T09:14:14.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163258+0000", 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5101000000000002e-05 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163292+0000", 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3237e-05 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163313+0000", 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1642e-05 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.394 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.395 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.395 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3910 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:14.395 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:14.395 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.204919858, 2024-09-13T09:14:14.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733837999999993, 2024-09-13T09:14:14.395 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.395 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:14.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:14.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:14.395 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3910 2024-09-13T09:14:14.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.395 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:14.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:14.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:14.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092716+0000", 2024-09-13T09:14:14.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.757e-06 2024-09-13T09:14:14.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092722+0000", 2024-09-13T09:14:14.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7869999999999998e-06 2024-09-13T09:14:14.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092726+0000", 2024-09-13T09:14:14.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.622e-06 2024-09-13T09:14:14.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161487+0000", 2024-09-13T09:14:14.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068761166999999998 2024-09-13T09:14:14.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161569+0000", 2024-09-13T09:14:14.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1185000000000003e-05 2024-09-13T09:14:14.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:14.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161868+0000", 2024-09-13T09:14:14.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029979799999999999 2024-09-13T09:14:14.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:14.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163366+0000", 2024-09-13T09:14:14.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001497888 2024-09-13T09:14:14.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:14.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163373+0000", 2024-09-13T09:14:14.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1640000000000004e-06 2024-09-13T09:14:14.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:14.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163412+0000", 2024-09-13T09:14:14.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8423999999999997e-05 2024-09-13T09:14:14.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163447+0000", 2024-09-13T09:14:14.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5441000000000002e-05 2024-09-13T09:14:14.400 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.400 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.400 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.400 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3911 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:14.401 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:14.401 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.204899942000001, 2024-09-13T09:14:14.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.072284372999999999, 2024-09-13T09:14:14.401 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:14.401 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:14.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:14.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:14.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:14.401 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3911 2024-09-13T09:14:14.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.401 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:14.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:14.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:14.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092736+0000", 2024-09-13T09:14:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9510000000000002e-06 2024-09-13T09:14:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092743+0000", 2024-09-13T09:14:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9e-06 2024-09-13T09:14:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092748+0000", 2024-09-13T09:14:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999998e-06 2024-09-13T09:14:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:14.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162257+0000", 2024-09-13T09:14:14.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.069508890000000004 2024-09-13T09:14:14.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:14.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162342+0000", 2024-09-13T09:14:14.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5886999999999999e-05 2024-09-13T09:14:14.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:14.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162682+0000", 2024-09-13T09:14:14.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00033971700000000001 2024-09-13T09:14:14.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:14.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164897+0000", 2024-09-13T09:14:14.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022152370000000001 2024-09-13T09:14:14.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:14.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164945+0000", 2024-09-13T09:14:14.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7243999999999997e-05 2024-09-13T09:14:14.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:14.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164985+0000", 2024-09-13T09:14:14.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0599000000000002e-05 2024-09-13T09:14:14.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:14.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:14.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:14.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.165018+0000", 2024-09-13T09:14:14.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2549000000000003e-05 2024-09-13T09:14:14.406 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.406 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.406 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.406 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:14.406 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:14.406 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:14.408 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:14:14.409 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:14.407+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:14.509 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:14.508+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:14.520 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:14.520 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:14.521 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:14.521 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:14.527 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:14:14.610 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:14.609+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:14.633 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:14.633 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:14.633 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:14.642 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:14:14.726 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:14.725+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:14.738 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-09-13T09:14:14.772 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:14.771+0000 7fa37d321640 -1 osd.2 121 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:14:14.801 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.269533854000002, 2024-09-13T09:14:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:14:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.801 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:14:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.802 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:14:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:14:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:14:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:14:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:14:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:14:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:14:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:14:14.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:14:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:14:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:14:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:14:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:14:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.805 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:14:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:14:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:14:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:14:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:14:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:14:14.806 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:14:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:14:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.269515819000006, 2024-09-13T09:14:14.807 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:14:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:14:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.808 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:14:14.809 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:14:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:14:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:14:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:14:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:14:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.810 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:14:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:14:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:14:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:14:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:14:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:14:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.811 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:14:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:14:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:14:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:14:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:14:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:14:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.812 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:14:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:14:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.269499148999998, 2024-09-13T09:14:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:14:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:14:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.814 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:14:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:14:14.815 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:14:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:14:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:14:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:14:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.816 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:14:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:14:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:14:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:14:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:14:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:14:14.817 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:14:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:14:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:14:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:14:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.818 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:14:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:14:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:14:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:14:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.819 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.269481517000003, 2024-09-13T09:14:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:14:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:14:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:14.820 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:14:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:14:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:14:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.822 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:14:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:14:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.823 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:14:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:14:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:14:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:14:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:14:14.824 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:14:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:14:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:14:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:14:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:14:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.825 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:14:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:14:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:14:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:14:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.826 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.269435212000005, 2024-09-13T09:14:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:14:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:14:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.827 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:14:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.828 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:14:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:14:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:14:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:14:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:14:14.829 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:14:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:14:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:14:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:14:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:14:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.830 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:14:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:14:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:14:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:14:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:14:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:14:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.831 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:14:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:14:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:14:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:14:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.832 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.269409910999997, 2024-09-13T09:14:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:14:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:14:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.833 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:14:14.834 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:14:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:14:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:14:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:14:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.835 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:14:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:14:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:14:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:14:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:14:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:14:14.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:14:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:14:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:14:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:14:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.837 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:14:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:14:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:14:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:14:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:14:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:14:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.838 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.269391682000006, 2024-09-13T09:14:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:14:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:14:14.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.840 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:14:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:14:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:14:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:14:14.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:14:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:14:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:14:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:14:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.842 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:14:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:14:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:14:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:14:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:14:14.843 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:14:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:14:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:14:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:14:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:14:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.844 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:14:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:14:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.269344363000002, 2024-09-13T09:14:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:14:14.845 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:14:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.846 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:14:14.847 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:14:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:14:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:14:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:14:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:14:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.848 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:14:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:14:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:14:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:14:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:14:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:14:14.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:14:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:14:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:14:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:14:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:14:14.850 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:14:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:14:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:14:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.268830124999994, 2024-09-13T09:14:14.851 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:14:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:14:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.852 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:14:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:14:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:14:14.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:14:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:14:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:14:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:14:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:14:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:14:14.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:14:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:14:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:14:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:14:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:14:14.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:14:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:14:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:14:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:14:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:14:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.856 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:14:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:14:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.268795595, 2024-09-13T09:14:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:14:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.857 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:14:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:14:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:14:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:14:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:14:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:14:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:14:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:14:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:14:14.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:14:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:14:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:14:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:14:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:14:14.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:14:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:14:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:14:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:14:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:14:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:14:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:14:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.268777688, 2024-09-13T09:14:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:14:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.863 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:14:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:14.864 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:14:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:14:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.865 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:14:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:14:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:14:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:14:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:14:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:14:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:14:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:14:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:14:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:14:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:14:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:14:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:14:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:14:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:14:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:14:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:14:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:14:14.868 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.259650840000006, 2024-09-13T09:14:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:14:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.869 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:14:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:14:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:14:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:14:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:14:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:14:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:14:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:14:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:14:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:14:14.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:14:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:14:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:14:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:14:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:14:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:14:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:14:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:14:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:14:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:14:14.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:14:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.259599260000002, 2024-09-13T09:14:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:14:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.875 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:14:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:14:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.877 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:14:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:14:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:14:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:14:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:14:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:14:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:14:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:14:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:14:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:14:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:14:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:14:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:14:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:14:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:14:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:14:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.880 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:14:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:14:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:14:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:14:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.259582984000005, 2024-09-13T09:14:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:14:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:14:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:14:14.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:14:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:14:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:14:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:14:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:14:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:14:14.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:14:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:14:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:14:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:14:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:14:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:14:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:14:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:14:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:14:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:14:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:14:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:14:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:14:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.25956721, 2024-09-13T09:14:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:14:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:14:14.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:14:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:14:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:14:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:14:14.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:14:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:14:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:14:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:14:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:14:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:14:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:14:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:14:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:14:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:14:14.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:14:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:14:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:14:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:14:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:14:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:14:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:14:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.259552334999995, 2024-09-13T09:14:14.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:14:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:14:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:14:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:14:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:14:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:14:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:14:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:14:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:14:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:14:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:14:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:14:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:14:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:14:14.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:14:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:14:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:14:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:14:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:14:14.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:14:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:14:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:14:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:14.900 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.259534873000007, 2024-09-13T09:14:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:14:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:14:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:14.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:14:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:14:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:14:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:14:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:14:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:14:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:14:14.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:14:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:14:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:14:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:14:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:14:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:14:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:14:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:14:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:14:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:14:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:14:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:14:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:14:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:14:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.257934241000001, 2024-09-13T09:14:14.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:14:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:14:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:14:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:14:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:14:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:14:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:14:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:14:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:14:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:14:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:14:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:14:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:14:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:14:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:14:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.257894190000002, 2024-09-13T09:14:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:14:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:14:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:14:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:14:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:14:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:14:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:14:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:14:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:14:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:14:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:14:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:14:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:14:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:14:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:14:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:14:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:14:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:14:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:14:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:14:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:14:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:14:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:14:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:14.917 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 92.257850426999994, 2024-09-13T09:14:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:14:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:14:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:14:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:14:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:14:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:14:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:14:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:14:14.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:14:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:14:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:14:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:14:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:14:14.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:14:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:14:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:14:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:14:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:14:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:14.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:14:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:14:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:14:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:14:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:14.923 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:14.923 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:14.925 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:14:14.929 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:14.924+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:14.978 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:14:14.979 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:14:15.025 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:15.025+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:15.074 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:15.074 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:15.074 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:15.074 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:15.083 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:14:15.119 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:15.119 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:15.119 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:15.128 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:15.126+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:15.132 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:14:15.132 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:14:15.149 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:14:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:14:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:14:15.166 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:14:15.166 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:14:15.182 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:14:15.183 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:14:15.199 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:14:15.199 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:14:15.202 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:14:15.203 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:14:15.226 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:15.227 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:15.227 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:15.225+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:15.235 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:14:15.328 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:15.327+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:15.381 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:15.381 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:15.390 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:14:15.429 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:15.428+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:15.495 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:15.496 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:15.504 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:14:15.547 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:15.547+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:15.607 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:15.607 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:15.616 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:14:15.631 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:15.630+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:15.732 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:15.731+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:15.762 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:15.762 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:15.772 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:14:15.833 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:15.832+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:15.915 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:15.915 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:15.925 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:14:15.970 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:15.969+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:16.028 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:16.029 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:16.034 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:16.033+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:16.038 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:14:16.136 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:16.135+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:16.181 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:16.181 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:16.190 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:14:16.236 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:16.236+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:16.292 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:16.292 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:16.301 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:14:16.337 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:16.337+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:16.438 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:16.438+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:16.442 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:16.442 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:16.451 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:14:16.539 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:16.539+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:16.594 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:16.594 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:16.602 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:14:16.641 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:16.640+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:16.722 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:16.722 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:16.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.722 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29717 3.1e 3.3c31401e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:14:16.723 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:16.723 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.97039231000000004, 2024-09-13T09:14:16.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.97041520299999995, 2024-09-13T09:14:16.723 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:16.723 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:14:16.723 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:16.723 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:16.723 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:16.723 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29717 2024-09-13T09:14:16.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.723 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:16.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.723 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:16.723 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:16.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:16.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:16.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:16.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:16.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:16.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751547+0000", 2024-09-13T09:14:16.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:16.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:16.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751552+0000", 2024-09-13T09:14:16.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:14:16.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:16.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751564+0000", 2024-09-13T09:14:16.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1793e-05 2024-09-13T09:14:16.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:16.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751571+0000", 2024-09-13T09:14:16.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2660000000000004e-06 2024-09-13T09:14:16.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:16.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753102+0000", 2024-09-13T09:14:16.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001531085 2024-09-13T09:14:16.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:16.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753115+0000", 2024-09-13T09:14:16.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3039e-05 2024-09-13T09:14:16.727 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:16.727 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:16.727 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:16.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.727 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29718 3.1e 3.3c31401e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:14:16.727 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:16.727 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.97035944500000004, 2024-09-13T09:14:16.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.97042811500000004, 2024-09-13T09:14:16.727 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:16.727 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "waiting for peered", 2024-09-13T09:14:16.728 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:16.728 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:16.728 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:16.728 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29718 2024-09-13T09:14:16.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.728 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:16.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:16.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:16.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:16.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:16.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:16.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:16.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:16.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:16.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:16.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:16.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751585+0000", 2024-09-13T09:14:16.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2050000000000004e-06 2024-09-13T09:14:16.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:16.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751591+0000", 2024-09-13T09:14:16.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.37e-06 2024-09-13T09:14:16.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:16.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751597+0000", 2024-09-13T09:14:16.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1190000000000002e-06 2024-09-13T09:14:16.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:16.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753150+0000", 2024-09-13T09:14:16.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001552351 2024-09-13T09:14:16.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:16.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:16.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:16.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753164+0000", 2024-09-13T09:14:16.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4076e-05 2024-09-13T09:14:16.731 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:16.731 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:16.731 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:16.731 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:16.731 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-09-13T09:14:16.731 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 2 2024-09-13T09:14:16.732 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:16.733 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:14:16.742 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:16.741+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:16.756 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:14:16.842 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:16.842+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:16.893 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:16.893 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:16.893 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:16.893 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:16.893 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:16.905 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:14:16.949 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:16.948+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:17.019 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:17.019 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:17.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:17.019 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:17.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.019 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29717 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [watch reconnect cookie 140383290562704 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:14:17.019 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:17.019 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.2647895250000001, 2024-09-13T09:14:17.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.006472936, 2024-09-13T09:14:17.020 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.020 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.020 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.020 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:17.020 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:17.020 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29717 2024-09-13T09:14:17.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.020 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:17.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:17.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751547+0000", 2024-09-13T09:14:17.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:17.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751552+0000", 2024-09-13T09:14:17.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:14:17.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751564+0000", 2024-09-13T09:14:17.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1793e-05 2024-09-13T09:14:17.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751571+0000", 2024-09-13T09:14:17.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2660000000000004e-06 2024-09-13T09:14:17.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753102+0000", 2024-09-13T09:14:17.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001531085 2024-09-13T09:14:17.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753115+0000", 2024-09-13T09:14:17.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3039e-05 2024-09-13T09:14:17.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.752858+0000", 2024-09-13T09:14:17.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.999742715 2024-09-13T09:14:17.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:17.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.753010+0000", 2024-09-13T09:14:17.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015227799999999999 2024-09-13T09:14:17.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755125+0000", 2024-09-13T09:14:17.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021150029999999998 2024-09-13T09:14:17.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-09-13T09:14:17.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755235+0000", 2024-09-13T09:14:17.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000109775 2024-09-13T09:14:17.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756059+0000", 2024-09-13T09:14:17.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00082370900000000005 2024-09-13T09:14:17.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756154+0000", 2024-09-13T09:14:17.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4945999999999995e-05 2024-09-13T09:14:17.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758021+0000", 2024-09-13T09:14:17.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001867446 2024-09-13T09:14:17.026 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.027 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.027 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.027 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29718 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:14:17.027 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:17.027 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.26475666, 2024-09-13T09:14:17.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0067812169999999, 2024-09-13T09:14:17.027 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.027 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:17.027 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.027 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:17.027 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:17.027 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29718 2024-09-13T09:14:17.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.028 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:17.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:17.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:17.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751585+0000", 2024-09-13T09:14:17.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2050000000000004e-06 2024-09-13T09:14:17.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751591+0000", 2024-09-13T09:14:17.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.37e-06 2024-09-13T09:14:17.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751597+0000", 2024-09-13T09:14:17.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1190000000000002e-06 2024-09-13T09:14:17.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753150+0000", 2024-09-13T09:14:17.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001552351 2024-09-13T09:14:17.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753164+0000", 2024-09-13T09:14:17.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4076e-05 2024-09-13T09:14:17.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.753307+0000", 2024-09-13T09:14:17.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0001434229999999 2024-09-13T09:14:17.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:17.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.753365+0000", 2024-09-13T09:14:17.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8014999999999999e-05 2024-09-13T09:14:17.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755501+0000", 2024-09-13T09:14:17.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021357260000000001 2024-09-13T09:14:17.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-09-13T09:14:17.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755558+0000", 2024-09-13T09:14:17.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7516000000000003e-05 2024-09-13T09:14:17.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756684+0000", 2024-09-13T09:14:17.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011259550000000001 2024-09-13T09:14:17.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756766+0000", 2024-09-13T09:14:17.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1755999999999994e-05 2024-09-13T09:14:17.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:17.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.757147+0000", 2024-09-13T09:14:17.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000381107 2024-09-13T09:14:17.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:17.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.757890+0000", 2024-09-13T09:14:17.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00074321499999999998 2024-09-13T09:14:17.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:17.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758178+0000", 2024-09-13T09:14:17.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00028755000000000002 2024-09-13T09:14:17.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:17.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758230+0000", 2024-09-13T09:14:17.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2317000000000001e-05 2024-09-13T09:14:17.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758362+0000", 2024-09-13T09:14:17.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013204100000000001 2024-09-13T09:14:17.036 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.036 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.036 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.036 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29724 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [watch unwatch cookie 140383290562704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-13T09:14:17.036 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.765164+0000", 2024-09-13T09:14:17.036 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.25117362599999998, 2024-09-13T09:14:17.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002552047, 2024-09-13T09:14:17.036 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.036 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:17.036 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.037 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:17.037 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:17.037 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29724 2024-09-13T09:14:17.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.037 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.765164+0000", 2024-09-13T09:14:17.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.765164+0000", 2024-09-13T09:14:17.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.765163+0000", 2024-09-13T09:14:17.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.765168+0000", 2024-09-13T09:14:17.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0199999999999996e-06 2024-09-13T09:14:17.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.765175+0000", 2024-09-13T09:14:17.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5469999999999999e-06 2024-09-13T09:14:17.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.765181+0000", 2024-09-13T09:14:17.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9909999999999999e-06 2024-09-13T09:14:17.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.766127+0000", 2024-09-13T09:14:17.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00094562900000000002 2024-09-13T09:14:17.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.766228+0000", 2024-09-13T09:14:17.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000101433 2024-09-13T09:14:17.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:17.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.766439+0000", 2024-09-13T09:14:17.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00021114699999999999 2024-09-13T09:14:17.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:17.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.767384+0000", 2024-09-13T09:14:17.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00094472499999999999 2024-09-13T09:14:17.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:17.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.767455+0000", 2024-09-13T09:14:17.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0560000000000002e-05 2024-09-13T09:14:17.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:17.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.767511+0000", 2024-09-13T09:14:17.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6919999999999997e-05 2024-09-13T09:14:17.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.767716+0000", 2024-09-13T09:14:17.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00020470199999999999 2024-09-13T09:14:17.042 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.042 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.043 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.043 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGPush(3.d 123/122 [PushOp(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29, version: 99'289, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29@99'289, size: 12582912, copy_subset: [0~12582912], clone_subset: {}, snapset: 29=[]:{29=[29]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:14:17.043 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.774123+0000", 2024-09-13T09:14:17.043 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.24221499499999999, 2024-09-13T09:14:17.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051988820000000002, 2024-09-13T09:14:17.043 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.043 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.043 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.774123+0000", 2024-09-13T09:14:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.774123+0000", 2024-09-13T09:14:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.774122+0000", 2024-09-13T09:14:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777696+0000", 2024-09-13T09:14:17.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003573676 2024-09-13T09:14:17.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777708+0000", 2024-09-13T09:14:17.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2349e-05 2024-09-13T09:14:17.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777718+0000", 2024-09-13T09:14:17.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0013999999999999e-05 2024-09-13T09:14:17.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777761+0000", 2024-09-13T09:14:17.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3167999999999997e-05 2024-09-13T09:14:17.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777771+0000", 2024-09-13T09:14:17.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3600000000000002e-06 2024-09-13T09:14:17.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.779322+0000", 2024-09-13T09:14:17.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015509740000000001 2024-09-13T09:14:17.046 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.047 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.047 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.047 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGPush(3.d 123/122 [PushOp(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29, version: 99'289, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29@99'289, size: 12582912, copy_subset: [0~12582912], clone_subset: {}, snapset: 29=[]:{29=[29]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-13T09:14:17.047 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.781564+0000", 2024-09-13T09:14:17.047 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.234774072, 2024-09-13T09:14:17.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025368389999999999, 2024-09-13T09:14:17.047 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.047 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.047 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.781564+0000", 2024-09-13T09:14:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.781564+0000", 2024-09-13T09:14:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.781563+0000", 2024-09-13T09:14:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.782887+0000", 2024-09-13T09:14:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0013238099999999999 2024-09-13T09:14:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.782904+0000", 2024-09-13T09:14:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7530999999999999e-05 2024-09-13T09:14:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.782928+0000", 2024-09-13T09:14:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3652999999999999e-05 2024-09-13T09:14:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.782996+0000", 2024-09-13T09:14:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7762999999999995e-05 2024-09-13T09:14:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.783008+0000", 2024-09-13T09:14:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1973e-05 2024-09-13T09:14:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.784101+0000", 2024-09-13T09:14:17.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010927809999999999 2024-09-13T09:14:17.051 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.051 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.051 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.051 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGPush(3.7 123/122 [PushOp(3:e50afc48:::rbd_data.112339c5624d.0000000000000005:head, version: 88'878, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:e50afc48:::rbd_data.112339c5624d.0000000000000005:head@88'878, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:0, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:14:17.051 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.783169+0000", 2024-09-13T09:14:17.051 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.23316843600000001, 2024-09-13T09:14:17.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025101429999999998, 2024-09-13T09:14:17.052 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.052 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.052 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.783169+0000", 2024-09-13T09:14:17.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.783169+0000", 2024-09-13T09:14:17.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.783169+0000", 2024-09-13T09:14:17.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.783186+0000", 2024-09-13T09:14:17.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7433999999999999e-05 2024-09-13T09:14:17.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.783198+0000", 2024-09-13T09:14:17.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.169e-05 2024-09-13T09:14:17.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.783206+0000", 2024-09-13T09:14:17.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8890000000000002e-06 2024-09-13T09:14:17.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.783271+0000", 2024-09-13T09:14:17.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5296000000000006e-05 2024-09-13T09:14:17.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.783280+0000", 2024-09-13T09:14:17.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9330000000000007e-06 2024-09-13T09:14:17.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.785679+0000", 2024-09-13T09:14:17.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023994630000000001 2024-09-13T09:14:17.055 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.055 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.056 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.056 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29738 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123)", 2024-09-13T09:14:17.056 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.783728+0000", 2024-09-13T09:14:17.056 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.23260974400000001, 2024-09-13T09:14:17.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025913109999999998, 2024-09-13T09:14:17.056 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.056 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.056 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.056 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:17.057 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:17.057 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29738 2024-09-13T09:14:17.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.057 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.783728+0000", 2024-09-13T09:14:17.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.783728+0000", 2024-09-13T09:14:17.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.783726+0000", 2024-09-13T09:14:17.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-09-13T09:14:17.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.783734+0000", 2024-09-13T09:14:17.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7069999999999992e-06 2024-09-13T09:14:17.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.783752+0000", 2024-09-13T09:14:17.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8439000000000001e-05 2024-09-13T09:14:17.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.783782+0000", 2024-09-13T09:14:17.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9431999999999999e-05 2024-09-13T09:14:17.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786003+0000", 2024-09-13T09:14:17.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022211589999999999 2024-09-13T09:14:17.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786098+0000", 2024-09-13T09:14:17.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5589999999999998e-05 2024-09-13T09:14:17.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786319+0000", 2024-09-13T09:14:17.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000220845 2024-09-13T09:14:17.060 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.060 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.060 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.060 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGPush(3.d 123/122 [PushOp(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29, version: 99'289, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29@99'289, size: 12582912, copy_subset: [0~12582912], clone_subset: {}, snapset: 29=[]:{29=[29]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-13T09:14:17.060 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.785201+0000", 2024-09-13T09:14:17.060 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.23113655399999999, 2024-09-13T09:14:17.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0024061320000000001, 2024-09-13T09:14:17.061 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.061 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.061 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.061 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.785201+0000", 2024-09-13T09:14:17.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.061 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.785201+0000", 2024-09-13T09:14:17.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.785201+0000", 2024-09-13T09:14:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786414+0000", 2024-09-13T09:14:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0012133770000000001 2024-09-13T09:14:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786433+0000", 2024-09-13T09:14:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9216e-05 2024-09-13T09:14:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786452+0000", 2024-09-13T09:14:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9162999999999999e-05 2024-09-13T09:14:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786530+0000", 2024-09-13T09:14:17.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7249000000000005e-05 2024-09-13T09:14:17.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786544+0000", 2024-09-13T09:14:17.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4635e-05 2024-09-13T09:14:17.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.787607+0000", 2024-09-13T09:14:17.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001063168 2024-09-13T09:14:17.064 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.065 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.065 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.065 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29739 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [watch watch cookie 140383086477584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-13T09:14:17.065 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.786571+0000", 2024-09-13T09:14:17.065 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.22976685899999999, 2024-09-13T09:14:17.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020597010000000002, 2024-09-13T09:14:17.065 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.065 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:17.065 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.065 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:17.065 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:17.066 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29739 2024-09-13T09:14:17.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.066 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786571+0000", 2024-09-13T09:14:17.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786571+0000", 2024-09-13T09:14:17.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786569+0000", 2024-09-13T09:14:17.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-09-13T09:14:17.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786576+0000", 2024-09-13T09:14:17.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5819999999999998e-06 2024-09-13T09:14:17.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786602+0000", 2024-09-13T09:14:17.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6310000000000001e-05 2024-09-13T09:14:17.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786621+0000", 2024-09-13T09:14:17.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9165e-05 2024-09-13T09:14:17.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786670+0000", 2024-09-13T09:14:17.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8263000000000002e-05 2024-09-13T09:14:17.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786763+0000", 2024-09-13T09:14:17.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2996000000000002e-05 2024-09-13T09:14:17.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:17.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.786940+0000", 2024-09-13T09:14:17.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017701199999999999 2024-09-13T09:14:17.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.070 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:17.070 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.788444+0000", 2024-09-13T09:14:17.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001504736 2024-09-13T09:14:17.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.070 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:17.070 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.788473+0000", 2024-09-13T09:14:17.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8507999999999999e-05 2024-09-13T09:14:17.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.070 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:17.070 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.788550+0000", 2024-09-13T09:14:17.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6723000000000004e-05 2024-09-13T09:14:17.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.788631+0000", 2024-09-13T09:14:17.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0884999999999996e-05 2024-09-13T09:14:17.071 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.071 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.071 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.071 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29741 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-13T09:14:17.071 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.789601+0000", 2024-09-13T09:14:17.071 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.226737092, 2024-09-13T09:14:17.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025299519999999998, 2024-09-13T09:14:17.071 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.072 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:17.072 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.072 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:17.072 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:17.072 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29741 2024-09-13T09:14:17.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.072 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.789601+0000", 2024-09-13T09:14:17.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.073 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.789601+0000", 2024-09-13T09:14:17.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.073 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.789600+0000", 2024-09-13T09:14:17.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:17.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.074 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.074 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.789604+0000", 2024-09-13T09:14:17.074 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5979999999999999e-06 2024-09-13T09:14:17.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.074 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.074 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.789614+0000", 2024-09-13T09:14:17.074 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2150000000000007e-06 2024-09-13T09:14:17.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.074 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.074 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.789619+0000", 2024-09-13T09:14:17.074 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2460000000000003e-06 2024-09-13T09:14:17.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.075 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.075 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.789770+0000", 2024-09-13T09:14:17.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000151061 2024-09-13T09:14:17.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.075 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.075 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.789856+0000", 2024-09-13T09:14:17.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489000000000005e-05 2024-09-13T09:14:17.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.075 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:17.075 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.789975+0000", 2024-09-13T09:14:17.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000118711 2024-09-13T09:14:17.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.076 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:17.076 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.792005+0000", 2024-09-13T09:14:17.076 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020301300000000002 2024-09-13T09:14:17.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.076 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:17.076 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.792051+0000", 2024-09-13T09:14:17.076 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5262999999999997e-05 2024-09-13T09:14:17.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:17.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.792095+0000", 2024-09-13T09:14:17.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4106000000000001e-05 2024-09-13T09:14:17.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.792131+0000", 2024-09-13T09:14:17.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5904e-05 2024-09-13T09:14:17.077 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.077 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.077 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.078 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGPush(3.d 123/122 [PushOp(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29, version: 99'289, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29@99'289, size: 12582912, copy_subset: [0~12582912], clone_subset: {}, snapset: 29=[]:{29=[29]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-13T09:14:17.078 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.791840+0000", 2024-09-13T09:14:17.078 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.22449749399999999, 2024-09-13T09:14:17.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0031981169999999999, 2024-09-13T09:14:17.078 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.078 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.078 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.791840+0000", 2024-09-13T09:14:17.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.791840+0000", 2024-09-13T09:14:17.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.791840+0000", 2024-09-13T09:14:17.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.080 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.080 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.793937+0000", 2024-09-13T09:14:17.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020970369999999999 2024-09-13T09:14:17.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.080 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.081 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.793955+0000", 2024-09-13T09:14:17.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7859999999999998e-05 2024-09-13T09:14:17.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.081 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.081 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.793968+0000", 2024-09-13T09:14:17.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3349e-05 2024-09-13T09:14:17.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.081 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.081 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.794034+0000", 2024-09-13T09:14:17.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6530000000000002e-05 2024-09-13T09:14:17.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.082 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.082 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.794049+0000", 2024-09-13T09:14:17.082 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5014999999999999e-05 2024-09-13T09:14:17.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.082 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.082 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.795038+0000", 2024-09-13T09:14:17.082 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098897200000000007 2024-09-13T09:14:17.082 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.082 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.083 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.083 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGPush(3.7 123/122 [PushOp(3:e2217efe:::rbd_data.1123c86d266d.0000000000000000:23, version: 92'895, data_included: [65536~71680,196608~65536,327680~73728,458752~16384,942080~32768,3227648~24576,3493888~45056,4517888~4096,5140480~36864,6881280~28160], data_size: 398848, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:e2217efe:::rbd_data.1123c86d266d.0000000000000000:23@92'895, size: 13145088, copy_subset: [0~13145088], clone_subset: {}, snapset: 23=[]:{23=[23]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:13145088, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:14:17.083 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.793662+0000", 2024-09-13T09:14:17.083 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.22267600400000001, 2024-09-13T09:14:17.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025507770000000002, 2024-09-13T09:14:17.083 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.083 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.083 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.083 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.083 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.793662+0000", 2024-09-13T09:14:17.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.793662+0000", 2024-09-13T09:14:17.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.793661+0000", 2024-09-13T09:14:17.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:17.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.794117+0000", 2024-09-13T09:14:17.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00045556600000000002 2024-09-13T09:14:17.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.794132+0000", 2024-09-13T09:14:17.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5597999999999999e-05 2024-09-13T09:14:17.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.794145+0000", 2024-09-13T09:14:17.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2724e-05 2024-09-13T09:14:17.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.794253+0000", 2024-09-13T09:14:17.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000108435 2024-09-13T09:14:17.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.794270+0000", 2024-09-13T09:14:17.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6922000000000002e-05 2024-09-13T09:14:17.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.796213+0000", 2024-09-13T09:14:17.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019423000000000001 2024-09-13T09:14:17.087 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.087 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.087 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.087 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29749 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [watch reconnect cookie 140383086477584 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-13T09:14:17.087 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:17.087 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.21691780199999999, 2024-09-13T09:14:17.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037671060000000001, 2024-09-13T09:14:17.087 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.087 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.087 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.087 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:17.087 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:17.088 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29749 2024-09-13T09:14:17.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.088 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:17.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:17.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:17.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799422+0000", 2024-09-13T09:14:17.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:14:17.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799429+0000", 2024-09-13T09:14:17.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4670000000000001e-06 2024-09-13T09:14:17.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799434+0000", 2024-09-13T09:14:17.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0520000000000001e-06 2024-09-13T09:14:17.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.801379+0000", 2024-09-13T09:14:17.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019449599999999999 2024-09-13T09:14:17.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.801426+0000", 2024-09-13T09:14:17.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7391000000000001e-05 2024-09-13T09:14:17.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.092 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.803187+0000", 2024-09-13T09:14:17.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0017609870000000001 2024-09-13T09:14:17.092 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.092 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.092 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.092 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29751 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-13T09:14:17.092 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.802816+0000", 2024-09-13T09:14:17.092 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.21352158700000001, 2024-09-13T09:14:17.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002360549, 2024-09-13T09:14:17.092 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.092 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:17.093 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.093 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:17.093 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:17.093 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29751 2024-09-13T09:14:17.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.093 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.093 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.802816+0000", 2024-09-13T09:14:17.093 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.093 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.802816+0000", 2024-09-13T09:14:17.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.802815+0000", 2024-09-13T09:14:17.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-09-13T09:14:17.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.802821+0000", 2024-09-13T09:14:17.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4359999999999999e-06 2024-09-13T09:14:17.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.802835+0000", 2024-09-13T09:14:17.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4236000000000001e-05 2024-09-13T09:14:17.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.802843+0000", 2024-09-13T09:14:17.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5379999999999997e-06 2024-09-13T09:14:17.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.803220+0000", 2024-09-13T09:14:17.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00037742199999999998 2024-09-13T09:14:17.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.803295+0000", 2024-09-13T09:14:17.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4547000000000003e-05 2024-09-13T09:14:17.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:17.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.803400+0000", 2024-09-13T09:14:17.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000104789 2024-09-13T09:14:17.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:17.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.804509+0000", 2024-09-13T09:14:17.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011092739999999999 2024-09-13T09:14:17.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.098 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:17.098 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.805113+0000", 2024-09-13T09:14:17.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00060408 2024-09-13T09:14:17.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.805141+0000", 2024-09-13T09:14:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8212999999999999e-05 2024-09-13T09:14:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.805177+0000", 2024-09-13T09:14:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5540000000000002e-05 2024-09-13T09:14:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.3.0:2633 3.d e123/122 3:b54ae000:::rbd_data.1123af309369.0000000000000068:1f v 123'326, mlcod=123'321)", 2024-09-13T09:14:17.099 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.827442+0000", 2024-09-13T09:14:17.100 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.18889563200000001, 2024-09-13T09:14:17.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002496313, 2024-09-13T09:14:17.100 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.100 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:17.100 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.100 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.827442+0000", 2024-09-13T09:14:17.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.100 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.827442+0000", 2024-09-13T09:14:17.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.827441+0000", 2024-09-13T09:14:17.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.827448+0000", 2024-09-13T09:14:17.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:14:17.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.827456+0000", 2024-09-13T09:14:17.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6090000000000001e-06 2024-09-13T09:14:17.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.827460+0000", 2024-09-13T09:14:17.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0500000000000002e-06 2024-09-13T09:14:17.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.827501+0000", 2024-09-13T09:14:17.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0596000000000002e-05 2024-09-13T09:14:17.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.103 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.827521+0000", 2024-09-13T09:14:17.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9936999999999999e-05 2024-09-13T09:14:17.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.103 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:17.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.829907+0000", 2024-09-13T09:14:17.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023862850000000001 2024-09-13T09:14:17.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.104 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.104 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.829938+0000", 2024-09-13T09:14:17.104 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1288999999999998e-05 2024-09-13T09:14:17.104 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.104 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.104 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.104 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.3.0:2634 3.d e123/122 3:b38c9f62:::rbd_data.1123af309369.0000000000000089:1f v 123'328, mlcod=123'321)", 2024-09-13T09:14:17.104 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.828263+0000", 2024-09-13T09:14:17.104 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.18807452799999999, 2024-09-13T09:14:17.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019775439999999999, 2024-09-13T09:14:17.105 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.105 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:17.105 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.105 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.105 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.828263+0000", 2024-09-13T09:14:17.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.105 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.105 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.828263+0000", 2024-09-13T09:14:17.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.828263+0000", 2024-09-13T09:14:17.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:17.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.828269+0000", 2024-09-13T09:14:17.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3790000000000003e-06 2024-09-13T09:14:17.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.828284+0000", 2024-09-13T09:14:17.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4973999999999999e-05 2024-09-13T09:14:17.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.828293+0000", 2024-09-13T09:14:17.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:14:17.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.829294+0000", 2024-09-13T09:14:17.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010012759999999999 2024-09-13T09:14:17.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.829314+0000", 2024-09-13T09:14:17.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0047999999999998e-05 2024-09-13T09:14:17.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:17.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.830206+0000", 2024-09-13T09:14:17.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00089248099999999998 2024-09-13T09:14:17.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.830241+0000", 2024-09-13T09:14:17.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4306e-05 2024-09-13T09:14:17.108 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.108 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.109 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.109 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.3.0:2636 3.d e123/122 3:b72d33da:::rbd_data.1123af309369.0000000000000396:1f v 123'331, mlcod=123'326)", 2024-09-13T09:14:17.109 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.831345+0000", 2024-09-13T09:14:17.109 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.18499237700000001, 2024-09-13T09:14:17.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026913789999999998, 2024-09-13T09:14:17.109 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.109 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:17.109 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.109 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.831345+0000", 2024-09-13T09:14:17.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.110 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.110 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.831345+0000", 2024-09-13T09:14:17.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.110 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.110 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.831345+0000", 2024-09-13T09:14:17.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.831351+0000", 2024-09-13T09:14:17.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5660000000000003e-06 2024-09-13T09:14:17.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.831373+0000", 2024-09-13T09:14:17.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2144e-05 2024-09-13T09:14:17.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.831380+0000", 2024-09-13T09:14:17.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3629999999999999e-06 2024-09-13T09:14:17.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.112 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.831450+0000", 2024-09-13T09:14:17.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0320000000000004e-05 2024-09-13T09:14:17.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.112 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.831466+0000", 2024-09-13T09:14:17.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5483000000000001e-05 2024-09-13T09:14:17.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:17.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.834013+0000", 2024-09-13T09:14:17.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025473919999999999 2024-09-13T09:14:17.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.834037+0000", 2024-09-13T09:14:17.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3771e-05 2024-09-13T09:14:17.113 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.113 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.113 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.113 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.3.0:2637 3.d e123/122 3:b7539c64:::rbd_data.1123af309369.0000000000000066:1f v 123'333, mlcod=123'326)", 2024-09-13T09:14:17.114 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.831647+0000", 2024-09-13T09:14:17.114 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.184691047, 2024-09-13T09:14:17.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029188909999999998, 2024-09-13T09:14:17.114 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.114 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:17.114 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.831647+0000", 2024-09-13T09:14:17.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.831647+0000", 2024-09-13T09:14:17.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.831646+0000", 2024-09-13T09:14:17.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.831651+0000", 2024-09-13T09:14:17.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4589999999999997e-06 2024-09-13T09:14:17.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.116 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.116 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.831666+0000", 2024-09-13T09:14:17.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4602e-05 2024-09-13T09:14:17.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.116 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.116 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.831673+0000", 2024-09-13T09:14:17.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8059999999999997e-06 2024-09-13T09:14:17.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.117 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.117 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.832329+0000", 2024-09-13T09:14:17.117 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00065609200000000005 2024-09-13T09:14:17.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.117 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.117 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.832347+0000", 2024-09-13T09:14:17.117 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8533e-05 2024-09-13T09:14:17.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:17.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.834535+0000", 2024-09-13T09:14:17.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021870629999999999 2024-09-13T09:14:17.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.834566+0000", 2024-09-13T09:14:17.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1043000000000001e-05 2024-09-13T09:14:17.118 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.118 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.118 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.119 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop_reply(osd.2.0:33 3.7 e123/122 ondisk, result = 0)", 2024-09-13T09:14:17.119 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.852900+0000", 2024-09-13T09:14:17.119 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.16343735400000001, 2024-09-13T09:14:17.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020484370000000002, 2024-09-13T09:14:17.119 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.119 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.119 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.852900+0000", 2024-09-13T09:14:17.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.852900+0000", 2024-09-13T09:14:17.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.852900+0000", 2024-09-13T09:14:17.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.852905+0000", 2024-09-13T09:14:17.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7350000000000004e-06 2024-09-13T09:14:17.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.852913+0000", 2024-09-13T09:14:17.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5529999999999993e-06 2024-09-13T09:14:17.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.852919+0000", 2024-09-13T09:14:17.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:14:17.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.854872+0000", 2024-09-13T09:14:17.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001953237 2024-09-13T09:14:17.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.854881+0000", 2024-09-13T09:14:17.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6119999999999999e-06 2024-09-13T09:14:17.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.854949+0000", 2024-09-13T09:14:17.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8306999999999995e-05 2024-09-13T09:14:17.123 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.123 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.123 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.123 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29818 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-13T09:14:17.123 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.876652+0000", 2024-09-13T09:14:17.123 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.13968592699999999, 2024-09-13T09:14:17.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002623677, 2024-09-13T09:14:17.124 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.124 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:17.124 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.124 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:17.124 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:17.124 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29818 2024-09-13T09:14:17.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.124 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.124 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.124 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.876652+0000", 2024-09-13T09:14:17.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.876652+0000", 2024-09-13T09:14:17.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.876650+0000", 2024-09-13T09:14:17.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-09-13T09:14:17.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.876657+0000", 2024-09-13T09:14:17.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5699999999999998e-06 2024-09-13T09:14:17.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.876670+0000", 2024-09-13T09:14:17.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2833999999999999e-05 2024-09-13T09:14:17.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.876676+0000", 2024-09-13T09:14:17.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7610000000000002e-06 2024-09-13T09:14:17.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.876723+0000", 2024-09-13T09:14:17.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7073999999999997e-05 2024-09-13T09:14:17.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.876819+0000", 2024-09-13T09:14:17.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5760000000000005e-05 2024-09-13T09:14:17.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:17.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.876997+0000", 2024-09-13T09:14:17.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017795699999999999 2024-09-13T09:14:17.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:17.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.877966+0000", 2024-09-13T09:14:17.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00096919300000000003 2024-09-13T09:14:17.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:17.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.879186+0000", 2024-09-13T09:14:17.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0012195590000000001 2024-09-13T09:14:17.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:17.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.879234+0000", 2024-09-13T09:14:17.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7904000000000001e-05 2024-09-13T09:14:17.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.879275+0000", 2024-09-13T09:14:17.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1696000000000002e-05 2024-09-13T09:14:17.129 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.129 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.129 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.129 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.129 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.130 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:17.131 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:17.130+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:17.131 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:14:17.204 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:14:17.231 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:17.231+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:17.279 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:17.279 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:17.288 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:14:17.333 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:17.331+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:17.338 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:14:17.338 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:14:17.355 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:14:17.356 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:14:17.372 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:14:17.372 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-13T09:14:17.393 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:14:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:14:17.396 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:17.396 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:17.396 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:17.396 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:17.396 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:17.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:14:17.414 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:14:17.414 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:14:17.432 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:14:17.433 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:14:17.433 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:17.432+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:17.523 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:17.523 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:17.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:17.524 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:17.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.524 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.524 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:17.524 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.992215118999994, 2024-09-13T09:14:17.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:14:17.524 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.524 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.524 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.524 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.525 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.525 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:14:17.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.525 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:17.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:17.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:14:17.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:14:17.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:14:17.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:14:17.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:14:17.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:14:17.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:14:17.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:14:17.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:14:17.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:14:17.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:14:17.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:14:17.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:14:17.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:14:17.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:14:17.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:14:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:14:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:14:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:14:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:14:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:14:17.529 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.530 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.530 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.530 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.530 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:17.530 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.992197083999997, 2024-09-13T09:14:17.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:14:17.530 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.530 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.530 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.531 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.531 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.531 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:14:17.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.531 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:17.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:17.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.532 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:17.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.532 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:14:17.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:14:17.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:14:17.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:14:17.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:14:17.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:14:17.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:14:17.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:14:17.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:14:17.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:14:17.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:14:17.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:14:17.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:14:17.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:14:17.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:14:17.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:14:17.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:14:17.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:14:17.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:14:17.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:14:17.536 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.536 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.536 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.536 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.536 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:17.536 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.992180414000003, 2024-09-13T09:14:17.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:14:17.536 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.536 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.536 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.536 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.537 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.537 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:14:17.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.537 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:17.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:17.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:14:17.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:14:17.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:17.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:14:17.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:14:17.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:14:17.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:14:17.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:14:17.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:14:17.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:14:17.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:14:17.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:14:17.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:14:17.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:14:17.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:14:17.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.541 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:14:17.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:14:17.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.541 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:14:17.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:14:17.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:14:17.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:14:17.542 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.542 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.542 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.542 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.542 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:17.542 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.992162781999994, 2024-09-13T09:14:17.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:14:17.542 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.543 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.543 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.543 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.543 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.543 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:14:17.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.543 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.543 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.543 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:17.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:17.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:17.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:14:17.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:14:17.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:14:17.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:14:17.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:14:17.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:14:17.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:14:17.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:14:17.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:14:17.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:14:17.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:14:17.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:14:17.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:14:17.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:14:17.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:14:17.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:14:17.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:14:17.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:14:17.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:14:17.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:14:17.548 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.548 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.548 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.548 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.548 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:17.549 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.992116476999996, 2024-09-13T09:14:17.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:14:17.549 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.549 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.549 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.549 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.549 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.549 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:14:17.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.549 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:17.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:17.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:14:17.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:14:17.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:14:17.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:14:17.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:14:17.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:14:17.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:14:17.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:14:17.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:14:17.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:14:17.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:14:17.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:14:17.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:14:17.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:14:17.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:14:17.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:14:17.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:14:17.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:14:17.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:14:17.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:14:17.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:14:17.555 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.555 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.555 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.555 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.555 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:17.555 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.992091176000002, 2024-09-13T09:14:17.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:14:17.556 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.556 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.556 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.556 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.556 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.556 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:14:17.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.556 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:17.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:17.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:14:17.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:14:17.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:14:17.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:14:17.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:14:17.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:14:17.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:14:17.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:14:17.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:14:17.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:14:17.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:14:17.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:14:17.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:14:17.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:14:17.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:14:17.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:14:17.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:14:17.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:14:17.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:14:17.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:14:17.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:14:17.561 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.561 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.561 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.561 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.561 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:17.561 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.992072946999997, 2024-09-13T09:14:17.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:14:17.562 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.562 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.562 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.562 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.562 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.562 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:14:17.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.562 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:17.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:17.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:17.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:14:17.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:14:17.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:14:17.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:14:17.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:14:17.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:14:17.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:14:17.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:14:17.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:14:17.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:14:17.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:14:17.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:14:17.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:14:17.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:14:17.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:14:17.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:14:17.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:14:17.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:14:17.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:14:17.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:14:17.567 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.567 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.567 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.567 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.567 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:17.567 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.992025627999993, 2024-09-13T09:14:17.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:14:17.567 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.567 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.568 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.568 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.568 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.568 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:14:17.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.568 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:17.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:17.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:17.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:14:17.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:14:17.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:14:17.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:14:17.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:14:17.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:14:17.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:14:17.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:14:17.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:14:17.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:14:17.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:14:17.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:14:17.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:14:17.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:14:17.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:14:17.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:14:17.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:14:17.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:14:17.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:14:17.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:14:17.573 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.574 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.574 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.574 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.574 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:17.574 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.991511389999999, 2024-09-13T09:14:17.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:14:17.574 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.574 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.574 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.574 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.575 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.575 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:14:17.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.575 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:17.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:17.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:14:17.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:17.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:14:17.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:14:17.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:14:17.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:14:17.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:14:17.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:14:17.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:14:17.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:14:17.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:14:17.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:14:17.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:14:17.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:14:17.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:14:17.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:14:17.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:14:17.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:14:17.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:14:17.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:14:17.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:14:17.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:14:17.580 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.580 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.580 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.580 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.580 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:17.580 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.991476860000006, 2024-09-13T09:14:17.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:14:17.581 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.581 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.581 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.581 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.581 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.581 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:14:17.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.581 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:17.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:17.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:17.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:14:17.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:14:17.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:14:17.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:14:17.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:14:17.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:14:17.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:14:17.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:14:17.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:14:17.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:14:17.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:14:17.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:14:17.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:14:17.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:14:17.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:14:17.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:14:17.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:14:17.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:14:17.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:14:17.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:14:17.589 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.589 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.589 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.590 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.590 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:17.590 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.991458953000006, 2024-09-13T09:14:17.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:14:17.590 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.590 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.590 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.590 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.590 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.590 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:14:17.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.590 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:17.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:17.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:17.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:14:17.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:14:17.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:14:17.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:14:17.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:14:17.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:14:17.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:14:17.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:14:17.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:14:17.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:14:17.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:14:17.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:14:17.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:14:17.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:14:17.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:14:17.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:14:17.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:14:17.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:14:17.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:14:17.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:14:17.595 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.595 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.596 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.596 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.596 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:17.596 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.982332104999998, 2024-09-13T09:14:17.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:14:17.596 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.596 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.596 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.596 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.596 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.596 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:14:17.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.597 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:17.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:17.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:17.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:14:17.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:14:17.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:14:17.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:14:17.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:14:17.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:14:17.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:14:17.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:14:17.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:14:17.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:14:17.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:14:17.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:14:17.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:14:17.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:14:17.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:14:17.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:14:17.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:14:17.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:14:17.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:14:17.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:14:17.601 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.602 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.602 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.602 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.602 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:17.602 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.982280524999993, 2024-09-13T09:14:17.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:14:17.602 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.602 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.602 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.602 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.602 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.603 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:14:17.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.603 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:17.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:17.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:14:17.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:14:17.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:14:17.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:14:17.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:14:17.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:14:17.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:14:17.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:14:17.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:14:17.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:14:17.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:14:17.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:14:17.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:14:17.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:14:17.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:14:17.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:14:17.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:14:17.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:14:17.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:14:17.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:14:17.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:14:17.608 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.608 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.608 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.608 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.608 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:17.608 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.982264248999996, 2024-09-13T09:14:17.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:14:17.608 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.608 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.609 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.609 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.609 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.609 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:14:17.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.609 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:17.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:17.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:14:17.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:14:17.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:14:17.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:14:17.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:14:17.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:14:17.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:14:17.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:14:17.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:14:17.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:14:17.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:14:17.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:14:17.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:14:17.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:14:17.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:14:17.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:14:17.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:14:17.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:14:17.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:14:17.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:14:17.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:14:17.614 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.614 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.614 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.614 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.614 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:17.614 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.982248475000006, 2024-09-13T09:14:17.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:14:17.614 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.614 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.614 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.615 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.615 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.615 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:14:17.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.615 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:17.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:17.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:14:17.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:14:17.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:14:17.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:14:17.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:14:17.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:14:17.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:14:17.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:14:17.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:14:17.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:14:17.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:14:17.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:14:17.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:14:17.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:14:17.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:14:17.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.620 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.620 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.620 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.620 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:17.620 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.982233600000001, 2024-09-13T09:14:17.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:14:17.620 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.620 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.620 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.620 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.620 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.621 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:14:17.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.621 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:17.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:17.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:17.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:14:17.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:14:17.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:14:17.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:14:17.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:14:17.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:14:17.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:14:17.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:14:17.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:14:17.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:14:17.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:14:17.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:14:17.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:14:17.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:14:17.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:14:17.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:14:17.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:14:17.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:14:17.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:14:17.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:14:17.626 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.626 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.626 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.626 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.626 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:17.626 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.982216137999998, 2024-09-13T09:14:17.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:14:17.626 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.626 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.626 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.626 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.626 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.627 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:14:17.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.627 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:17.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:17.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:14:17.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:14:17.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:14:17.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:14:17.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:14:17.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:14:17.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:14:17.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:14:17.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:14:17.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:14:17.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:14:17.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:14:17.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:14:17.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:14:17.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:14:17.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:14:17.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:14:17.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:14:17.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:14:17.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:14:17.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:14:17.631 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.631 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.632 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.632 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.632 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:17.632 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.980615506000007, 2024-09-13T09:14:17.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:14:17.632 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.632 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.632 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.632 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.632 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.632 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:14:17.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.634 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:17.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:17.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:14:17.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:14:17.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:14:17.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:14:17.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:14:17.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:14:17.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:14:17.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:14:17.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:14:17.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:14:17.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:14:17.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:14:17.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:14:17.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:14:17.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:14:17.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:14:17.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:14:17.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:14:17.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:14:17.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:14:17.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:14:17.638 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.638 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.639 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.639 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.639 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:17.639 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.980575454999993, 2024-09-13T09:14:17.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:14:17.639 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.639 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.640 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.640 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.640 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.640 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:14:17.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.640 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:17.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:17.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:14:17.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:14:17.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:14:17.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:14:17.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:14:17.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:14:17.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:14:17.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:14:17.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:14:17.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:14:17.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:14:17.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:14:17.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:14:17.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:14:17.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:14:17.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:14:17.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:14:17.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:14:17.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:14:17.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:14:17.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:14:17.645 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.645 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.645 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.645 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:17.645 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:17.645 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 94.980531692, 2024-09-13T09:14:17.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:14:17.645 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:17.645 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:14:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:17.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:17.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:17.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:17.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:17.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:17.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:14:17.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:14:17.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:17.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:14:17.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:14:17.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:17.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:14:17.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:14:17.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:14:17.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:14:17.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:14:17.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:14:17.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:14:17.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:14:17.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:17.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:14:17.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:14:17.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:17.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:14:17.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:14:17.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:17.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:14:17.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:14:17.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:17.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:17.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:17.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:14:17.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:14:17.650 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.650 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.650 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.651 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:17.651 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:17.651 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:17.652 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:14:17.653 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:17.651+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:17.747 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-09-13T09:14:17.753 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:17.752+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:17.816 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:17.816 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:17.816 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:17.817 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:17.825 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:14:17.869 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:17.868+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:17.945 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:17.945 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:17.945 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:17.945 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:17.945 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:17.955 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:14:17.955 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:17.954+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:18.056 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:18.055+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:18.104 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:18.104 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598780712999996, 2024-09-13T09:14:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:14:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:14:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:14:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:14:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:14:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:14:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:14:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:14:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:14:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:14:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:14:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:14:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:14:18.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:14:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:14:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598753028999994, 2024-09-13T09:14:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:14:18.109 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.110 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.110 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.110 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.110 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.110 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:14:18.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.110 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.110 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.110 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:18.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:14:18.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:14:18.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:14:18.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:14:18.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:14:18.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:14:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:14:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:14:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:14:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:14:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:14:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:14:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598728687999994, 2024-09-13T09:14:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:14:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.115 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:14:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:18.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:14:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:14:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:14:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.117 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:14:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:14:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:14:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:14:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:14:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:14:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:14:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:14:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:14:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:14:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:18.119 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598712367999994, 2024-09-13T09:14:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:14:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:14:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:18.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:14:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:14:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:14:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:14:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:14:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.122 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:14:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:14:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:14:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:14:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:14:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:14:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.123 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598695953000004, 2024-09-13T09:14:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:14:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:14:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.124 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:14:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:14:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:14:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:14:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:14:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:14:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:14:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:14:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:14:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:14:18.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:14:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:14:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598679845000007, 2024-09-13T09:14:18.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:14:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:14:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:14:18.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:14:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:14:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:14:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:14:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:14:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:14:18.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:14:18.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:14:18.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:14:18.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:14:18.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:14:18.132 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.132 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.133 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.133 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.133 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:18.133 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598664283999994, 2024-09-13T09:14:18.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:14:18.133 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.133 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.133 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.133 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.133 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.133 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:14:18.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.134 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.134 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:18.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.134 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:18.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.135 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:18.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.135 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:14:18.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:14:18.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.135 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:14:18.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:14:18.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:14:18.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:14:18.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:14:18.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:14:18.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:14:18.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:14:18.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:14:18.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:14:18.137 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.137 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.137 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.137 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.137 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:18.137 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598647446000001, 2024-09-13T09:14:18.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:14:18.138 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.138 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.138 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.138 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.138 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.138 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:14:18.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.138 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.138 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.138 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:18.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:18.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:14:18.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:14:18.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:14:18.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:14:18.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:14:18.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:14:18.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:14:18.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:14:18.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:14:18.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.141 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.141 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:14:18.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:14:18.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.141 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.141 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:14:18.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:14:18.142 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.142 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.142 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.142 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.142 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:18.142 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598631454, 2024-09-13T09:14:18.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:14:18.142 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.142 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.142 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.143 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.143 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.143 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:14:18.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.143 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:18.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:18.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:14:18.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:14:18.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:14:18.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:14:18.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:14:18.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:14:18.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:14:18.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:14:18.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:14:18.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:14:18.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:14:18.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:14:18.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:14:18.146 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.146 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.146 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.146 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.146 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:18.147 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598603956000005, 2024-09-13T09:14:18.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:14:18.147 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.147 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.147 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.147 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.147 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.147 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:14:18.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.147 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.147 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.147 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:18.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:18.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:18.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:14:18.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:14:18.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:14:18.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:14:18.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:14:18.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:14:18.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:14:18.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:14:18.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:14:18.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:14:18.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:14:18.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:14:18.150 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.151 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.151 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.151 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.151 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:18.151 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598578097000001, 2024-09-13T09:14:18.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:14:18.151 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.151 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.151 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.151 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.151 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.152 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:14:18.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.152 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:18.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:18.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:18.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:14:18.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:14:18.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:14:18.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:14:18.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:14:18.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:14:18.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:14:18.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:14:18.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:14:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:14:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:14:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:14:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.156 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:18.157 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598536607, 2024-09-13T09:14:18.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:14:18.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.157 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.157 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.157 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.157 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.157 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:14:18.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.157 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:18.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:18.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:14:18.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:14:18.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:14:18.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:14:18.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:14:18.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:14:18.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:14:18.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:14:18.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:14:18.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:14:18.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:14:18.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:14:18.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:14:18.161 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.161 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.161 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.162 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.162 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:18.162 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598515368999998, 2024-09-13T09:14:18.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:14:18.162 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.162 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.162 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.162 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.162 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.162 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:14:18.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.162 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:18.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:18.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:18.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:14:18.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:14:18.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:14:18.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:14:18.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:14:18.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:14:18.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:14:18.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:14:18.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:14:18.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:14:18.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:14:18.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:14:18.166 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.166 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.166 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.166 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.166 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:18.166 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598492569000001, 2024-09-13T09:14:18.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:14:18.166 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.166 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.166 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.166 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.167 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.167 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:14:18.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.167 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:18.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.167 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:18.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:14:18.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:14:18.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:14:18.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:14:18.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:14:18.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:14:18.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:14:18.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:14:18.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:14:18.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:14:18.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:14:18.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:14:18.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:14:18.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.170 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.170 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.171 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:18.171 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598476113000004, 2024-09-13T09:14:18.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:14:18.171 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.171 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.171 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:14:18.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.171 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:18.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:18.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:18.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:14:18.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:14:18.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:14:18.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:14:18.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:14:18.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:14:18.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:14:18.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:14:18.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:14:18.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:14:18.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:14:18.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:14:18.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.175 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.175 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.175 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:18.175 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.598230975999996, 2024-09-13T09:14:18.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:14:18.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.175 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.175 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.175 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.175 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.176 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:14:18.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.176 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:18.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:18.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:14:18.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:14:18.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:14:18.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:14:18.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:14:18.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:14:18.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:14:18.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:14:18.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:14:18.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:14:18.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:14:18.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:14:18.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:14:18.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:14:18.179 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.179 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.179 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.179 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.179 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:18.180 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.596345416999995, 2024-09-13T09:14:18.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:14:18.180 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.180 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.180 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.180 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.180 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.180 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:14:18.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.180 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.181 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:18.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.181 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.181 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:18.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.181 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.181 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:14:18.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:14:18.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:14:18.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:14:18.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:14:18.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:14:18.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:14:18.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:14:18.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:14:18.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:14:18.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:14:18.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:14:18.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:14:18.184 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.184 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.184 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.184 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.184 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:18.184 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.596324050999996, 2024-09-13T09:14:18.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:14:18.184 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.184 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.185 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.185 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.185 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.185 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:14:18.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.185 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:18.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:18.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:18.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:14:18.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:14:18.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.187 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:14:18.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:14:18.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.187 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:14:18.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:14:18.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.187 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:14:18.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:14:18.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:14:18.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:14:18.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:14:18.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:14:18.188 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.188 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.188 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.188 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:18.189 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:18.189 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 95.596118683, 2024-09-13T09:14:18.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:14:18.189 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.189 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:18.189 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.189 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:18.189 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.189 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:14:18.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.189 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:18.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:18.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:14:18.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:14:18.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:14:18.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:14:18.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:14:18.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:14:18.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:14:18.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:14:18.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:14:18.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:14:18.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:14:18.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:14:18.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:14:18.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:14:18.193 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.193 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.193 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.193 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5272.0:1775 3.0 3:02a42fca:::rbd_data.11235782ee8d.0000000000000030:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2244608~40448] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:14:18.193 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:18.193 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 67.854576842, 2024-09-13T09:14:18.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.666843732, 2024-09-13T09:14:18.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:18.193 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:18.193 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:18.194 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5272", 2024-09-13T09:14:18.194 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4089902097", 2024-09-13T09:14:18.194 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1775 2024-09-13T09:14:18.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.194 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:18.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:18.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:18.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:18.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:18.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:18.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:18.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247780+0000", 2024-09-13T09:14:18.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:18.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:18.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247865+0000", 2024-09-13T09:14:18.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4359999999999999e-05 2024-09-13T09:14:18.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:18.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247876+0000", 2024-09-13T09:14:18.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1456e-05 2024-09-13T09:14:18.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247883+0000", 2024-09-13T09:14:18.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:14:18.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247926+0000", 2024-09-13T09:14:18.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3203e-05 2024-09-13T09:14:18.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247932+0000", 2024-09-13T09:14:18.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:14:18.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909639+0000", 2024-09-13T09:14:18.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6617069099999999 2024-09-13T09:14:18.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:18.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909828+0000", 2024-09-13T09:14:18.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000189075 2024-09-13T09:14:18.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:18.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911676+0000", 2024-09-13T09:14:18.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001847235 2024-09-13T09:14:18.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:18.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911765+0000", 2024-09-13T09:14:18.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9530000000000005e-05 2024-09-13T09:14:18.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:14:18.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911901+0000", 2024-09-13T09:14:18.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013603899999999999 2024-09-13T09:14:18.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:18.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.912889+0000", 2024-09-13T09:14:18.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098807400000000003 2024-09-13T09:14:18.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914438+0000", 2024-09-13T09:14:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015491070000000001 2024-09-13T09:14:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914505+0000", 2024-09-13T09:14:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:14:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:18.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914625+0000", 2024-09-13T09:14:18.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120108 2024-09-13T09:14:18.201 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.201 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.201 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.201 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:18.201 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:18.201 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:18.202 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:14:18.203 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:18.201+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:18.303 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:18.302+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:18.306 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:18.306 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:18.306 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:18.306 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:18.315 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:14:18.404 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:18.403+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:18.417 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:18.417 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:18.417 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:18.426 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:14:18.511 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:18.510+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:18.570 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:18.571 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:18.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:18.571 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:18.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.571 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.571 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:18.571 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.047341904000007, 2024-09-13T09:14:18.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:14:18.571 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.571 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.571 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.571 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.571 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.572 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:14:18.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.572 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:18.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:18.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:18.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:14:18.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:14:18.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:14:18.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:14:18.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:14:18.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:14:18.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:14:18.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:14:18.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:14:18.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:14:18.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:14:18.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:14:18.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:14:18.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:14:18.575 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.575 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.576 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.576 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.576 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:18.576 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.047292847999998, 2024-09-13T09:14:18.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:14:18.576 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.576 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.577 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.577 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.577 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.577 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:14:18.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.577 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:18.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:18.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:18.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:14:18.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:14:18.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:14:18.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:14:18.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:14:18.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:14:18.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:14:18.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:14:18.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:14:18.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:14:18.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:14:18.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:14:18.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:14:18.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:14:18.581 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.581 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.581 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.582 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.582 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:18.582 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.047252772999997, 2024-09-13T09:14:18.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:14:18.582 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.582 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.582 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.582 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.582 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.582 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:14:18.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.583 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:18.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:18.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:18.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:14:18.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:14:18.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:14:18.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:14:18.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:14:18.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:14:18.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:14:18.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:14:18.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:14:18.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:14:18.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:14:18.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:14:18.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:14:18.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:14:18.587 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.587 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.587 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.587 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.587 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:18.587 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.047222958000006, 2024-09-13T09:14:18.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:14:18.587 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.587 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.588 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.588 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.588 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:14:18.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.588 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:18.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:18.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:18.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:14:18.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:14:18.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:14:18.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:14:18.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:14:18.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:14:18.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:14:18.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:14:18.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:14:18.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:14:18.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:14:18.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:14:18.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:14:18.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:14:18.592 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.592 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.592 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.592 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.592 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:18.592 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.047193297000007, 2024-09-13T09:14:18.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:14:18.592 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.593 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.593 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.593 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.593 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.593 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:14:18.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.593 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:18.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:18.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:18.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:14:18.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:14:18.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:14:18.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:14:18.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:14:18.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:14:18.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:14:18.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:14:18.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:14:18.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:14:18.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:14:18.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:14:18.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:14:18.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:14:18.596 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.597 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.597 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.597 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.597 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:18.597 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.047164089999995, 2024-09-13T09:14:18.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:14:18.597 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.597 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.597 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.597 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.597 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.598 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:14:18.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.598 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:18.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:18.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:18.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:14:18.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:14:18.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:14:18.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:14:18.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:14:18.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:14:18.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:14:18.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:14:18.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:14:18.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:14:18.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:14:18.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:14:18.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:14:18.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:14:18.602 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.602 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.602 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.602 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.602 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:18.602 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.047107909000005, 2024-09-13T09:14:18.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:14:18.602 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.602 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.602 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.602 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.602 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.603 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:14:18.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.603 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:18.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:18.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:18.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:14:18.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:14:18.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:14:18.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:14:18.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:14:18.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:14:18.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:14:18.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:14:18.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:14:18.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:14:18.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:14:18.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:14:18.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:14:18.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:14:18.606 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.606 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.606 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.607 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.607 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:18.607 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.047054447999997, 2024-09-13T09:14:18.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:14:18.607 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.607 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.607 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.607 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.607 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.607 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:14:18.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.607 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:18.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:18.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:14:18.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:14:18.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:14:18.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:14:18.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:14:18.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:14:18.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:14:18.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:14:18.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:14:18.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:14:18.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:14:18.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:14:18.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:14:18.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:14:18.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:14:18.611 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.611 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.611 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.611 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.611 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:18.611 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.046997618000006, 2024-09-13T09:14:18.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:14:18.612 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.612 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.612 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:14:18.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.612 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:18.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:18.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:14:18.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:14:18.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:14:18.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:14:18.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:14:18.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:14:18.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:14:18.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:14:18.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:14:18.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:14:18.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:14:18.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:14:18.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:14:18.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:14:18.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:14:18.616 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.616 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.616 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.616 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.616 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:18.616 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.046946590999994, 2024-09-13T09:14:18.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:14:18.616 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.616 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.616 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.616 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.617 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.617 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:14:18.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.617 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:18.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:18.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:14:18.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:14:18.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:14:18.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:14:18.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:14:18.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:14:18.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:14:18.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:14:18.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:14:18.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:14:18.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:14:18.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:14:18.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:14:18.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:14:18.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:14:18.623 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.623 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.624 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.624 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.624 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:18.624 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.046905401000004, 2024-09-13T09:14:18.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:14:18.624 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.624 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.624 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:14:18.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.625 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:18.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:18.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:18.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:14:18.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:14:18.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:14:18.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:14:18.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:14:18.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:14:18.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:14:18.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:14:18.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:14:18.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:14:18.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:14:18.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:14:18.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:14:18.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:14:18.628 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.628 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.628 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.629 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.629 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:18.629 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.046863285000001, 2024-09-13T09:14:18.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:14:18.629 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.629 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.629 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.629 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.629 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.629 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:14:18.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.629 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:18.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:18.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:18.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:18.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:14:18.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:14:18.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:14:18.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:14:18.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:14:18.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:14:18.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:14:18.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:14:18.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:14:18.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:14:18.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:14:18.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:14:18.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:14:18.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:14:18.633 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.633 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.633 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.633 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.633 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:18.633 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.046819405999997, 2024-09-13T09:14:18.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:14:18.634 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.634 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.634 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.634 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.634 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.634 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:14:18.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.634 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:18.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:18.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:18.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:18.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:14:18.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:14:18.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:14:18.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:14:18.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:14:18.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:14:18.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:14:18.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:14:18.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:14:18.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:14:18.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:14:18.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:14:18.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:14:18.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:14:18.638 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.638 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.638 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.638 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.638 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:18.638 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.046773489000003, 2024-09-13T09:14:18.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:14:18.638 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.638 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.638 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.639 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.639 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.639 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:14:18.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.639 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:18.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:18.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:14:18.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:14:18.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:14:18.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:14:18.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:14:18.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:14:18.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:14:18.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:14:18.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:14:18.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:14:18.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:14:18.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:14:18.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:14:18.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:14:18.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:14:18.642 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.642 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.643 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.643 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.643 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:18.643 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.046723717999996, 2024-09-13T09:14:18.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:14:18.643 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.643 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.643 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.644 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:14:18.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.644 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:18.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:18.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:18.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:18.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:14:18.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:14:18.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:14:18.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:14:18.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:14:18.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:14:18.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:14:18.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:14:18.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:14:18.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:14:18.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:14:18.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:14:18.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:14:18.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:14:18.648 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.648 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.648 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.648 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.648 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:18.648 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.046678314000005, 2024-09-13T09:14:18.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:14:18.648 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.648 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.649 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:14:18.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.649 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:18.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:18.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:18.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:18.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:14:18.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:14:18.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:14:18.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:14:18.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:14:18.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:14:18.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:14:18.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:14:18.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:14:18.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:14:18.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:14:18.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:14:18.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:14:18.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:14:18.652 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.652 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.652 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.653 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.653 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:18.653 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.046627676, 2024-09-13T09:14:18.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:14:18.653 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.653 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.653 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:14:18.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.653 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:14:18.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:14:18.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:14:18.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:14:18.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:14:18.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:14:18.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:14:18.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:14:18.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:14:18.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:14:18.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:14:18.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:14:18.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:14:18.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:14:18.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:14:18.657 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.657 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.657 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.657 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.658 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:18.658 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.046571833000002, 2024-09-13T09:14:18.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:14:18.658 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.658 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.658 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.658 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.658 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.658 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:14:18.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.658 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:18.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:18.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:18.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:14:18.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:14:18.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:14:18.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:14:18.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:14:18.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:14:18.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:14:18.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:14:18.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:14:18.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:14:18.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:14:18.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:14:18.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:14:18.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:14:18.662 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.662 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.662 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.662 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.662 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:18.662 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.046514911000003, 2024-09-13T09:14:18.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:14:18.662 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.663 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.663 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.663 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:14:18.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.663 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:18.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:18.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:18.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:18.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:14:18.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:14:18.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:14:18.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:14:18.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:14:18.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:14:18.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:14:18.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:14:18.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:14:18.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:14:18.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:14:18.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:14:18.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:14:18.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:14:18.667 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.667 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.667 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.667 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:18.667 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:18.667 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 96.046461347999994, 2024-09-13T09:14:18.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:14:18.667 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.667 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:18.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:18.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:18.668 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:14:18.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.668 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:18.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:18.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:18.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:18.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:14:18.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:14:18.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:14:18.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:14:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:14:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:14:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:14:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:14:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:14:18.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:14:18.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:14:18.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:14:18.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:18.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:14:18.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:14:18.671 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.671 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.671 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.672 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.672 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.672 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:18.673 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:14:18.673 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:18.672+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:18.774 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:18.773+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:18.831 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:18.831 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:18.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.831 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:30110 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=78b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-13T09:14:18.831 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:14:18.830284+0000", 2024-09-13T09:14:18.832 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.000388458, 2024-09-13T09:14:18.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00040557599999999999, 2024-09-13T09:14:18.832 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:18.832 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:14:18.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:18.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:18.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:18.832 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 30110 2024-09-13T09:14:18.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.832 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:18.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:18.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.830284+0000", 2024-09-13T09:14:18.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:18.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.830284+0000", 2024-09-13T09:14:18.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:18.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:18.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.830283+0000", 2024-09-13T09:14:18.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:14:18.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:18.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.830288+0000", 2024-09-13T09:14:18.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3800000000000002e-06 2024-09-13T09:14:18.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:18.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.830298+0000", 2024-09-13T09:14:18.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0288999999999999e-05 2024-09-13T09:14:18.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:18.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.830304+0000", 2024-09-13T09:14:18.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8259999999999998e-06 2024-09-13T09:14:18.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:18.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.830338+0000", 2024-09-13T09:14:18.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3840999999999997e-05 2024-09-13T09:14:18.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:18.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.830390+0000", 2024-09-13T09:14:18.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2212000000000001e-05 2024-09-13T09:14:18.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:18.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:18.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-13T09:14:18.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.830518+0000", 2024-09-13T09:14:18.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012784600000000001 2024-09-13T09:14:18.836 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.836 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:18.836 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.836 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:18.836 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-13T09:14:18.836 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 1 2024-09-13T09:14:18.836 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:18.841 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:14:18.875 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:18.874+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:18.976 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:18.975+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:18.984 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:18.984 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:18.984 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:18.985 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:18.985 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:18.993 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:14:19.077 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:19.076+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:19.097 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:19.097 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:19.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:19.097 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:19.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.097 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2185 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.098 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:19.098 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569538311999999, 2024-09-13T09:14:19.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063348994000000006, 2024-09-13T09:14:19.098 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.098 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.098 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.098 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.098 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.098 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2185 2024-09-13T09:14:19.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.098 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:19.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:19.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525763+0000", 2024-09-13T09:14:19.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525765+0000", 2024-09-13T09:14:19.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2390000000000001e-06 2024-09-13T09:14:19.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525772+0000", 2024-09-13T09:14:19.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:14:19.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525777+0000", 2024-09-13T09:14:19.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0799999999999996e-06 2024-09-13T09:14:19.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589020+0000", 2024-09-13T09:14:19.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063242501000000007 2024-09-13T09:14:19.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589063+0000", 2024-09-13T09:14:19.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2673999999999998e-05 2024-09-13T09:14:19.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589112+0000", 2024-09-13T09:14:19.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9286000000000002e-05 2024-09-13T09:14:19.102 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.102 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.102 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.102 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2187 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 892928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.102 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:19.102 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569518764999998, 2024-09-13T09:14:19.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063500102000000003, 2024-09-13T09:14:19.103 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.103 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.103 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2187 2024-09-13T09:14:19.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.103 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:19.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:19.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525782+0000", 2024-09-13T09:14:19.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525784+0000", 2024-09-13T09:14:19.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1629999999999999e-06 2024-09-13T09:14:19.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525792+0000", 2024-09-13T09:14:19.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:14:19.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525796+0000", 2024-09-13T09:14:19.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7389999999999999e-06 2024-09-13T09:14:19.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589132+0000", 2024-09-13T09:14:19.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063336119999999996 2024-09-13T09:14:19.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589187+0000", 2024-09-13T09:14:19.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4126000000000003e-05 2024-09-13T09:14:19.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589283+0000", 2024-09-13T09:14:19.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6032999999999993e-05 2024-09-13T09:14:19.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.107 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.107 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2188 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.107 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:19.107 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569500718999997, 2024-09-13T09:14:19.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063724617999999997, 2024-09-13T09:14:19.107 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.107 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.107 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.107 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.107 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.108 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2188 2024-09-13T09:14:19.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.108 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:19.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:19.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:19.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525802+0000", 2024-09-13T09:14:19.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0439999999999998e-06 2024-09-13T09:14:19.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525809+0000", 2024-09-13T09:14:19.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.238e-06 2024-09-13T09:14:19.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525814+0000", 2024-09-13T09:14:19.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-13T09:14:19.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589327+0000", 2024-09-13T09:14:19.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063513257000000004 2024-09-13T09:14:19.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589418+0000", 2024-09-13T09:14:19.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0796000000000003e-05 2024-09-13T09:14:19.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589525+0000", 2024-09-13T09:14:19.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010693500000000001 2024-09-13T09:14:19.111 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.111 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.111 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.111 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2190 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.111 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:19.112 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569482633, 2024-09-13T09:14:19.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063863981, 2024-09-13T09:14:19.112 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.112 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.112 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.112 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.112 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.112 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2190 2024-09-13T09:14:19.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.112 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:19.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:19.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525818+0000", 2024-09-13T09:14:19.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525820+0000", 2024-09-13T09:14:19.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:14:19.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525827+0000", 2024-09-13T09:14:19.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0589999999999999e-06 2024-09-13T09:14:19.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525831+0000", 2024-09-13T09:14:19.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7870000000000001e-06 2024-09-13T09:14:19.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589562+0000", 2024-09-13T09:14:19.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063731038000000004 2024-09-13T09:14:19.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589612+0000", 2024-09-13T09:14:19.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0087999999999999e-05 2024-09-13T09:14:19.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589683+0000", 2024-09-13T09:14:19.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0121000000000002e-05 2024-09-13T09:14:19.115 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.116 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.116 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.116 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2192 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.116 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:19.116 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569465673000003, 2024-09-13T09:14:19.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064050299000000005, 2024-09-13T09:14:19.116 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.116 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.116 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.116 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2192 2024-09-13T09:14:19.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.117 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:19.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:19.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525835+0000", 2024-09-13T09:14:19.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525837+0000", 2024-09-13T09:14:19.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:14:19.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525845+0000", 2024-09-13T09:14:19.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3660000000000003e-06 2024-09-13T09:14:19.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525850+0000", 2024-09-13T09:14:19.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0030000000000002e-06 2024-09-13T09:14:19.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589720+0000", 2024-09-13T09:14:19.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063870176000000001 2024-09-13T09:14:19.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589801+0000", 2024-09-13T09:14:19.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0889000000000004e-05 2024-09-13T09:14:19.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589886+0000", 2024-09-13T09:14:19.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5183000000000001e-05 2024-09-13T09:14:19.120 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.120 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.120 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.120 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.120 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:19.120 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569447570000001, 2024-09-13T09:14:19.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:14:19.121 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.121 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.121 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.121 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:14:19.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.121 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:19.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:19.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:14:19.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:14:19.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:14:19.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:14:19.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:14:19.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:14:19.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:14:19.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:14:19.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:14:19.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:14:19.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:14:19.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:14:19.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:14:19.124 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.125 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.125 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.125 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:19.125 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569429872000001, 2024-09-13T09:14:19.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:14:19.125 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.126 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.126 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:14:19.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.126 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:19.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:19.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:19.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:14:19.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:14:19.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:14:19.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:14:19.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:14:19.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:14:19.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:14:19.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:14:19.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:14:19.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:14:19.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:14:19.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:14:19.130 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.130 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.130 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.130 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.130 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:19.130 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569409256999997, 2024-09-13T09:14:19.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:14:19.130 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.130 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.130 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.130 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.130 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.131 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:14:19.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.131 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:19.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:19.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:19.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:14:19.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:14:19.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:14:19.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:14:19.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:14:19.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:14:19.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:14:19.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:14:19.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:14:19.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:14:19.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:14:19.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:14:19.134 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.135 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.135 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.135 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.135 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:19.135 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569390300999999, 2024-09-13T09:14:19.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:14:19.135 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.135 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.135 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.135 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.135 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.135 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:14:19.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.136 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:19.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:19.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:19.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:14:19.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:14:19.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:14:19.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:14:19.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:14:19.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:14:19.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:14:19.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:14:19.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:14:19.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:14:19.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:14:19.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:14:19.139 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.139 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.139 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.139 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.139 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:19.139 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569373042999999, 2024-09-13T09:14:19.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:14:19.139 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.140 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.140 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.140 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.140 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.140 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:14:19.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.140 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:19.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:19.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:19.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:14:19.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:14:19.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:14:19.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:14:19.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:14:19.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:14:19.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:14:19.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:14:19.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:14:19.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:14:19.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:14:19.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:14:19.143 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.143 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.143 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.144 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.144 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:19.144 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569353028999998, 2024-09-13T09:14:19.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:14:19.144 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.144 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.144 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.144 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:14:19.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.144 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:19.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:19.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:14:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:14:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:14:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:14:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:14:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:14:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:14:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:14:19.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:14:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:14:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:14:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:14:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.148 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569327446999999, 2024-09-13T09:14:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:14:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:14:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.149 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:14:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:14:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:14:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:14:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:14:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:14:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:14:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:14:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:14:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:14:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:14:19.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:14:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:14:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569308163000002, 2024-09-13T09:14:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:14:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.153 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:14:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:19.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:19.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:14:19.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:14:19.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:14:19.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:14:19.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:14:19.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:14:19.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:14:19.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:14:19.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:14:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:14:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:14:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:14:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.157 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569290907000003, 2024-09-13T09:14:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:14:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:14:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:19.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:14:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:14:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:14:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:14:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:14:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:14:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:14:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:14:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:14:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:14:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:14:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:14:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.161 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569271426999997, 2024-09-13T09:14:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:14:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:14:19.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:14:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:14:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:14:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:14:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:14:19.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:14:19.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:14:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:14:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:14:19.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:14:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:14:19.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:14:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:14:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569254497999999, 2024-09-13T09:14:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:14:19.170 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:14:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:14:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:14:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:14:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:14:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:14:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:14:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:14:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:14:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:14:19.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:14:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:14:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:14:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:14:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569235349000003, 2024-09-13T09:14:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:14:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:14:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:14:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:14:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:14:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:14:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:14:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:14:19.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:14:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:14:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:14:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:14:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:14:19.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:14:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.569217209999998, 2024-09-13T09:14:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:14:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.179 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:14:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:14:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:14:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:14:19.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:14:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:14:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:14:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:14:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:14:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:14:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:14:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:14:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:14:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.568806715000001, 2024-09-13T09:14:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:14:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.184 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:14:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:14:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:14:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:14:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:14:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:14:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:14:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:14:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:14:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:14:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:14:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:14:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:14:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.568783814, 2024-09-13T09:14:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:14:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:14:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:14:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:14:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:14:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:14:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:14:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:14:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:14:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:14:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:14:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:14:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:14:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:14:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.193 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.193 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:19.195 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:14:19.195 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:19.194+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:19.295 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:19.295+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:19.337 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:19.337 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:19.337 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:19.337 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:19.347 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:14:19.397 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:19.396+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:19.444 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:14:19.489 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:19.489 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:19.489 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:19.489 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:19.489 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:19.498 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:14:19.498 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:19.496+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:19.580 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:14:19.581 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:14:19.597 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:14:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:14:19.598 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:19.597+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:19.603 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:19.603 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:19.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:19.603 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:19.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.603 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9036 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2572288~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:19.604 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:19.604 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.855911390000003, 2024-09-13T09:14:19.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070551717, 2024-09-13T09:14:19.604 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.604 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:19.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:19.604 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9036 2024-09-13T09:14:19.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.604 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:19.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:19.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745614+0000", 2024-09-13T09:14:19.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745615+0000", 2024-09-13T09:14:19.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.283e-06 2024-09-13T09:14:19.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745618+0000", 2024-09-13T09:14:19.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0280000000000001e-06 2024-09-13T09:14:19.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745620+0000", 2024-09-13T09:14:19.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:14:19.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815909+0000", 2024-09-13T09:14:19.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070288929999999999 2024-09-13T09:14:19.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.815958+0000", 2024-09-13T09:14:19.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8826000000000003e-05 2024-09-13T09:14:19.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816166+0000", 2024-09-13T09:14:19.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00020754599999999999 2024-09-13T09:14:19.608 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.608 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.608 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.608 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9037 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 2801664~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:19.608 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:19.608 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.855902712000002, 2024-09-13T09:14:19.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070879633999999997, 2024-09-13T09:14:19.608 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.609 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.609 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.609 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:19.609 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:19.609 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9037 2024-09-13T09:14:19.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.609 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:19.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:19.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745622+0000", 2024-09-13T09:14:19.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:19.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745624+0000", 2024-09-13T09:14:19.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.353e-06 2024-09-13T09:14:19.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745627+0000", 2024-09-13T09:14:19.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3189999999999999e-06 2024-09-13T09:14:19.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745629+0000", 2024-09-13T09:14:19.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-06 2024-09-13T09:14:19.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816193+0000", 2024-09-13T09:14:19.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070564136999999999 2024-09-13T09:14:19.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816247+0000", 2024-09-13T09:14:19.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3808999999999999e-05 2024-09-13T09:14:19.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816502+0000", 2024-09-13T09:14:19.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025490799999999998 2024-09-13T09:14:19.613 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.613 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.613 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.613 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9038 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:19.613 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.855893726000005, 2024-09-13T09:14:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071192319000000004, 2024-09-13T09:14:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9038 2024-09-13T09:14:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745631+0000", 2024-09-13T09:14:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745632+0000", 2024-09-13T09:14:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:14:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745635+0000", 2024-09-13T09:14:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.774e-06 2024-09-13T09:14:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745637+0000", 2024-09-13T09:14:19.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4140000000000001e-06 2024-09-13T09:14:19.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816527+0000", 2024-09-13T09:14:19.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070889592000000001 2024-09-13T09:14:19.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816574+0000", 2024-09-13T09:14:19.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6984000000000003e-05 2024-09-13T09:14:19.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816824+0000", 2024-09-13T09:14:19.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024970400000000001 2024-09-13T09:14:19.618 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.618 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.618 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.618 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9039 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 3260416~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:19.618 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:19.618 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.855885753999999, 2024-09-13T09:14:19.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071304928000000004, 2024-09-13T09:14:19.618 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.619 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:19.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:19.619 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9039 2024-09-13T09:14:19.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.619 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:19.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:19.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745639+0000", 2024-09-13T09:14:19.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:19.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745640+0000", 2024-09-13T09:14:19.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1069999999999999e-06 2024-09-13T09:14:19.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745643+0000", 2024-09-13T09:14:19.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:14:19.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.745645+0000", 2024-09-13T09:14:19.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:14:19.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816843+0000", 2024-09-13T09:14:19.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071197619000000004 2024-09-13T09:14:19.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816889+0000", 2024-09-13T09:14:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6306e-05 2024-09-13T09:14:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.816944+0000", 2024-09-13T09:14:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5083000000000001e-05 2024-09-13T09:14:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.623 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9043 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4177920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:19.623 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:19.623 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.854010919999993, 2024-09-13T09:14:19.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070449611999999995, 2024-09-13T09:14:19.623 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.623 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.623 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.623 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:19.623 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:19.623 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9043 2024-09-13T09:14:19.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.623 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:19.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:19.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747514+0000", 2024-09-13T09:14:19.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747515+0000", 2024-09-13T09:14:19.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.282e-06 2024-09-13T09:14:19.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747520+0000", 2024-09-13T09:14:19.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:14:19.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747523+0000", 2024-09-13T09:14:19.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:14:19.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817689+0000", 2024-09-13T09:14:19.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070165863999999994 2024-09-13T09:14:19.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817734+0000", 2024-09-13T09:14:19.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5082e-05 2024-09-13T09:14:19.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817964+0000", 2024-09-13T09:14:19.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023012099999999999 2024-09-13T09:14:19.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.627 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.627 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9044 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4407296~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:19.627 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:19.627 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.854000259000003, 2024-09-13T09:14:19.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070745892000000005, 2024-09-13T09:14:19.627 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.627 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:19.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:19.628 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9044 2024-09-13T09:14:19.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.628 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:19.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:19.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747525+0000", 2024-09-13T09:14:19.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:19.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747526+0000", 2024-09-13T09:14:19.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1039999999999999e-06 2024-09-13T09:14:19.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747529+0000", 2024-09-13T09:14:19.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-13T09:14:19.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747531+0000", 2024-09-13T09:14:19.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-13T09:14:19.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.817985+0000", 2024-09-13T09:14:19.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070454381999999996 2024-09-13T09:14:19.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818033+0000", 2024-09-13T09:14:19.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7169000000000002e-05 2024-09-13T09:14:19.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818271+0000", 2024-09-13T09:14:19.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000238275 2024-09-13T09:14:19.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.631 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.632 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9045 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4636672~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:19.632 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:19.632 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.853978283000004, 2024-09-13T09:14:19.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070981385999999994, 2024-09-13T09:14:19.632 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.632 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:19.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:19.632 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9045 2024-09-13T09:14:19.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.632 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:19.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:19.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747547+0000", 2024-09-13T09:14:19.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747548+0000", 2024-09-13T09:14:19.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.39e-06 2024-09-13T09:14:19.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747551+0000", 2024-09-13T09:14:19.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1590000000000002e-06 2024-09-13T09:14:19.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747558+0000", 2024-09-13T09:14:19.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5409999999999997e-06 2024-09-13T09:14:19.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818291+0000", 2024-09-13T09:14:19.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733115999999999 2024-09-13T09:14:19.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818336+0000", 2024-09-13T09:14:19.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-05 2024-09-13T09:14:19.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818528+0000", 2024-09-13T09:14:19.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019231000000000001 2024-09-13T09:14:19.636 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.636 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.636 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.636 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9046 3.c 3:36d19fad:::rbd_data.11238aebf726.0000000000000006:15 [read 4866048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:19.636 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:19.636 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.853964781000002, 2024-09-13T09:14:19.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071090600000000004, 2024-09-13T09:14:19.636 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.636 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:19.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:19.637 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9046 2024-09-13T09:14:19.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.637 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:19.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:19.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747560+0000", 2024-09-13T09:14:19.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747561+0000", 2024-09-13T09:14:19.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.339e-06 2024-09-13T09:14:19.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747564+0000", 2024-09-13T09:14:19.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:14:19.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.747567+0000", 2024-09-13T09:14:19.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:14:19.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818551+0000", 2024-09-13T09:14:19.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070984571999999996 2024-09-13T09:14:19.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818598+0000", 2024-09-13T09:14:19.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6331000000000001e-05 2024-09-13T09:14:19.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.818651+0000", 2024-09-13T09:14:19.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3349e-05 2024-09-13T09:14:19.640 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.640 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.640 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.640 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9057 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 212992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:19.641 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:19.641 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.851955683, 2024-09-13T09:14:19.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070583848000000005, 2024-09-13T09:14:19.641 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.641 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.641 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.641 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:19.641 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:19.641 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9057 2024-09-13T09:14:19.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.641 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:19.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:19.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749569+0000", 2024-09-13T09:14:19.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749570+0000", 2024-09-13T09:14:19.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1909999999999999e-06 2024-09-13T09:14:19.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749573+0000", 2024-09-13T09:14:19.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:14:19.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749576+0000", 2024-09-13T09:14:19.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:14:19.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819862+0000", 2024-09-13T09:14:19.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070285993000000005 2024-09-13T09:14:19.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.819907+0000", 2024-09-13T09:14:19.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5343000000000003e-05 2024-09-13T09:14:19.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820153+0000", 2024-09-13T09:14:19.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00024642000000000002 2024-09-13T09:14:19.645 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.645 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.645 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.645 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9058 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 442368~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:19.645 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:19.645 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.851931360999998, 2024-09-13T09:14:19.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070835993999999999, 2024-09-13T09:14:19.645 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.646 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.646 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.646 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:19.646 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:19.646 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9058 2024-09-13T09:14:19.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.646 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:19.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:19.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749594+0000", 2024-09-13T09:14:19.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749595+0000", 2024-09-13T09:14:19.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:14:19.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749600+0000", 2024-09-13T09:14:19.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:14:19.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749602+0000", 2024-09-13T09:14:19.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4880000000000001e-06 2024-09-13T09:14:19.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820164+0000", 2024-09-13T09:14:19.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070561852999999994 2024-09-13T09:14:19.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820208+0000", 2024-09-13T09:14:19.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3717000000000002e-05 2024-09-13T09:14:19.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820430+0000", 2024-09-13T09:14:19.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022177699999999999 2024-09-13T09:14:19.650 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.650 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.650 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9059 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 671744~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.851920387000007, 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.071104853999999995, 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9059 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:14:19.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749605+0000", 2024-09-13T09:14:19.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749604+0000", 2024-09-13T09:14:19.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749606+0000", 2024-09-13T09:14:19.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.279e-06 2024-09-13T09:14:19.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749609+0000", 2024-09-13T09:14:19.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-13T09:14:19.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.749612+0000", 2024-09-13T09:14:19.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.202e-06 2024-09-13T09:14:19.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820440+0000", 2024-09-13T09:14:19.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070828300999999996 2024-09-13T09:14:19.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820495+0000", 2024-09-13T09:14:19.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4635000000000001e-05 2024-09-13T09:14:19.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.820710+0000", 2024-09-13T09:14:19.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021482399999999999 2024-09-13T09:14:19.655 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.655 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.655 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.655 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9064 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:19.655 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:19.655 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.850002762000003, 2024-09-13T09:14:19.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070689360000000007, 2024-09-13T09:14:19.655 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.655 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.655 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:19.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:19.656 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9064 2024-09-13T09:14:19.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.656 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:19.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:19.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751522+0000", 2024-09-13T09:14:19.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751523+0000", 2024-09-13T09:14:19.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.319e-06 2024-09-13T09:14:19.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751527+0000", 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751530+0000", 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.322e-06 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821911+0000", 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070381552 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.821957+0000", 2024-09-13T09:14:19.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5642000000000001e-05 2024-09-13T09:14:19.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822212+0000", 2024-09-13T09:14:19.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025497699999999998 2024-09-13T09:14:19.659 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.659 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.659 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.659 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4934.0:9065 3.c 3:37dddf28:::rbd_data.11238aebf726.0000000000000007:15 [read 2048000~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:14:19.659 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:19.660 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.849993365000003, 2024-09-13T09:14:19.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070972404000000003, 2024-09-13T09:14:19.660 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.660 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.660 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.660 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4934", 2024-09-13T09:14:19.660 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3631001601", 2024-09-13T09:14:19.660 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 9065 2024-09-13T09:14:19.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.660 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:19.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:19.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751532+0000", 2024-09-13T09:14:19.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:19.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751536+0000", 2024-09-13T09:14:19.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1420000000000003e-06 2024-09-13T09:14:19.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751540+0000", 2024-09-13T09:14:19.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.912e-06 2024-09-13T09:14:19.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.751542+0000", 2024-09-13T09:14:19.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-13T09:14:19.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822226+0000", 2024-09-13T09:14:19.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070684278000000003 2024-09-13T09:14:19.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822271+0000", 2024-09-13T09:14:19.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4382000000000003e-05 2024-09-13T09:14:19.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:57.822504+0000", 2024-09-13T09:14:19.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023351099999999999 2024-09-13T09:14:19.663 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.663 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.664 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.664 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3181 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T09:14:19.664 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:14:19.664 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.579629189000002, 2024-09-13T09:14:19.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.480957603, 2024-09-13T09:14:19.664 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.664 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:19.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:14:19.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:14:19.664 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3181 2024-09-13T09:14:19.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.665 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:14:19.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021896+0000", 2024-09-13T09:14:19.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021894+0000", 2024-09-13T09:14:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999986 2024-09-13T09:14:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021903+0000", 2024-09-13T09:14:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4109999999999996e-06 2024-09-13T09:14:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021919+0000", 2024-09-13T09:14:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6225000000000001e-05 2024-09-13T09:14:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021926+0000", 2024-09-13T09:14:19.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4120000000000004e-06 2024-09-13T09:14:19.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.021975+0000", 2024-09-13T09:14:19.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8526000000000003e-05 2024-09-13T09:14:19.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022049+0000", 2024-09-13T09:14:19.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4018999999999997e-05 2024-09-13T09:14:19.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:14:19.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022230+0000", 2024-09-13T09:14:19.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018085599999999999 2024-09-13T09:14:19.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:19.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.022796+0000", 2024-09-13T09:14:19.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00056638999999999995 2024-09-13T09:14:19.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499540+0000", 2024-09-13T09:14:19.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.47674342800000002 2024-09-13T09:14:19.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:19.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499547+0000", 2024-09-13T09:14:19.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:14:19.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502794+0000", 2024-09-13T09:14:19.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.003246818 2024-09-13T09:14:19.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502854+0000", 2024-09-13T09:14:19.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9925e-05 2024-09-13T09:14:19.670 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.670 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.670 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.670 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5473.0:3182 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 139818970156544 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:14:19.670 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:14:19.670 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.106394633999997, 2024-09-13T09:14:19.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0089058390000001, 2024-09-13T09:14:19.670 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.670 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5473", 2024-09-13T09:14:19.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3388292267", 2024-09-13T09:14:19.671 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3182 2024-09-13T09:14:19.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.671 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:14:19.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495131+0000", 2024-09-13T09:14:19.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495128+0000", 2024-09-13T09:14:19.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999976 2024-09-13T09:14:19.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495139+0000", 2024-09-13T09:14:19.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0599e-05 2024-09-13T09:14:19.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495157+0000", 2024-09-13T09:14:19.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7830999999999999e-05 2024-09-13T09:14:19.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495167+0000", 2024-09-13T09:14:19.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0689999999999999e-05 2024-09-13T09:14:19.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495229+0000", 2024-09-13T09:14:19.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1898999999999998e-05 2024-09-13T09:14:19.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:14:19.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495243+0000", 2024-09-13T09:14:19.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3349e-05 2024-09-13T09:14:19.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499575+0000", 2024-09-13T09:14:19.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0043324410000000002 2024-09-13T09:14:19.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:19.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499582+0000", 2024-09-13T09:14:19.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0849999999999999e-06 2024-09-13T09:14:19.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.502872+0000", 2024-09-13T09:14:19.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0032895150000001 2024-09-13T09:14:19.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503016+0000", 2024-09-13T09:14:19.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014394200000000001 2024-09-13T09:14:19.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504036+0000", 2024-09-13T09:14:19.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010208680000000001 2024-09-13T09:14:19.676 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.676 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.676 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.676 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:19151 3.5 3:a40ed0a5:::rbd_header.112339c5624d:head [watch reconnect cookie 140383089882784 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:14:19.676 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:14:19.676 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 49.106383174999998, 2024-09-13T09:14:19.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.009071287, 2024-09-13T09:14:19.676 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.677 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:19.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:19.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:19.677 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19151 2024-09-13T09:14:19.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.677 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:14:19.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495142+0000", 2024-09-13T09:14:19.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495140+0000", 2024-09-13T09:14:19.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:14:19.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495149+0000", 2024-09-13T09:14:19.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0759999999999992e-06 2024-09-13T09:14:19.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495165+0000", 2024-09-13T09:14:19.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5322e-05 2024-09-13T09:14:19.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495173+0000", 2024-09-13T09:14:19.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0979999999999994e-06 2024-09-13T09:14:19.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495318+0000", 2024-09-13T09:14:19.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014553 2024-09-13T09:14:19.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:14:19.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.495325+0000", 2024-09-13T09:14:19.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:14:19.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499598+0000", 2024-09-13T09:14:19.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.004273064 2024-09-13T09:14:19.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:19.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:30.499602+0000", 2024-09-13T09:14:19.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8539999999999999e-06 2024-09-13T09:14:19.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503507+0000", 2024-09-13T09:14:19.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0039045520000001 2024-09-13T09:14:19.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.503583+0000", 2024-09-13T09:14:19.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6283000000000001e-05 2024-09-13T09:14:19.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:31.504213+0000", 2024-09-13T09:14:19.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00063044700000000002 2024-09-13T09:14:19.682 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.682 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.682 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.682 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3908 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:19.682 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:14:19.682 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.508846284000001, 2024-09-13T09:14:19.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070530317999999995, 2024-09-13T09:14:19.682 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.682 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:19.682 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:19.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:19.683 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3908 2024-09-13T09:14:19.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.683 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:14:19.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092679+0000", 2024-09-13T09:14:19.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092678+0000", 2024-09-13T09:14:19.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092681+0000", 2024-09-13T09:14:19.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.903e-06 2024-09-13T09:14:19.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092687+0000", 2024-09-13T09:14:19.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0859999999999998e-06 2024-09-13T09:14:19.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092692+0000", 2024-09-13T09:14:19.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7589999999999997e-06 2024-09-13T09:14:19.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.159711+0000", 2024-09-13T09:14:19.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067018922999999994 2024-09-13T09:14:19.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.159796+0000", 2024-09-13T09:14:19.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5247000000000006e-05 2024-09-13T09:14:19.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:19.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160111+0000", 2024-09-13T09:14:19.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00031449499999999998 2024-09-13T09:14:19.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:19.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162219+0000", 2024-09-13T09:14:19.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0021085570000000001 2024-09-13T09:14:19.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:19.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163127+0000", 2024-09-13T09:14:19.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00090723400000000004 2024-09-13T09:14:19.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:19.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163172+0000", 2024-09-13T09:14:19.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5383000000000002e-05 2024-09-13T09:14:19.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163209+0000", 2024-09-13T09:14:19.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7132000000000003e-05 2024-09-13T09:14:19.687 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.688 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.688 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.688 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3909 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:19.688 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:19.688 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.508828792999999, 2024-09-13T09:14:19.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070616834000000003, 2024-09-13T09:14:19.688 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.688 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:19.688 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.688 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:19.688 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:19.688 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3909 2024-09-13T09:14:19.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.689 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:19.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:19.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092696+0000", 2024-09-13T09:14:19.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092699+0000", 2024-09-13T09:14:19.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:14:19.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092704+0000", 2024-09-13T09:14:19.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5940000000000001e-06 2024-09-13T09:14:19.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092709+0000", 2024-09-13T09:14:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5109999999999997e-06 2024-09-13T09:14:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160604+0000", 2024-09-13T09:14:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067895633999999996 2024-09-13T09:14:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160658+0000", 2024-09-13T09:14:19.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3292999999999999e-05 2024-09-13T09:14:19.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:19.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.160935+0000", 2024-09-13T09:14:19.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000276879 2024-09-13T09:14:19.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:19.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163223+0000", 2024-09-13T09:14:19.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022886320000000001 2024-09-13T09:14:19.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:19.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163258+0000", 2024-09-13T09:14:19.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5101000000000002e-05 2024-09-13T09:14:19.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:19.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163292+0000", 2024-09-13T09:14:19.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3237e-05 2024-09-13T09:14:19.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163313+0000", 2024-09-13T09:14:19.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1642e-05 2024-09-13T09:14:19.693 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.693 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.693 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.694 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3910 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:19.694 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:19.694 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.508811729000001, 2024-09-13T09:14:19.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.070733837999999993, 2024-09-13T09:14:19.694 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.694 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:19.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:19.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:19.694 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3910 2024-09-13T09:14:19.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.695 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:19.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:19.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092713+0000", 2024-09-13T09:14:19.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092716+0000", 2024-09-13T09:14:19.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.757e-06 2024-09-13T09:14:19.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092722+0000", 2024-09-13T09:14:19.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7869999999999998e-06 2024-09-13T09:14:19.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092726+0000", 2024-09-13T09:14:19.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.622e-06 2024-09-13T09:14:19.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161487+0000", 2024-09-13T09:14:19.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.068761166999999998 2024-09-13T09:14:19.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161569+0000", 2024-09-13T09:14:19.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1185000000000003e-05 2024-09-13T09:14:19.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:19.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.161868+0000", 2024-09-13T09:14:19.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00029979799999999999 2024-09-13T09:14:19.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:19.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163366+0000", 2024-09-13T09:14:19.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001497888 2024-09-13T09:14:19.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:19.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163373+0000", 2024-09-13T09:14:19.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1640000000000004e-06 2024-09-13T09:14:19.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:19.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163412+0000", 2024-09-13T09:14:19.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8423999999999997e-05 2024-09-13T09:14:19.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.163447+0000", 2024-09-13T09:14:19.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5441000000000002e-05 2024-09-13T09:14:19.699 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.699 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.699 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.699 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5710.0:3911 3.16 3:6ae44565:::rbd_object_map.112360a70c4c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T09:14:19.700 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:19.700 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.508791812999998, 2024-09-13T09:14:19.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.072284372999999999, 2024-09-13T09:14:19.700 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:19.700 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:19.700 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:19.700 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5710", 2024-09-13T09:14:19.700 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/433584378", 2024-09-13T09:14:19.700 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 3911 2024-09-13T09:14:19.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.700 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:19.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:19.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:19.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:19.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:19.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:19.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:19.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092733+0000", 2024-09-13T09:14:19.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:19.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:19.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092736+0000", 2024-09-13T09:14:19.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9510000000000002e-06 2024-09-13T09:14:19.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:19.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092743+0000", 2024-09-13T09:14:19.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9e-06 2024-09-13T09:14:19.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:19.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.092748+0000", 2024-09-13T09:14:19.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7999999999999998e-06 2024-09-13T09:14:19.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:19.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162257+0000", 2024-09-13T09:14:19.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.069508890000000004 2024-09-13T09:14:19.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:19.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162342+0000", 2024-09-13T09:14:19.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5886999999999999e-05 2024-09-13T09:14:19.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:19.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.162682+0000", 2024-09-13T09:14:19.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00033971700000000001 2024-09-13T09:14:19.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:19.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164897+0000", 2024-09-13T09:14:19.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0022152370000000001 2024-09-13T09:14:19.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:19.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164945+0000", 2024-09-13T09:14:19.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7243999999999997e-05 2024-09-13T09:14:19.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:19.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.164985+0000", 2024-09-13T09:14:19.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0599000000000002e-05 2024-09-13T09:14:19.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:19.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:19.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:19.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:48.165018+0000", 2024-09-13T09:14:19.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2549000000000003e-05 2024-09-13T09:14:19.705 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.705 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.705 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.705 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:19.705 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:19.705 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:19.707 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:14:19.707 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:19.706+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:19.772 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:14:19.817 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:19.816+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:19.830 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:14:19.831 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:14:19.855 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:19.856 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:19.859 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:14:19.859 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:14:19.865 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:14:19.876 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:14:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:14:19.892 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:14:19.892 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:14:19.908 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:19.908+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:19.994 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:19.994 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:20.003 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:14:20.009 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:20.008+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:20.109 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:20.110 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:20.110 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:20.109+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:20.118 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:14:20.129 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 2] 2024-09-13T09:14:20.129 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:14:20.129 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T09:14:20.129 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.034909 2024-09-13T09:14:20.129 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.03490949068635818 2024-09-13T09:14:20.211 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:20.210+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:20.225 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:20.225 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:20.234 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:14:20.318 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:20.317+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:20.382 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:20.382 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:20.390 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:14:20.413 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:20.412+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:20.514 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:20.513+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:20.537 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:20.537 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:20.546 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:14:20.630 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:20.629+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:20.693 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:20.693 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:20.702 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:14:20.743 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:20.742+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.1 primary-affinity to 0.0349095 (822872) 2024-09-13T09:14:20.786 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-09-13T09:14:20.817 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:20.817+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:20.856 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:20.856 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:20.864 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:14:20.918 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:20.917+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:20.972 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:20.972 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:20.981 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:14:21.025 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:21.025+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:21.090 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:21.090 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:21.099 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:14:21.120 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:21.120+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:21.221 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:21.221+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:21.241 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:21.241 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:21.249 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:14:21.322 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:21.321+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:21.351 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:21.352 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:21.361 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:14:21.423 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:21.422+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:21.463 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:21.463 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:21.472 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:14:21.524 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:21.523+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:21.576 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:21.576 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:21.584 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:14:21.629 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:21.628+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:21.687 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:21.687 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:21.697 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:14:21.739 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:21.738+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:21.827 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:21.825+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:21.843 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:21.844 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:21.853 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:14:21.905 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:14:21.949 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:21.948+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:21.956 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:21.956 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:21.965 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:14:22.027 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:22.027+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:22.028 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:14:22.029 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:14:22.045 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:14:22.046 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:14:22.061 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:14:22.061 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:14:22.067 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:22.067 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:22.076 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:14:22.077 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:14:22.077 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:14:22.094 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:14:22.094 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:14:22.113 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:14:22.113 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:14:22.129 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:22.128+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:22.185 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:22.185 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:22.185 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:22.185 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:22.193 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:14:22.229 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:22.229+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:22.300 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:22.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:22.300 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:22.300 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:22.300 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:22.308 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:14:22.356 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:22.355+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:22.421 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:22.421 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:22.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:22.421 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:22.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.421 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29717 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [watch reconnect cookie 140383290562704 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:14:22.422 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:22.422 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 6.6671306139999995, 2024-09-13T09:14:22.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.006472936, 2024-09-13T09:14:22.422 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.422 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:22.422 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:22.422 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:22.422 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:22.422 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29717 2024-09-13T09:14:22.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.422 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:22.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:22.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751547+0000", 2024-09-13T09:14:22.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:22.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.424 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751552+0000", 2024-09-13T09:14:22.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:14:22.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.424 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751564+0000", 2024-09-13T09:14:22.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1793e-05 2024-09-13T09:14:22.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751571+0000", 2024-09-13T09:14:22.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2660000000000004e-06 2024-09-13T09:14:22.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753102+0000", 2024-09-13T09:14:22.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001531085 2024-09-13T09:14:22.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:22.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753115+0000", 2024-09-13T09:14:22.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3039e-05 2024-09-13T09:14:22.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.752858+0000", 2024-09-13T09:14:22.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.999742715 2024-09-13T09:14:22.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:22.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.753010+0000", 2024-09-13T09:14:22.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015227799999999999 2024-09-13T09:14:22.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755125+0000", 2024-09-13T09:14:22.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021150029999999998 2024-09-13T09:14:22.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-09-13T09:14:22.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755235+0000", 2024-09-13T09:14:22.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000109775 2024-09-13T09:14:22.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756059+0000", 2024-09-13T09:14:22.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00082370900000000005 2024-09-13T09:14:22.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756154+0000", 2024-09-13T09:14:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4945999999999995e-05 2024-09-13T09:14:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758021+0000", 2024-09-13T09:14:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001867446 2024-09-13T09:14:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29718 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:14:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 6.6670977489999999, 2024-09-13T09:14:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0067812169999999, 2024-09-13T09:14:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29718 2024-09-13T09:14:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:22.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:22.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751585+0000", 2024-09-13T09:14:22.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2050000000000004e-06 2024-09-13T09:14:22.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751591+0000", 2024-09-13T09:14:22.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.37e-06 2024-09-13T09:14:22.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751597+0000", 2024-09-13T09:14:22.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1190000000000002e-06 2024-09-13T09:14:22.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753150+0000", 2024-09-13T09:14:22.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001552351 2024-09-13T09:14:22.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:22.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753164+0000", 2024-09-13T09:14:22.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4076e-05 2024-09-13T09:14:22.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.753307+0000", 2024-09-13T09:14:22.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0001434229999999 2024-09-13T09:14:22.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:22.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.753365+0000", 2024-09-13T09:14:22.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8014999999999999e-05 2024-09-13T09:14:22.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755501+0000", 2024-09-13T09:14:22.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021357260000000001 2024-09-13T09:14:22.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-09-13T09:14:22.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755558+0000", 2024-09-13T09:14:22.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7516000000000003e-05 2024-09-13T09:14:22.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756684+0000", 2024-09-13T09:14:22.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011259550000000001 2024-09-13T09:14:22.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756766+0000", 2024-09-13T09:14:22.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1755999999999994e-05 2024-09-13T09:14:22.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:22.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.757147+0000", 2024-09-13T09:14:22.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000381107 2024-09-13T09:14:22.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.438 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:22.438 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.757890+0000", 2024-09-13T09:14:22.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00074321499999999998 2024-09-13T09:14:22.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.438 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:22.438 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758178+0000", 2024-09-13T09:14:22.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00028755000000000002 2024-09-13T09:14:22.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:22.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758230+0000", 2024-09-13T09:14:22.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2317000000000001e-05 2024-09-13T09:14:22.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758362+0000", 2024-09-13T09:14:22.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013204100000000001 2024-09-13T09:14:22.439 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.440 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.440 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.440 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGPush(3.d 123/122 [PushOp(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29, version: 99'289, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29@99'289, size: 12582912, copy_subset: [0~12582912], clone_subset: {}, snapset: 29=[]:{29=[29]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:14:22.440 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.774123+0000", 2024-09-13T09:14:22.440 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.6445560839999995, 2024-09-13T09:14:22.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051988820000000002, 2024-09-13T09:14:22.440 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.440 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:22.441 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.774123+0000", 2024-09-13T09:14:22.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.774123+0000", 2024-09-13T09:14:22.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.442 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.442 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.774122+0000", 2024-09-13T09:14:22.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:22.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.442 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.442 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777696+0000", 2024-09-13T09:14:22.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003573676 2024-09-13T09:14:22.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.442 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.443 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777708+0000", 2024-09-13T09:14:22.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2349e-05 2024-09-13T09:14:22.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.443 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777718+0000", 2024-09-13T09:14:22.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0013999999999999e-05 2024-09-13T09:14:22.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.443 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777761+0000", 2024-09-13T09:14:22.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3167999999999997e-05 2024-09-13T09:14:22.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.444 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777771+0000", 2024-09-13T09:14:22.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3600000000000002e-06 2024-09-13T09:14:22.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.444 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.779322+0000", 2024-09-13T09:14:22.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015509740000000001 2024-09-13T09:14:22.444 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.444 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.445 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.445 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29749 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [watch reconnect cookie 140383086477584 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-13T09:14:22.445 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:22.445 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.6192588910000003, 2024-09-13T09:14:22.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037671060000000001, 2024-09-13T09:14:22.445 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.445 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:22.445 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:22.445 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:22.446 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:22.446 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29749 2024-09-13T09:14:22.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.446 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:22.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:22.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:22.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:22.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799422+0000", 2024-09-13T09:14:22.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:14:22.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799429+0000", 2024-09-13T09:14:22.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4670000000000001e-06 2024-09-13T09:14:22.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799434+0000", 2024-09-13T09:14:22.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0520000000000001e-06 2024-09-13T09:14:22.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.801379+0000", 2024-09-13T09:14:22.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019449599999999999 2024-09-13T09:14:22.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.801426+0000", 2024-09-13T09:14:22.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7391000000000001e-05 2024-09-13T09:14:22.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.803187+0000", 2024-09-13T09:14:22.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0017609870000000001 2024-09-13T09:14:22.450 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.450 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.450 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.450 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.3.0:2661 3.d e124/122 3:b5ba65cb:::rbd_data.1123fa58739e.0000000000000006:12 v 124'356, mlcod=123'351)", 2024-09-13T09:14:22.450 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:22.450 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.6705314449999999, 2024-09-13T09:14:22.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034693089999999998, 2024-09-13T09:14:22.450 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.450 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:22.451 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:22.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:22.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.452 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:22.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:22.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.452 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748155+0000", 2024-09-13T09:14:22.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3769999999999996e-06 2024-09-13T09:14:22.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748163+0000", 2024-09-13T09:14:22.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0120000000000005e-06 2024-09-13T09:14:22.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748169+0000", 2024-09-13T09:14:22.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0610000000000003e-06 2024-09-13T09:14:22.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748695+0000", 2024-09-13T09:14:22.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052687199999999995 2024-09-13T09:14:22.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748716+0000", 2024-09-13T09:14:22.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0126e-05 2024-09-13T09:14:22.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:22.455 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.751587+0000", 2024-09-13T09:14:22.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028710839999999999 2024-09-13T09:14:22.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.455 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.455 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.751617+0000", 2024-09-13T09:14:22.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0096000000000001e-05 2024-09-13T09:14:22.455 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.455 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.455 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.456 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGPush(3.9 124/123 [PushOp(3:94c9dbee:::rbd_data.112339c5624d.0000000000000000:head, version: 88'1057, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:94c9dbee:::rbd_data.112339c5624d.0000000000000000:head@88'1057, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:0, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:14:22.456 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:17.795705+0000", 2024-09-13T09:14:22.456 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.6229742859999998, 2024-09-13T09:14:22.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0038339609999999999, 2024-09-13T09:14:22.456 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.456 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:22.456 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.456 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795705+0000", 2024-09-13T09:14:22.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795705+0000", 2024-09-13T09:14:22.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795704+0000", 2024-09-13T09:14:22.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:22.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795715+0000", 2024-09-13T09:14:22.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1025e-05 2024-09-13T09:14:22.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795727+0000", 2024-09-13T09:14:22.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2031000000000001e-05 2024-09-13T09:14:22.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795736+0000", 2024-09-13T09:14:22.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2220000000000008e-06 2024-09-13T09:14:22.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.460 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795816+0000", 2024-09-13T09:14:22.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9338000000000005e-05 2024-09-13T09:14:22.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.460 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795829+0000", 2024-09-13T09:14:22.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3677e-05 2024-09-13T09:14:22.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.799539+0000", 2024-09-13T09:14:22.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037092169999999999 2024-09-13T09:14:22.461 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.461 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.461 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.462 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.3.0:2692 3.d e125/122 3:b6fe9194:::rbd_data.112326fb43d3.000000000000002a:11 v 125'358, mlcod=124'354)", 2024-09-13T09:14:22.462 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:18.770322+0000", 2024-09-13T09:14:22.462 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.648357034, 2024-09-13T09:14:22.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047215089999999996, 2024-09-13T09:14:22.462 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.462 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:22.462 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770322+0000", 2024-09-13T09:14:22.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.463 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770322+0000", 2024-09-13T09:14:22.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.463 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770321+0000", 2024-09-13T09:14:22.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:22.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.464 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770333+0000", 2024-09-13T09:14:22.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2014000000000001e-05 2024-09-13T09:14:22.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.464 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770346+0000", 2024-09-13T09:14:22.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2771e-05 2024-09-13T09:14:22.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770354+0000", 2024-09-13T09:14:22.465 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1249999999999993e-06 2024-09-13T09:14:22.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770459+0000", 2024-09-13T09:14:22.465 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010558200000000001 2024-09-13T09:14:22.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770483+0000", 2024-09-13T09:14:22.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3125e-05 2024-09-13T09:14:22.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:22.466 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.775015+0000", 2024-09-13T09:14:22.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045322449999999999 2024-09-13T09:14:22.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.466 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.775043+0000", 2024-09-13T09:14:22.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8532999999999999e-05 2024-09-13T09:14:22.467 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.467 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.467 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.467 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2444 3.9 e126/125 3:9113591d:::rbd_data.112326fb43d3.0000000000000108:11 v 126'1138, mlcod=126'1134)", 2024-09-13T09:14:22.467 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.781633+0000", 2024-09-13T09:14:22.467 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.6370453500000002, 2024-09-13T09:14:22.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0032950620000000001, 2024-09-13T09:14:22.467 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.467 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:22.467 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.468 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.468 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.781633+0000", 2024-09-13T09:14:22.468 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.468 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.468 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.781633+0000", 2024-09-13T09:14:22.468 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.469 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.469 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.781633+0000", 2024-09-13T09:14:22.469 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:22.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.469 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.469 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.781641+0000", 2024-09-13T09:14:22.469 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9740000000000002e-06 2024-09-13T09:14:22.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.469 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.469 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.781651+0000", 2024-09-13T09:14:22.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0200000000000001e-05 2024-09-13T09:14:22.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.781657+0000", 2024-09-13T09:14:22.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.516e-06 2024-09-13T09:14:22.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.781714+0000", 2024-09-13T09:14:22.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6456000000000003e-05 2024-09-13T09:14:22.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.781734+0000", 2024-09-13T09:14:22.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0591999999999999e-05 2024-09-13T09:14:22.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:22.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784860+0000", 2024-09-13T09:14:22.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003125592 2024-09-13T09:14:22.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.472 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.472 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784929+0000", 2024-09-13T09:14:22.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8499999999999998e-05 2024-09-13T09:14:22.472 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.472 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.472 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.472 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.472 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2445 3.9 e126/125 3:91fd9b46:::rbd_data.112326fb43d3.000000000000010f:11 v 126'1140, mlcod=126'1134)", 2024-09-13T09:14:22.472 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:22.472 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.636500689, 2024-09-13T09:14:22.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0036771899999999999, 2024-09-13T09:14:22.473 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.473 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:22.473 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.473 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.473 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.473 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:22.473 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.473 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.473 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.473 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:22.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.474 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:22.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:22.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.474 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782184+0000", 2024-09-13T09:14:22.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5729999999999996e-06 2024-09-13T09:14:22.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782196+0000", 2024-09-13T09:14:22.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1817000000000001e-05 2024-09-13T09:14:22.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782202+0000", 2024-09-13T09:14:22.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:14:22.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.783326+0000", 2024-09-13T09:14:22.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011234509999999999 2024-09-13T09:14:22.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.783343+0000", 2024-09-13T09:14:22.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7504e-05 2024-09-13T09:14:22.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:22.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.785708+0000", 2024-09-13T09:14:22.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023648829999999999 2024-09-13T09:14:22.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.785855+0000", 2024-09-13T09:14:22.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014743199999999999 2024-09-13T09:14:22.478 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.478 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.478 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.478 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.1.0:2475 3.4 e126/125 3:2095a19f:::rbd_data.1123b428d19a.000000000000001b:16 v 126'630, mlcod=126'626)", 2024-09-13T09:14:22.478 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.784119+0000", 2024-09-13T09:14:22.478 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.6345601840000001, 2024-09-13T09:14:22.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033969479999999999, 2024-09-13T09:14:22.478 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.478 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:22.478 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784119+0000", 2024-09-13T09:14:22.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784119+0000", 2024-09-13T09:14:22.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784118+0000", 2024-09-13T09:14:22.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:22.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784126+0000", 2024-09-13T09:14:22.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6490000000000008e-06 2024-09-13T09:14:22.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784136+0000", 2024-09-13T09:14:22.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9699999999999994e-06 2024-09-13T09:14:22.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784141+0000", 2024-09-13T09:14:22.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8730000000000004e-06 2024-09-13T09:14:22.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784219+0000", 2024-09-13T09:14:22.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7650000000000004e-05 2024-09-13T09:14:22.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784239+0000", 2024-09-13T09:14:22.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9794999999999999e-05 2024-09-13T09:14:22.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:22.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.787481+0000", 2024-09-13T09:14:22.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0032426320000000001 2024-09-13T09:14:22.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.787516+0000", 2024-09-13T09:14:22.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4143000000000002e-05 2024-09-13T09:14:22.483 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.483 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.483 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.483 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2447 3.9 e126/125 3:9541a82a:::rbd_data.1123b428d19a.000000000000000a:16 v 126'1144, mlcod=126'1140)", 2024-09-13T09:14:22.483 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.793072+0000", 2024-09-13T09:14:22.483 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.625606968, 2024-09-13T09:14:22.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046128150000000001, 2024-09-13T09:14:22.484 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.484 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:22.484 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793072+0000", 2024-09-13T09:14:22.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793072+0000", 2024-09-13T09:14:22.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793071+0000", 2024-09-13T09:14:22.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:22.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793077+0000", 2024-09-13T09:14:22.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1080000000000001e-06 2024-09-13T09:14:22.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793085+0000", 2024-09-13T09:14:22.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7309999999999999e-06 2024-09-13T09:14:22.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793091+0000", 2024-09-13T09:14:22.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6300000000000003e-06 2024-09-13T09:14:22.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793160+0000", 2024-09-13T09:14:22.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8681000000000002e-05 2024-09-13T09:14:22.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793178+0000", 2024-09-13T09:14:22.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8861000000000001e-05 2024-09-13T09:14:22.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:22.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.797650+0000", 2024-09-13T09:14:22.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004471193 2024-09-13T09:14:22.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.797685+0000", 2024-09-13T09:14:22.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5042000000000001e-05 2024-09-13T09:14:22.488 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.488 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.488 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.488 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop_reply(osd.2.0:98 3.18 e126/125 ondisk, result = 0)", 2024-09-13T09:14:22.489 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.794045+0000", 2024-09-13T09:14:22.489 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.624634141, 2024-09-13T09:14:22.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050958949999999996, 2024-09-13T09:14:22.489 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.489 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:22.489 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794045+0000", 2024-09-13T09:14:22.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794045+0000", 2024-09-13T09:14:22.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794044+0000", 2024-09-13T09:14:22.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:22.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794049+0000", 2024-09-13T09:14:22.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8600000000000001e-06 2024-09-13T09:14:22.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794060+0000", 2024-09-13T09:14:22.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0801e-05 2024-09-13T09:14:22.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794068+0000", 2024-09-13T09:14:22.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1550000000000007e-06 2024-09-13T09:14:22.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.799120+0000", 2024-09-13T09:14:22.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050525040000000002 2024-09-13T09:14:22.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.799128+0000", 2024-09-13T09:14:22.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2819999999999996e-06 2024-09-13T09:14:22.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.799141+0000", 2024-09-13T09:14:22.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2098e-05 2024-09-13T09:14:22.493 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.493 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.493 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.493 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.1.0:2481 3.4 e126/125 3:212326b5:::rbd_data.11239759ab94.0000000000000006:1b v 126'642, mlcod=126'638)", 2024-09-13T09:14:22.493 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.800631+0000", 2024-09-13T09:14:22.494 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.618048006, 2024-09-13T09:14:22.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0032122819999999999, 2024-09-13T09:14:22.494 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.494 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:22.494 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.800631+0000", 2024-09-13T09:14:22.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.800631+0000", 2024-09-13T09:14:22.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.800630+0000", 2024-09-13T09:14:22.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:22.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.800636+0000", 2024-09-13T09:14:22.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0109999999999997e-06 2024-09-13T09:14:22.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.800647+0000", 2024-09-13T09:14:22.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0567000000000001e-05 2024-09-13T09:14:22.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.800653+0000", 2024-09-13T09:14:22.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.443e-06 2024-09-13T09:14:22.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.800709+0000", 2024-09-13T09:14:22.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5766000000000001e-05 2024-09-13T09:14:22.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.800728+0000", 2024-09-13T09:14:22.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8732e-05 2024-09-13T09:14:22.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:22.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.803796+0000", 2024-09-13T09:14:22.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0030683720000000002 2024-09-13T09:14:22.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.803843+0000", 2024-09-13T09:14:22.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7080999999999999e-05 2024-09-13T09:14:22.498 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.498 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.498 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.498 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2450 3.9 e126/125 3:95b1a540:::rbd_data.1123e3c0a1e8.0000000000000030:18 v 126'1148, mlcod=126'1144)", 2024-09-13T09:14:22.499 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:22.499 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.6170505049999999, 2024-09-13T09:14:22.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0035455719999999999, 2024-09-13T09:14:22.499 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.499 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:22.499 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:22.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:22.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:22.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:22.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801634+0000", 2024-09-13T09:14:22.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9539999999999998e-06 2024-09-13T09:14:22.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801646+0000", 2024-09-13T09:14:22.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1795e-05 2024-09-13T09:14:22.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801653+0000", 2024-09-13T09:14:22.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7259999999999996e-06 2024-09-13T09:14:22.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.803017+0000", 2024-09-13T09:14:22.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001363997 2024-09-13T09:14:22.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.803038+0000", 2024-09-13T09:14:22.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1294e-05 2024-09-13T09:14:22.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:22.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.805138+0000", 2024-09-13T09:14:22.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020996259999999998 2024-09-13T09:14:22.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.805174+0000", 2024-09-13T09:14:22.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5970999999999998e-05 2024-09-13T09:14:22.503 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.503 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.503 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.504 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2452 3.9 e126/125 3:9030a72f:::rbd_data.1123af309369.0000000000000023:1f v 126'1152, mlcod=126'1148)", 2024-09-13T09:14:22.504 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.812010+0000", 2024-09-13T09:14:22.504 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.6066691550000001, 2024-09-13T09:14:22.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033163540000000001, 2024-09-13T09:14:22.504 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.504 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:22.504 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812010+0000", 2024-09-13T09:14:22.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812010+0000", 2024-09-13T09:14:22.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812009+0000", 2024-09-13T09:14:22.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:22.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812093+0000", 2024-09-13T09:14:22.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4022000000000003e-05 2024-09-13T09:14:22.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812121+0000", 2024-09-13T09:14:22.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.826e-05 2024-09-13T09:14:22.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812128+0000", 2024-09-13T09:14:22.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6270000000000002e-06 2024-09-13T09:14:22.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812170+0000", 2024-09-13T09:14:22.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1785000000000001e-05 2024-09-13T09:14:22.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812202+0000", 2024-09-13T09:14:22.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2505999999999997e-05 2024-09-13T09:14:22.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:22.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.815226+0000", 2024-09-13T09:14:22.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003024185 2024-09-13T09:14:22.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.815326+0000", 2024-09-13T09:14:22.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9637999999999997e-05 2024-09-13T09:14:22.508 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.508 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.508 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.509 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2453 3.9 e126/125 3:94dbe637:::rbd_data.1123af309369.00000000000001dc:1f v 126'1154, mlcod=126'1148)", 2024-09-13T09:14:22.509 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.812843+0000", 2024-09-13T09:14:22.509 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.605836295, 2024-09-13T09:14:22.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034843890000000001, 2024-09-13T09:14:22.509 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.509 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:22.509 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812843+0000", 2024-09-13T09:14:22.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812843+0000", 2024-09-13T09:14:22.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812842+0000", 2024-09-13T09:14:22.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:22.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812848+0000", 2024-09-13T09:14:22.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4189999999999999e-06 2024-09-13T09:14:22.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812859+0000", 2024-09-13T09:14:22.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0778000000000001e-05 2024-09-13T09:14:22.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812866+0000", 2024-09-13T09:14:22.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7619999999999998e-06 2024-09-13T09:14:22.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.813231+0000", 2024-09-13T09:14:22.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00036497299999999999 2024-09-13T09:14:22.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.813251+0000", 2024-09-13T09:14:22.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0511999999999999e-05 2024-09-13T09:14:22.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:22.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.816298+0000", 2024-09-13T09:14:22.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003046801 2024-09-13T09:14:22.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.816327+0000", 2024-09-13T09:14:22.514 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8943000000000001e-05 2024-09-13T09:14:22.514 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.514 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.514 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.514 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2455 3.9 e126/125 3:95f6e286:::rbd_data.1123af309369.00000000000000ba:1f v 126'1158, mlcod=126'1152)", 2024-09-13T09:14:22.514 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.817505+0000", 2024-09-13T09:14:22.515 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.6011738179999999, 2024-09-13T09:14:22.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034876989999999999, 2024-09-13T09:14:22.515 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.515 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:22.515 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817505+0000", 2024-09-13T09:14:22.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817505+0000", 2024-09-13T09:14:22.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817504+0000", 2024-09-13T09:14:22.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:22.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817512+0000", 2024-09-13T09:14:22.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1389999999999995e-06 2024-09-13T09:14:22.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817532+0000", 2024-09-13T09:14:22.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9089e-05 2024-09-13T09:14:22.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817538+0000", 2024-09-13T09:14:22.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5799999999999997e-06 2024-09-13T09:14:22.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.818301+0000", 2024-09-13T09:14:22.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00076256200000000005 2024-09-13T09:14:22.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.818318+0000", 2024-09-13T09:14:22.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7428999999999999e-05 2024-09-13T09:14:22.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:22.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820959+0000", 2024-09-13T09:14:22.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026407129999999998 2024-09-13T09:14:22.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820993+0000", 2024-09-13T09:14:22.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3912000000000003e-05 2024-09-13T09:14:22.519 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.519 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.519 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.520 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop_reply(osd.2.0:107 3.18 e126/125 ondisk, result = 0)", 2024-09-13T09:14:22.520 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.820584+0000", 2024-09-13T09:14:22.520 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.598095227, 2024-09-13T09:14:22.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045400709999999997, 2024-09-13T09:14:22.520 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.520 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:22.520 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.520 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820584+0000", 2024-09-13T09:14:22.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820584+0000", 2024-09-13T09:14:22.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820583+0000", 2024-09-13T09:14:22.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:22.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820588+0000", 2024-09-13T09:14:22.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7029999999999997e-06 2024-09-13T09:14:22.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820598+0000", 2024-09-13T09:14:22.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0179e-05 2024-09-13T09:14:22.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820604+0000", 2024-09-13T09:14:22.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:14:22.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825098+0000", 2024-09-13T09:14:22.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044948009999999997 2024-09-13T09:14:22.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825108+0000", 2024-09-13T09:14:22.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3759999999999997e-06 2024-09-13T09:14:22.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825124+0000", 2024-09-13T09:14:22.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5950000000000001e-05 2024-09-13T09:14:22.524 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.524 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.524 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.524 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2456 3.9 e126/125 3:93610890:::rbd_data.1123af309369.00000000000003af:1f v 126'1160, mlcod=126'1156)", 2024-09-13T09:14:22.524 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.821541+0000", 2024-09-13T09:14:22.524 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.5971378459999999, 2024-09-13T09:14:22.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003753915, 2024-09-13T09:14:22.524 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.525 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:22.525 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821541+0000", 2024-09-13T09:14:22.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821541+0000", 2024-09-13T09:14:22.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821540+0000", 2024-09-13T09:14:22.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:22.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821546+0000", 2024-09-13T09:14:22.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7370000000000003e-06 2024-09-13T09:14:22.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821604+0000", 2024-09-13T09:14:22.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8072000000000002e-05 2024-09-13T09:14:22.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821616+0000", 2024-09-13T09:14:22.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1956999999999999e-05 2024-09-13T09:14:22.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821711+0000", 2024-09-13T09:14:22.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5079000000000002e-05 2024-09-13T09:14:22.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821731+0000", 2024-09-13T09:14:22.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9661999999999999e-05 2024-09-13T09:14:22.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:22.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825263+0000", 2024-09-13T09:14:22.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0035322399999999999 2024-09-13T09:14:22.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825295+0000", 2024-09-13T09:14:22.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1813000000000002e-05 2024-09-13T09:14:22.529 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.529 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.529 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.529 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2457 3.9 e126/125 3:9313c095:::rbd_data.1123af309369.00000000000000be:1f v 126'1162, mlcod=126'1156)", 2024-09-13T09:14:22.529 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.822601+0000", 2024-09-13T09:14:22.529 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.5960782880000002, 2024-09-13T09:14:22.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003671147, 2024-09-13T09:14:22.529 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.529 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:22.529 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822601+0000", 2024-09-13T09:14:22.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822601+0000", 2024-09-13T09:14:22.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822600+0000", 2024-09-13T09:14:22.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:22.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822605+0000", 2024-09-13T09:14:22.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6979999999999997e-06 2024-09-13T09:14:22.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822612+0000", 2024-09-13T09:14:22.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8589999999999999e-06 2024-09-13T09:14:22.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.532 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822617+0000", 2024-09-13T09:14:22.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:14:22.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.532 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.823075+0000", 2024-09-13T09:14:22.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00045848500000000001 2024-09-13T09:14:22.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.823094+0000", 2024-09-13T09:14:22.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8740999999999999e-05 2024-09-13T09:14:22.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:22.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.826239+0000", 2024-09-13T09:14:22.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0031454709999999999 2024-09-13T09:14:22.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.826272+0000", 2024-09-13T09:14:22.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2330999999999999e-05 2024-09-13T09:14:22.534 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.534 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.534 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.534 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.534 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.534 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:22.535 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:22.535+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:22.536 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:14:22.636 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:22.635+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:22.683 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:22.684 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:22.684 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:22.684 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:22.692 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:14:22.737 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:22.737+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:22.803 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:14:22.838 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:22.838+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:22.849 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:22.849 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:22.849 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:22.849 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:22.849 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:22.859 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:14:22.940 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:22.939+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:22.973 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:22.973 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:22.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:22.973 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:22.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.973 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:22.973 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:22.973 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.441538873, 2024-09-13T09:14:22.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:14:22.974 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.974 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:22.974 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:22.974 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:22.974 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:22.974 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:14:22.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.974 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:22.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:22.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:14:22.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:22.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:14:22.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:14:22.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:14:22.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:14:22.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:14:22.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:14:22.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:14:22.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:14:22.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:22.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:14:22.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:14:22.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:14:22.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:14:22.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:22.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:14:22.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:14:22.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:14:22.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:14:22.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:14:22.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:14:22.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:14:22.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:14:22.979 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.979 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.979 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.979 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:22.979 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:22.980 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.441520838, 2024-09-13T09:14:22.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:14:22.980 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.980 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:22.980 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:22.980 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:22.980 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:22.980 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:14:22.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.980 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:22.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:22.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:22.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:22.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:14:22.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:14:22.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:14:22.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:14:22.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:14:22.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:14:22.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:14:22.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:14:22.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:22.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:14:22.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:14:22.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:14:22.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:14:22.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:22.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:14:22.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:14:22.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:14:22.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:14:22.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:14:22.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:14:22.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:14:22.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:14:22.985 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.985 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.985 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.985 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:22.986 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:22.986 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.44150416799999, 2024-09-13T09:14:22.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:14:22.986 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.986 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:22.986 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:22.986 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:22.986 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:22.986 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:14:22.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.986 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:22.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:22.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:14:22.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:22.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:14:22.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:22.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:14:22.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:14:22.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:14:22.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:14:22.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:14:22.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:14:22.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:22.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:14:22.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:14:22.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:14:22.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:14:22.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:22.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:14:22.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:14:22.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:14:22.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:14:22.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:14:22.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:14:22.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:14:22.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:14:22.992 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.992 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.992 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.992 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:22.992 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:22.992 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.441486536, 2024-09-13T09:14:22.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:14:22.993 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.993 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:22.993 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:22.993 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:22.993 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:22.993 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:14:22.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.993 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:22.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:22.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:22.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:22.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:22.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:22.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:22.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:22.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:22.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:22.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:14:22.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:14:22.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:22.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:14:22.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:14:22.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:22.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:14:22.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:14:22.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:14:22.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:14:22.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:22.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:14:22.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:14:22.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:14:22.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:14:22.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:22.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:14:22.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:14:22.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:22.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:14:22.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:14:22.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:22.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:14:22.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:14:22.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:22.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:14:22.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:14:22.998 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.998 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:22.998 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:22.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:22.999 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:22.999 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:22.999 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.441440231, 2024-09-13T09:14:22.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:14:22.999 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:22.999 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:22.999 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:22.999 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:22.999 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:22.999 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:14:22.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:22.999 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:23.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:23.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:14:23.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:14:23.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:14:23.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:14:23.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:14:23.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:14:23.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:14:23.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:14:23.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:14:23.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:14:23.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:14:23.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:14:23.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:14:23.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:14:23.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:14:23.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:14:23.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:14:23.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:14:23.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:14:23.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:14:23.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:14:23.005 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.005 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.005 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.005 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.005 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:23.005 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.44141492999999, 2024-09-13T09:14:23.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:14:23.005 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.005 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.005 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.005 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.006 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.006 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:14:23.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.006 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:23.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:23.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:14:23.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:14:23.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:14:23.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:14:23.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:14:23.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:14:23.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:14:23.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:14:23.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:14:23.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:14:23.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:14:23.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:14:23.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:14:23.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:14:23.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:14:23.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:14:23.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:14:23.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:14:23.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:14:23.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:14:23.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:14:23.011 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.011 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.011 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.011 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.011 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:23.011 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.441396701, 2024-09-13T09:14:23.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:14:23.011 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.011 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.011 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.011 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.011 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.012 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:14:23.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.012 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:23.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:23.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:23.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:14:23.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:14:23.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:14:23.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:14:23.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:14:23.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:14:23.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:14:23.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:14:23.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.015 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:14:23.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:14:23.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.015 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:14:23.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:14:23.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.015 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:14:23.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:14:23.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:14:23.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:14:23.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:14:23.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:14:23.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:14:23.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:14:23.017 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.017 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.017 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.017 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.017 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:23.017 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.441349382, 2024-09-13T09:14:23.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:14:23.017 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.017 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.018 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.018 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.018 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.018 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:14:23.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.018 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.018 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.018 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:23.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:23.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:23.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:14:23.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:14:23.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:14:23.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:14:23.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:14:23.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:14:23.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:14:23.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:14:23.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:14:23.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:14:23.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:14:23.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:14:23.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:14:23.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:14:23.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:14:23.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:14:23.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:14:23.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:14:23.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:14:23.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:14:23.023 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.023 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.023 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.023 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.024 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:23.024 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.440835144, 2024-09-13T09:14:23.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:14:23.024 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.024 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.024 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.024 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.024 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.024 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:14:23.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.024 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:23.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:23.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:14:23.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:23.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:14:23.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:14:23.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:14:23.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:14:23.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:14:23.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:14:23.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:14:23.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:14:23.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:14:23.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:14:23.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:14:23.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:14:23.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:14:23.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:14:23.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:14:23.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:14:23.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:14:23.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:14:23.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:14:23.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:14:23.030 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.030 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.030 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.031 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.031 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:23.031 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.440800614, 2024-09-13T09:14:23.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:14:23.031 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.031 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.031 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.031 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.031 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.031 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:14:23.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.031 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:23.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:23.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:23.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:14:23.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:14:23.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:14:23.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:14:23.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:14:23.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:14:23.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:14:23.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:14:23.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:14:23.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:14:23.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:14:23.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:14:23.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:14:23.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:14:23.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:14:23.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:14:23.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.036 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.036 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:14:23.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:14:23.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.036 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.036 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:14:23.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:14:23.036 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.036 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.036 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.037 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.037 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:23.037 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.440782707, 2024-09-13T09:14:23.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:14:23.037 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.037 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.037 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.037 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.037 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.037 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:14:23.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.038 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:23.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:23.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:23.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:14:23.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:14:23.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:14:23.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:14:23.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:14:23.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:14:23.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:14:23.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:14:23.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:14:23.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:14:23.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:14:23.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:14:23.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:14:23.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:14:23.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:14:23.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:14:23.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:14:23.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:14:23.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:14:23.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:14:23.042 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.042 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.042 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.043 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.043 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:23.043 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.431655859, 2024-09-13T09:14:23.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:14:23.043 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.043 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.043 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:14:23.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.043 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:23.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:23.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:23.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:14:23.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:14:23.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:14:23.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:14:23.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:14:23.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:14:23.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:14:23.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:14:23.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:14:23.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:14:23.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:14:23.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:14:23.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:14:23.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:14:23.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:14:23.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:14:23.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:14:23.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:14:23.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:14:23.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:14:23.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.048 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.049 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.049 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:23.049 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.431604279, 2024-09-13T09:14:23.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:14:23.049 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.049 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.049 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.049 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.049 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.049 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:14:23.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.049 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:23.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:23.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:14:23.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:14:23.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:14:23.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:14:23.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:14:23.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:14:23.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:14:23.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:14:23.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:14:23.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:14:23.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:14:23.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:14:23.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:14:23.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:14:23.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:14:23.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:14:23.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:14:23.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:14:23.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:14:23.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:14:23.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:14:23.054 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.054 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.054 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.055 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.055 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:23.055 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.431588003, 2024-09-13T09:14:23.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:14:23.055 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.055 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.055 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.055 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.055 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.055 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:14:23.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.055 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:23.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:23.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:14:23.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:14:23.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:14:23.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:14:23.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:14:23.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:14:23.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:14:23.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:14:23.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:14:23.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:14:23.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:14:23.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:14:23.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:14:23.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:14:23.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:14:23.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:14:23.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:14:23.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:14:23.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:14:23.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:14:23.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:14:23.060 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.060 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.061 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.061 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.061 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:23.061 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.431572229, 2024-09-13T09:14:23.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:14:23.061 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.061 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.061 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.061 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.061 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.061 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:14:23.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.062 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:23.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:23.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:14:23.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:14:23.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:14:23.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:14:23.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:14:23.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:14:23.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:14:23.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:14:23.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:14:23.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:14:23.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:14:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:14:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:14:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:14:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:14:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:14:23.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:14:23.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:14:23.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:14:23.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:14:23.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:14:23.067 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.067 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.067 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.067 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.067 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:23.067 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.43155735400001, 2024-09-13T09:14:23.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:14:23.067 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.067 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.068 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.068 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.068 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.068 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:14:23.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.068 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:23.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:23.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:23.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:14:23.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:14:23.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.070 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.070 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:14:23.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:14:23.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.070 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.070 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:14:23.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:14:23.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.070 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.070 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:14:23.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:14:23.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:14:23.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:14:23.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:14:23.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:14:23.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:14:23.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:14:23.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:14:23.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:14:23.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:14:23.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:14:23.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.073 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:14:23.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:14:23.073 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.073 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.073 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.073 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.073 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:23.073 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.431539892, 2024-09-13T09:14:23.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:14:23.073 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.073 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.074 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.074 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.074 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.074 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:14:23.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.074 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.074 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.074 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:23.074 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.074 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.075 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:23.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.075 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.075 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:14:23.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.075 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.075 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:14:23.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:14:23.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.076 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.076 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:14:23.076 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:14:23.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.076 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.076 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:14:23.076 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:14:23.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.076 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.076 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:14:23.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:14:23.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:14:23.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:14:23.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:14:23.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:14:23.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:14:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:14:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:14:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:14:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:14:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:14:23.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:14:23.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:14:23.079 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.079 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.079 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.079 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.079 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:23.079 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.42993926, 2024-09-13T09:14:23.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:14:23.079 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.080 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.080 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.080 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.080 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.080 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:14:23.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.080 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.080 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.080 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:23.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.081 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.081 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:23.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.081 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.081 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:14:23.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.081 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.081 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:14:23.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:14:23.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.082 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.082 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:14:23.082 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:14:23.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.082 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.082 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:14:23.082 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:14:23.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.082 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.082 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:14:23.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:14:23.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.083 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.083 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:14:23.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:14:23.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.083 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.083 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:14:23.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:14:23.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.083 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:14:23.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:14:23.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:14:23.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:14:23.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:14:23.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:14:23.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:14:23.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:14:23.085 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.085 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.085 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.085 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.085 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:23.085 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.429899209, 2024-09-13T09:14:23.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:14:23.085 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.086 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.086 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.086 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.086 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.086 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:14:23.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.086 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.086 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.086 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:23.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.087 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.087 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:23.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.087 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.087 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:14:23.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.087 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.087 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:14:23.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:14:23.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:14:23.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:14:23.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:14:23.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:14:23.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:14:23.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:14:23.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:14:23.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:14:23.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:14:23.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:14:23.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:14:23.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:14:23.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:14:23.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:14:23.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:14:23.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:14:23.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:14:23.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:14:23.091 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.091 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.091 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.091 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.091 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:23.091 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 100.429855446, 2024-09-13T09:14:23.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:14:23.092 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.092 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.092 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.092 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.092 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.092 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:14:23.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.092 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.093 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:23.093 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.093 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:23.093 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.093 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:23.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:14:23.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:14:23.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:14:23.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:14:23.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:14:23.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:14:23.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:14:23.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:14:23.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:14:23.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:14:23.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:14:23.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:14:23.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:14:23.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:14:23.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:14:23.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:14:23.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:14:23.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:14:23.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:14:23.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:14:23.097 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.097 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.097 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.097 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.097 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.098 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:23.099 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:14:23.100 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:23.098+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:23.200 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:23.199+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:23.246 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:23.247 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:23.247 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:23.247 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:23.255 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:14:23.301 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:23.300+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:23.401 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:23.401 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:23.401 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:23.401 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:23.401 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:23.402 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:23.401+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:23.410 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:14:23.502 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:23.502+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:23.518 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:23.518 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:23.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:23.518 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:23.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.518 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.518 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:23.518 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.012555147, 2024-09-13T09:14:23.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:14:23.518 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.518 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.519 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.519 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.519 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.519 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:14:23.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.519 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:23.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.520 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:23.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.520 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:14:23.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.520 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:14:23.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:14:23.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:14:23.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:14:23.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:14:23.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:14:23.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:14:23.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:14:23.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:14:23.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:14:23.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:14:23.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:14:23.522 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.522 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.522 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.523 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.523 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:23.523 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.012527463, 2024-09-13T09:14:23.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:14:23.523 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.523 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.523 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.523 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.523 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.523 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:14:23.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.523 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:23.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:23.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:23.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:14:23.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:14:23.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:14:23.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:14:23.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:14:23.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:14:23.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:14:23.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:14:23.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:14:23.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:14:23.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:14:23.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:14:23.526 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.527 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.527 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.527 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.527 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:23.527 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.012503122, 2024-09-13T09:14:23.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:14:23.527 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.527 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.527 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.527 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.527 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.527 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:14:23.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.528 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:23.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:23.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:14:23.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:14:23.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:14:23.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:14:23.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:14:23.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:14:23.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:14:23.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:14:23.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:14:23.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:14:23.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:14:23.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:14:23.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:14:23.531 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.531 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.531 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.532 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.532 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:23.532 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.012486802, 2024-09-13T09:14:23.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:14:23.532 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.532 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.532 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.532 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.532 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.532 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:14:23.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.532 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:23.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:23.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:23.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:14:23.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:23.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:14:23.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:14:23.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:14:23.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:14:23.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:14:23.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:14:23.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:14:23.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:14:23.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:14:23.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:14:23.536 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.536 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.536 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.537 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.537 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:23.537 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.01247038699999, 2024-09-13T09:14:23.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:14:23.537 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.537 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.537 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.537 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.537 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.537 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:14:23.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.537 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:23.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:23.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:23.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:14:23.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:14:23.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:14:23.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:14:23.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:14:23.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:14:23.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:14:23.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:14:23.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:14:23.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:14:23.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.541 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:14:23.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:14:23.541 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.541 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.541 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.541 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.541 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:23.541 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.012454279, 2024-09-13T09:14:23.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:14:23.541 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.542 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.542 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.542 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.542 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.542 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:14:23.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.542 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:23.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.543 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.543 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:23.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.543 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.543 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:23.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.543 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.543 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:14:23.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:14:23.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:14:23.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:14:23.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:14:23.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:14:23.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:14:23.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:14:23.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:14:23.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:14:23.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:14:23.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:14:23.546 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.546 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.546 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.546 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.546 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:23.546 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.012438718, 2024-09-13T09:14:23.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:14:23.547 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.547 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.547 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.547 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.547 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.547 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:14:23.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.547 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:23.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:23.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:23.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:14:23.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:14:23.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:14:23.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:14:23.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:14:23.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:14:23.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:14:23.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:14:23.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:14:23.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:14:23.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:14:23.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:14:23.551 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.551 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.551 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.551 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.551 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:23.552 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.01242188000001, 2024-09-13T09:14:23.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:14:23.552 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.552 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.552 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.552 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.552 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.552 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:14:23.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.552 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:23.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:23.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:14:23.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:14:23.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:14:23.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:14:23.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:14:23.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:14:23.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:14:23.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:14:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:14:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:14:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:14:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:14:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:14:23.555 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.012405888, 2024-09-13T09:14:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:14:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.556 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:14:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:14:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:14:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:14:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:14:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:14:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:14:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:14:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:14:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:14:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:14:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:14:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:14:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:14:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.01237838999999, 2024-09-13T09:14:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:14:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.560 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:14:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:23.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:23.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:14:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:14:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:14:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:14:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:14:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:14:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:14:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:14:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:14:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:14:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:14:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:14:23.564 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.012352531, 2024-09-13T09:14:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:14:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.565 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:14:23.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.566 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:23.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:14:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:14:23.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:14:23.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:14:23.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:14:23.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:14:23.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:14:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:14:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:14:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:14:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:14:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:14:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.569 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.012311041, 2024-09-13T09:14:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:14:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:14:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.570 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:23.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:23.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:14:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:14:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:14:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:14:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:14:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:14:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:14:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:14:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:14:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:14:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:14:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:14:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:14:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.574 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.012289803, 2024-09-13T09:14:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:14:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:14:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:23.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:14:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:14:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:14:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:14:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:14:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:14:23.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:14:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:14:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:14:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:14:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:14:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:14:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.01226700300001, 2024-09-13T09:14:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:14:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.579 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:14:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:14:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:14:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:14:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:14:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:14:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:14:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:14:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:14:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:14:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:14:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:14:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:14:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:14:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.583 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.01225054699999, 2024-09-13T09:14:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:14:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:14:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:14:23.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:14:23.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:14:23.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:14:23.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:14:23.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:14:23.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:14:23.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:14:23.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:14:23.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:14:23.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:14:23.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:14:23.588 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.588 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.588 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.588 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.588 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:23.588 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.01200541, 2024-09-13T09:14:23.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:14:23.588 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.589 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.589 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.589 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.589 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.589 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:14:23.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.589 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:23.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:23.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:14:23.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:14:23.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:14:23.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:14:23.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:14:23.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:14:23.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:14:23.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:14:23.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:14:23.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:14:23.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:14:23.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:14:23.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:14:23.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:14:23.592 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.592 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.592 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.593 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.593 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:23.593 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.010119851, 2024-09-13T09:14:23.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:14:23.593 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.593 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.593 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.593 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.593 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.593 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:14:23.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.593 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:23.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:23.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:14:23.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:14:23.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:14:23.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:14:23.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:14:23.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:14:23.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:14:23.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:14:23.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:14:23.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:14:23.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:14:23.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:14:23.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:14:23.597 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.597 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.597 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.598 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.598 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:23.598 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.010098485, 2024-09-13T09:14:23.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:14:23.598 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.598 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.598 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.598 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.598 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.598 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:14:23.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.598 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:23.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:23.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:23.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:23.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:14:23.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:14:23.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:14:23.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:14:23.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:14:23.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:14:23.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:14:23.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:14:23.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:14:23.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:14:23.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.602 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:14:23.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:14:23.602 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.602 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.602 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.602 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:23.602 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:23.602 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 101.009893117, 2024-09-13T09:14:23.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:14:23.602 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.603 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:23.603 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.603 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:23.603 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:23.603 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:14:23.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.603 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:23.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:23.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:14:23.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:14:23.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:14:23.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:14:23.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:14:23.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:14:23.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:14:23.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:14:23.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:14:23.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:14:23.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:14:23.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:14:23.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:14:23.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:14:23.607 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.607 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.607 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.607 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5272.0:1775 3.0 3:02a42fca:::rbd_data.11235782ee8d.0000000000000030:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2244608~40448] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:14:23.607 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:23.607 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 73.268351276000004, 2024-09-13T09:14:23.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.666843732, 2024-09-13T09:14:23.607 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:23.607 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:23.607 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:23.607 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5272", 2024-09-13T09:14:23.608 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4089902097", 2024-09-13T09:14:23.608 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1775 2024-09-13T09:14:23.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.608 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:23.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:23.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:23.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:23.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:23.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:23.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:23.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247780+0000", 2024-09-13T09:14:23.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:23.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:23.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247865+0000", 2024-09-13T09:14:23.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4359999999999999e-05 2024-09-13T09:14:23.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:23.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247876+0000", 2024-09-13T09:14:23.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1456e-05 2024-09-13T09:14:23.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:23.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247883+0000", 2024-09-13T09:14:23.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:14:23.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247926+0000", 2024-09-13T09:14:23.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3203e-05 2024-09-13T09:14:23.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:23.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247932+0000", 2024-09-13T09:14:23.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:14:23.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909639+0000", 2024-09-13T09:14:23.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6617069099999999 2024-09-13T09:14:23.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:23.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909828+0000", 2024-09-13T09:14:23.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000189075 2024-09-13T09:14:23.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:23.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911676+0000", 2024-09-13T09:14:23.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001847235 2024-09-13T09:14:23.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:23.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911765+0000", 2024-09-13T09:14:23.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9530000000000005e-05 2024-09-13T09:14:23.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:14:23.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911901+0000", 2024-09-13T09:14:23.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013603899999999999 2024-09-13T09:14:23.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:23.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.912889+0000", 2024-09-13T09:14:23.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098807400000000003 2024-09-13T09:14:23.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:23.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914438+0000", 2024-09-13T09:14:23.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015491070000000001 2024-09-13T09:14:23.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:23.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914505+0000", 2024-09-13T09:14:23.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:14:23.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:23.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:23.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:23.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914625+0000", 2024-09-13T09:14:23.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120108 2024-09-13T09:14:23.614 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.614 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.614 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.614 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:23.614 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:23.614 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:23.616 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:14:23.616 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:23.615+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:23.716 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:23.715+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:23.730 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:23.730 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:23.730 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:23.730 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:23.744 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:14:23.810 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-09-13T09:14:23.818 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:23.817+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:23.855 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:23.855 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:23.855 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:23.856 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:23.856 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:23.865 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:14:23.952 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:23.951+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:24.013 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:24.013 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:24.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:24.013 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:24.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.014 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.014 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:24.014 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.49012661099999, 2024-09-13T09:14:24.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:14:24.014 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.014 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.014 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:14:24.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.014 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:24.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:24.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:24.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:24.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:14:24.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:14:24.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:14:24.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:14:24.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:14:24.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:14:24.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:14:24.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:14:24.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:14:24.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:14:24.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:14:24.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:14:24.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:14:24.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:14:24.018 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.018 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.018 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.019 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.019 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:24.019 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.490077555, 2024-09-13T09:14:24.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:14:24.019 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.019 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.019 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:14:24.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.019 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:24.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:24.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:24.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:24.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:14:24.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:14:24.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:14:24.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:14:24.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:14:24.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:14:24.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:14:24.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:14:24.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:14:24.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:14:24.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:14:24.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:14:24.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:14:24.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:14:24.024 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.024 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.024 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.024 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.024 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:24.024 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.49003748, 2024-09-13T09:14:24.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:14:24.024 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.024 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.025 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.025 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.025 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.025 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:14:24.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.025 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:24.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:24.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:24.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:24.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:14:24.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:14:24.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:14:24.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:14:24.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:14:24.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:14:24.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:14:24.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:14:24.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:14:24.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:14:24.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:14:24.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:14:24.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:14:24.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:14:24.029 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.029 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.029 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.029 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.029 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:24.029 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.49000766499999, 2024-09-13T09:14:24.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:14:24.029 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.029 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.030 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.030 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.030 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.030 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:14:24.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.030 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:24.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:24.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:24.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:24.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:14:24.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:14:24.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:14:24.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:14:24.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:14:24.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:14:24.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:14:24.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:14:24.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:14:24.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:14:24.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:14:24.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:14:24.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:14:24.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:14:24.034 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.034 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.034 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.034 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.034 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:24.035 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.48997800399999, 2024-09-13T09:14:24.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:14:24.035 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.035 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.035 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.035 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.035 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.035 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:14:24.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.035 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:24.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:24.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:24.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:24.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:14:24.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:14:24.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:14:24.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:14:24.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:14:24.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:14:24.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:14:24.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:14:24.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:14:24.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:14:24.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:14:24.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:14:24.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:14:24.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:14:24.039 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.039 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.039 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.039 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.039 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:24.040 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.489948797, 2024-09-13T09:14:24.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:14:24.040 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.040 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.040 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.040 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.040 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.040 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:14:24.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.040 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:24.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:24.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:24.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:24.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:14:24.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:14:24.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:14:24.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:14:24.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:14:24.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:14:24.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:14:24.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:14:24.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:14:24.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:14:24.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:14:24.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:14:24.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:14:24.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:14:24.044 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.044 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.044 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.045 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.045 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:24.045 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.48989261600001, 2024-09-13T09:14:24.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:14:24.045 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.045 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.045 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.045 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.045 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.045 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:14:24.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.045 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:24.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:24.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:24.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:24.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:14:24.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:14:24.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:14:24.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:14:24.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:14:24.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:14:24.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:14:24.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:14:24.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:14:24.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:14:24.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:14:24.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:14:24.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:14:24.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:14:24.049 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.049 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.049 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.050 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.050 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:24.050 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.489839155, 2024-09-13T09:14:24.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:14:24.050 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.050 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.050 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.050 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.050 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.050 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:14:24.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.050 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:24.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:24.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:14:24.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:24.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:14:24.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:14:24.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:14:24.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:14:24.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:14:24.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:14:24.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:14:24.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:14:24.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:14:24.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:14:24.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:14:24.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:14:24.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:14:24.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:14:24.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.054 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.055 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.055 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:24.055 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.48978232499999, 2024-09-13T09:14:24.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:14:24.055 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.055 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.055 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:14:24.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.055 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:24.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:24.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:14:24.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:24.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:14:24.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:14:24.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:14:24.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:14:24.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:14:24.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:14:24.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:14:24.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:14:24.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:14:24.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:14:24.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:14:24.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:14:24.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:14:24.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:14:24.059 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.059 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.059 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.059 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.060 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:24.060 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.489731298, 2024-09-13T09:14:24.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:14:24.060 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.060 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.060 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.060 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.060 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.060 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:14:24.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.060 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:24.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:24.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:14:24.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:24.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:14:24.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:14:24.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:14:24.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:14:24.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:14:24.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:14:24.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:14:24.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:14:24.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:14:24.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:14:24.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:14:24.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:14:24.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:14:24.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:14:24.064 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.064 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.064 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.064 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.065 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:24.065 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.489690108, 2024-09-13T09:14:24.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:14:24.065 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.065 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.065 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.065 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.065 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.065 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:14:24.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.065 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:24.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:24.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:24.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:24.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:14:24.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:14:24.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:14:24.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:14:24.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:14:24.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:14:24.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:14:24.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:14:24.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:14:24.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:14:24.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:14:24.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:14:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:14:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:14:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.489647992, 2024-09-13T09:14:24.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:14:24.070 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.070 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.070 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.070 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.070 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.070 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:14:24.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.070 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:24.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:24.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:24.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:24.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:14:24.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:14:24.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:14:24.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:14:24.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:14:24.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:14:24.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:14:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:14:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:14:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:14:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:14:24.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:14:24.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:14:24.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:14:24.074 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.074 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.074 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.074 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.074 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:24.074 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.489604113, 2024-09-13T09:14:24.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:14:24.075 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.075 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.075 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.075 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.075 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.075 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:14:24.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.075 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:24.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:24.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:24.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:24.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:14:24.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:14:24.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:14:24.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:14:24.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:14:24.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:14:24.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:14:24.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:14:24.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:14:24.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:14:24.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:14:24.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:14:24.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:14:24.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:14:24.079 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.079 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.079 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.079 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.079 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:24.079 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.489558196, 2024-09-13T09:14:24.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:14:24.079 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.080 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.080 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:14:24.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.080 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:24.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:24.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:14:24.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:24.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:14:24.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:14:24.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:14:24.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:14:24.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:14:24.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:14:24.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:14:24.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:14:24.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:14:24.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:14:24.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:14:24.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:14:24.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:14:24.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:14:24.084 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.084 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.084 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.084 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.084 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:24.085 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.489508425, 2024-09-13T09:14:24.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:14:24.085 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.085 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.085 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:14:24.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.085 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:24.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:24.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:24.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:24.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:14:24.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:14:24.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:14:24.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:14:24.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:14:24.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:14:24.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:14:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:14:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:14:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:14:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:14:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:14:24.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:14:24.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:14:24.089 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.089 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.089 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.089 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.089 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:24.089 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.48946302100001, 2024-09-13T09:14:24.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:14:24.090 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.090 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.090 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.090 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.090 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.090 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:14:24.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.090 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:24.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:24.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:24.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:24.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:14:24.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:14:24.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:14:24.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:14:24.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:14:24.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:14:24.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:14:24.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:14:24.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:14:24.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:14:24.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:14:24.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:14:24.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:14:24.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:14:24.094 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.094 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.094 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.094 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.094 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:24.094 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.489412383, 2024-09-13T09:14:24.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:14:24.095 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.095 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.095 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.095 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.095 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.095 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:14:24.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.095 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:24.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:24.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:14:24.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:24.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:14:24.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:14:24.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:14:24.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:14:24.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:14:24.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:14:24.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:14:24.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:14:24.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:14:24.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:14:24.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:14:24.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:14:24.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:14:24.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:14:24.099 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.099 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.099 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.099 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.099 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:24.099 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.48935654, 2024-09-13T09:14:24.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:14:24.100 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.100 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.100 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.100 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.100 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.100 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:14:24.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.100 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:24.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:24.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:24.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:24.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:14:24.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:14:24.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:14:24.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:14:24.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:14:24.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:14:24.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:14:24.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:14:24.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:14:24.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:14:24.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:14:24.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:14:24.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:14:24.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:14:24.104 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.104 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.104 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.104 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.104 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:24.104 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.489299618, 2024-09-13T09:14:24.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:14:24.104 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.105 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.105 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.105 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.105 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.105 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:14:24.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.105 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:24.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:24.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:24.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:24.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:14:24.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:14:24.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:14:24.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:14:24.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:14:24.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:14:24.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:14:24.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:14:24.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:14:24.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:14:24.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:14:24.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:14:24.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:14:24.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:14:24.109 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.109 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.109 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.109 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:24.109 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:24.109 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 101.489246055, 2024-09-13T09:14:24.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:14:24.109 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:24.109 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:24.109 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:24.110 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:24.110 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:24.110 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:14:24.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.110 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:24.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:24.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:24.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:24.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:24.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:24.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:24.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:24.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:24.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:24.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:14:24.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:14:24.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:24.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:14:24.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:14:24.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:24.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:14:24.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:14:24.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:14:24.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:14:24.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:24.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:14:24.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:14:24.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:24.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:14:24.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:14:24.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:24.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:24.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:24.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:14:24.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:14:24.113 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.113 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.114 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.114 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:24.114 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:24.114 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:24.115 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:14:24.116 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:24.114+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:24.124 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:14:24.216 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:24.215+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:24.246 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:14:24.247 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:14:24.261 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:24.262 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:24.262 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:24.262 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:24.263 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:14:24.263 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:14:24.270 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:14:24.279 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:14:24.279 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:14:24.295 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:14:24.295 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:14:24.312 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:14:24.312 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:14:24.316 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:24.315+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:24.328 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-09-13T09:14:24.329 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:14:24.413 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:24.413 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:24.417 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:24.417+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:24.421 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:14:24.518 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:24.517+0000 7fcb3fd87640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31628) UID: 0 2024-09-13T09:14:24.524 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:24.524 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:24.533 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:14:24.619 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:24.618+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:24.676 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:24.676 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:24.684 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:14:24.720 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:24.719+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:24.787 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:24.787 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:24.795 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:14:24.821 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:24.820+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:24.913 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:24.913 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:24.922 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:14:24.922 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:24.921+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:25.023 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:25.022+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:25.034 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:25.034 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:25.044 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:14:25.124 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:25.124+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:25.150 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:25.150 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:25.159 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:14:25.225 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:25.224+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:25.265 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:25.266 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:25.274 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:14:25.358 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:25.357+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:25.421 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:25.421 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:25.427 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:25.426+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:25.430 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:14:25.528 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:25.527+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:25.577 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:25.577 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:25.586 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:14:25.630 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:25.629+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:25.730 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:25.729+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:25.732 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:25.733 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:25.742 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:14:25.803 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 3, 4, 5, 2] 2024-09-13T09:14:25.803 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:14:25.803 INFO:tasks.thrashosds.thrasher:Killing osd 5, live_osds are [0, 1, 3, 4, 5, 2] 2024-09-13T09:14:25.803 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T09:14:25.803 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:14:25.826 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:14:25.841 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T09:14:25.841 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.5 2024-09-13T09:14:25.841 INFO:tasks.thrashosds.thrasher:remote for osd 5 is ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T09:14:25.841 DEBUG:teuthology.orchestra.run.smithi129:> type ceph-objectstore-tool 2024-09-13T09:14:25.853 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:25.853 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:25.854 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:25.852+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:25.862 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:14:25.898 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T09:14:25.898 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op list-pgs 2024-09-13T09:14:25.940 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:25.939+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:26.011 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:26.012 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:26.020 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:14:26.032 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:26.032+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:26.127 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:26.127 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:26.134 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:26.133+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:26.136 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:14:26.235 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:26.234+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:26.239 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:26.239 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:26.248 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:14:26.336 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:26.335+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:26.340 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:14:26.354 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:26.355 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:26.364 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:14:26.437 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:26.436+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:26.462 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:14:26.462 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:14:26.471 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:26.471 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:26.478 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:14:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:14:26.480 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:14:26.494 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:14:26.494 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:14:26.511 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:14:26.512 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:14:26.528 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:14:26.529 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:14:26.538 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:26.537+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:26.623 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:26.623 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:26.623 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:26.623 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:26.632 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:14:26.676 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:26.675+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:26.735 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:26.735 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:26.735 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:26.735 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:26.735 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:26.739 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:26.739+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:26.743 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:14:26.828 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-09-13T09:14:26.840 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:26.840+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:26.894 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:26.894 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:26.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:26.894 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:26.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.894 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2188 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 901120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.894 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:26.894 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.366143417000004, 2024-09-13T09:14:26.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063724617999999997, 2024-09-13T09:14:26.895 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.895 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.895 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.895 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2188 2024-09-13T09:14:26.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.895 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:26.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:26.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525800+0000", 2024-09-13T09:14:26.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525802+0000", 2024-09-13T09:14:26.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0439999999999998e-06 2024-09-13T09:14:26.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525809+0000", 2024-09-13T09:14:26.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.238e-06 2024-09-13T09:14:26.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525814+0000", 2024-09-13T09:14:26.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6630000000000001e-06 2024-09-13T09:14:26.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589327+0000", 2024-09-13T09:14:26.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063513257000000004 2024-09-13T09:14:26.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589418+0000", 2024-09-13T09:14:26.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0796000000000003e-05 2024-09-13T09:14:26.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589525+0000", 2024-09-13T09:14:26.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010693500000000001 2024-09-13T09:14:26.898 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.898 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.898 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.899 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2190 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.899 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:26.899 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.366125331000006, 2024-09-13T09:14:26.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063863981, 2024-09-13T09:14:26.899 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.899 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.899 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.899 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2190 2024-09-13T09:14:26.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.900 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:26.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525819+0000", 2024-09-13T09:14:26.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525818+0000", 2024-09-13T09:14:26.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525820+0000", 2024-09-13T09:14:26.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:14:26.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525827+0000", 2024-09-13T09:14:26.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0589999999999999e-06 2024-09-13T09:14:26.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525831+0000", 2024-09-13T09:14:26.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7870000000000001e-06 2024-09-13T09:14:26.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589562+0000", 2024-09-13T09:14:26.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063731038000000004 2024-09-13T09:14:26.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589612+0000", 2024-09-13T09:14:26.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0087999999999999e-05 2024-09-13T09:14:26.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589683+0000", 2024-09-13T09:14:26.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0121000000000002e-05 2024-09-13T09:14:26.903 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.903 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.903 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.903 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2192 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.903 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:26.904 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.366108370999996, 2024-09-13T09:14:26.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064050299000000005, 2024-09-13T09:14:26.904 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.904 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.904 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.904 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2192 2024-09-13T09:14:26.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.904 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:26.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525836+0000", 2024-09-13T09:14:26.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525835+0000", 2024-09-13T09:14:26.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525837+0000", 2024-09-13T09:14:26.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9910000000000001e-06 2024-09-13T09:14:26.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525845+0000", 2024-09-13T09:14:26.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3660000000000003e-06 2024-09-13T09:14:26.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525850+0000", 2024-09-13T09:14:26.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0030000000000002e-06 2024-09-13T09:14:26.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589720+0000", 2024-09-13T09:14:26.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.063870176000000001 2024-09-13T09:14:26.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589801+0000", 2024-09-13T09:14:26.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0889000000000004e-05 2024-09-13T09:14:26.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589886+0000", 2024-09-13T09:14:26.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5183000000000001e-05 2024-09-13T09:14:26.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.908 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.909 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.909 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:26.909 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.366090267999994, 2024-09-13T09:14:26.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:14:26.909 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.909 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.909 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:14:26.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.910 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:26.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:26.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:14:26.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:14:26.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:14:26.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:14:26.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:14:26.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:14:26.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:14:26.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:14:26.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:14:26.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:14:26.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:14:26.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:14:26.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:14:26.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.913 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.913 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.913 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:26.913 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.36607257, 2024-09-13T09:14:26.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:14:26.913 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.914 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.914 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:14:26.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.914 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:26.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:26.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:26.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:14:26.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:14:26.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:14:26.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:14:26.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:14:26.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:14:26.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:14:26.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:14:26.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:14:26.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:14:26.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:14:26.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:14:26.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.917 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.918 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.918 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:26.918 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.366051955000003, 2024-09-13T09:14:26.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:14:26.918 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.918 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.918 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:14:26.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.919 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:26.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:26.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:26.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:14:26.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:14:26.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:14:26.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:14:26.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:14:26.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:14:26.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:14:26.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:14:26.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:14:26.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:14:26.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:14:26.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:14:26.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.922 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.923 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.923 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:26.923 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.366032998999998, 2024-09-13T09:14:26.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:14:26.923 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.923 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.923 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:14:26.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.923 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:26.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:26.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:26.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:14:26.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:14:26.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:14:26.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:14:26.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:14:26.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:14:26.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:14:26.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:14:26.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:14:26.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:14:26.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:14:26.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:14:26.927 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.927 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.927 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.927 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.927 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:26.927 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.366015740999998, 2024-09-13T09:14:26.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:14:26.927 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.927 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.928 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:14:26.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.928 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:26.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:26.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:26.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:14:26.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:14:26.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:14:26.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:14:26.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:14:26.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:14:26.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:14:26.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:14:26.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:14:26.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:14:26.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:14:26.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:14:26.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.931 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.932 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.932 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:26.932 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.365995726999998, 2024-09-13T09:14:26.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:14:26.932 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.932 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.932 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:14:26.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.932 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:26.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:26.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:26.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:14:26.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:14:26.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:14:26.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:14:26.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:14:26.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:14:26.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:14:26.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:14:26.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:14:26.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:14:26.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:14:26.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:14:26.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.936 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.936 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.936 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:26.936 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.365970145000006, 2024-09-13T09:14:26.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:14:26.936 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.936 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.937 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:14:26.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.937 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:26.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:26.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:14:26.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:14:26.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:14:26.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:14:26.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:14:26.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:14:26.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:14:26.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:14:26.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:14:26.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:14:26.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:14:26.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:14:26.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:14:26.940 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.940 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.940 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.940 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.940 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:26.941 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.365950861000002, 2024-09-13T09:14:26.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:14:26.941 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.941 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.941 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:14:26.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.941 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:26.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:26.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:26.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:14:26.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:14:26.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:14:26.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:14:26.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:14:26.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:14:26.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:14:26.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:14:26.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:14:26.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:14:26.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:14:26.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:14:26.945 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.945 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.945 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.945 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.945 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:26.945 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.365933604999995, 2024-09-13T09:14:26.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:14:26.945 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.945 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.946 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:14:26.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.946 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:26.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:26.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:26.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:14:26.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:14:26.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:14:26.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:14:26.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:14:26.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:14:26.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:14:26.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:14:26.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:14:26.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:14:26.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:14:26.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:14:26.949 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.949 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.949 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.950 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.950 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:26.950 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.365914125000003, 2024-09-13T09:14:26.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:14:26.950 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.950 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.950 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:14:26.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.950 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:26.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:26.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:14:26.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:14:26.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:14:26.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:14:26.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:14:26.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:14:26.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:14:26.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:14:26.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:14:26.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:14:26.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:14:26.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:14:26.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:14:26.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.954 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.954 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.954 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:26.954 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.365897196000006, 2024-09-13T09:14:26.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:14:26.954 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.954 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.955 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:14:26.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.955 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:26.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:26.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:14:26.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:14:26.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:14:26.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:14:26.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:14:26.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:14:26.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:14:26.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:14:26.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:14:26.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:14:26.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:14:26.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:14:26.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:14:26.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.959 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.959 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.959 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.959 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:26.959 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.365878046999995, 2024-09-13T09:14:26.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:14:26.959 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.959 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.960 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:14:26.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.960 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:26.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:26.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:26.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:14:26.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:14:26.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:14:26.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:14:26.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:14:26.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:14:26.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:14:26.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:14:26.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:14:26.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:14:26.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:14:26.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:14:26.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.963 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.963 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.964 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:26.964 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.365859908000004, 2024-09-13T09:14:26.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:14:26.964 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.964 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.964 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:14:26.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.964 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:26.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:26.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:26.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:14:26.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:14:26.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:14:26.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:14:26.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:14:26.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:14:26.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:14:26.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:14:26.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:14:26.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:14:26.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:14:26.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:14:26.968 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.968 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.968 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.968 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.968 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:26.968 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.365449412999993, 2024-09-13T09:14:26.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:14:26.968 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.968 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.969 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:14:26.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.969 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:26.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:26.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:26.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:14:26.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:14:26.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:14:26.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:14:26.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:14:26.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:14:26.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:14:26.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:14:26.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:14:26.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:14:26.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:14:26.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:14:26.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.972 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.972 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:26.972 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:26.972 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 71.365426511999999, 2024-09-13T09:14:26.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:14:26.973 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.973 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:26.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:26.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:26.973 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:14:26.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.973 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:26.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:26.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:26.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:26.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:14:26.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:14:26.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:14:26.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:14:26.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:14:26.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:14:26.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:14:26.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:14:26.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:14:26.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:14:26.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:14:26.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:14:26.977 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.977 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.977 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.978 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:30141 3.9 3:908b9f7f:::rbd_data.112326fb43d3.0000000000000001:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-13T09:14:26.978 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:14:18.853518+0000", 2024-09-13T09:14:26.978 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.0384260239999996, 2024-09-13T09:14:26.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.92492906200000002, 2024-09-13T09:14:26.978 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.978 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:26.978 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.978 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:26.978 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:26.978 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 30141 2024-09-13T09:14:26.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.978 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853518+0000", 2024-09-13T09:14:26.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853518+0000", 2024-09-13T09:14:26.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853516+0000", 2024-09-13T09:14:26.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:14:26.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853527+0000", 2024-09-13T09:14:26.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1420000000000001e-05 2024-09-13T09:14:26.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853556+0000", 2024-09-13T09:14:26.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8496e-05 2024-09-13T09:14:26.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853569+0000", 2024-09-13T09:14:26.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3446e-05 2024-09-13T09:14:26.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853691+0000", 2024-09-13T09:14:26.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012215500000000001 2024-09-13T09:14:26.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:26.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853715+0000", 2024-09-13T09:14:26.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3266000000000001e-05 2024-09-13T09:14:26.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.775199+0000", 2024-09-13T09:14:26.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.92148457100000003 2024-09-13T09:14:26.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.775330+0000", 2024-09-13T09:14:26.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013118200000000001 2024-09-13T09:14:26.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:14:26.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.775498+0000", 2024-09-13T09:14:26.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016740200000000001 2024-09-13T09:14:26.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:26.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.777064+0000", 2024-09-13T09:14:26.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015657469999999999 2024-09-13T09:14:26.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:26.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778397+0000", 2024-09-13T09:14:26.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00133325 2024-09-13T09:14:26.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:26.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778426+0000", 2024-09-13T09:14:26.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9612e-05 2024-09-13T09:14:26.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778447+0000", 2024-09-13T09:14:26.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0494999999999999e-05 2024-09-13T09:14:26.984 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.984 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.984 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.984 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:30143 3.9 3:93c2f121:::rbd_data.112326fb43d3.0000000000000003:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-13T09:14:26.985 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:14:18.853590+0000", 2024-09-13T09:14:26.985 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.0383538179999992, 2024-09-13T09:14:26.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.92495849799999996, 2024-09-13T09:14:26.985 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:26.985 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:26.985 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:26.985 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:26.985 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:26.985 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 30143 2024-09-13T09:14:26.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.985 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:26.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:26.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853590+0000", 2024-09-13T09:14:26.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:26.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853590+0000", 2024-09-13T09:14:26.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:26.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:26.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853589+0000", 2024-09-13T09:14:26.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:14:26.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:26.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853595+0000", 2024-09-13T09:14:26.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:14:26.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:26.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853616+0000", 2024-09-13T09:14:26.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1313000000000001e-05 2024-09-13T09:14:26.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:26.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853630+0000", 2024-09-13T09:14:26.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3370000000000001e-05 2024-09-13T09:14:26.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853801+0000", 2024-09-13T09:14:26.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00017078899999999999 2024-09-13T09:14:26.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:26.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853823+0000", 2024-09-13T09:14:26.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2557000000000001e-05 2024-09-13T09:14:26.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:26.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.776396+0000", 2024-09-13T09:14:26.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.92257259800000002 2024-09-13T09:14:26.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:26.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.776478+0000", 2024-09-13T09:14:26.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2528999999999995e-05 2024-09-13T09:14:26.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:14:26.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.776559+0000", 2024-09-13T09:14:26.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1043000000000003e-05 2024-09-13T09:14:26.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:26.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778313+0000", 2024-09-13T09:14:26.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0017531440000000001 2024-09-13T09:14:26.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:26.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778479+0000", 2024-09-13T09:14:26.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016687000000000001 2024-09-13T09:14:26.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:26.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778508+0000", 2024-09-13T09:14:26.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8311999999999999e-05 2024-09-13T09:14:26.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:26.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:26.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:26.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778549+0000", 2024-09-13T09:14:26.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0907e-05 2024-09-13T09:14:26.991 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.991 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.991 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.991 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:26.991 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:26.991 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:26.992 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:14:26.993 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:26.992+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:27.093 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:27.092+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:27.105 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:27.105 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:27.105 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:27.105 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:27.114 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:14:27.142 INFO:teuthology.orchestra.run.smithi129.stdout:3.f 2024-09-13T09:14:27.143 INFO:teuthology.orchestra.run.smithi129.stdout:3.e 2024-09-13T09:14:27.143 INFO:teuthology.orchestra.run.smithi129.stdout:3.c 2024-09-13T09:14:27.143 INFO:teuthology.orchestra.run.smithi129.stdout:3.2 2024-09-13T09:14:27.143 INFO:teuthology.orchestra.run.smithi129.stdout:3.1d 2024-09-13T09:14:27.143 INFO:teuthology.orchestra.run.smithi129.stdout:3.17 2024-09-13T09:14:27.143 INFO:teuthology.orchestra.run.smithi129.stdout:3.16 2024-09-13T09:14:27.143 INFO:teuthology.orchestra.run.smithi129.stdout:3.5 2024-09-13T09:14:27.143 INFO:teuthology.orchestra.run.smithi129.stdout:3.13 2024-09-13T09:14:27.143 INFO:teuthology.orchestra.run.smithi129.stdout:3.10 2024-09-13T09:14:27.143 INFO:teuthology.orchestra.run.smithi129.stdout:3.1c 2024-09-13T09:14:27.143 INFO:teuthology.orchestra.run.smithi129.stdout:3.0 2024-09-13T09:14:27.144 INFO:teuthology.orchestra.run.smithi129.stdout:2.7 2024-09-13T09:14:27.144 INFO:teuthology.orchestra.run.smithi129.stdout:2.3 2024-09-13T09:14:27.144 INFO:teuthology.orchestra.run.smithi129.stdout:2.2 2024-09-13T09:14:27.198 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:27.197+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:27.261 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:27.261 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:27.261 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:27.261 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:27.261 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:27.270 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:14:27.295 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:27.295+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:27.397 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:27.395+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:27.418 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:27.418 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:27.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:27.418 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:27.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.418 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29717 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [watch reconnect cookie 140383290562704 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:14:27.418 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:27.419 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.664574231, 2024-09-13T09:14:27.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.006472936, 2024-09-13T09:14:27.419 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.419 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:27.419 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:27.419 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:27.419 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:27.419 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29717 2024-09-13T09:14:27.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.419 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:27.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:27.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751547+0000", 2024-09-13T09:14:27.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:27.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751552+0000", 2024-09-13T09:14:27.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:14:27.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751564+0000", 2024-09-13T09:14:27.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1793e-05 2024-09-13T09:14:27.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751571+0000", 2024-09-13T09:14:27.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2660000000000004e-06 2024-09-13T09:14:27.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753102+0000", 2024-09-13T09:14:27.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001531085 2024-09-13T09:14:27.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753115+0000", 2024-09-13T09:14:27.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3039e-05 2024-09-13T09:14:27.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.752858+0000", 2024-09-13T09:14:27.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.999742715 2024-09-13T09:14:27.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:27.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.753010+0000", 2024-09-13T09:14:27.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015227799999999999 2024-09-13T09:14:27.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.424 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755125+0000", 2024-09-13T09:14:27.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021150029999999998 2024-09-13T09:14:27.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-09-13T09:14:27.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755235+0000", 2024-09-13T09:14:27.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000109775 2024-09-13T09:14:27.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756059+0000", 2024-09-13T09:14:27.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00082370900000000005 2024-09-13T09:14:27.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756154+0000", 2024-09-13T09:14:27.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4945999999999995e-05 2024-09-13T09:14:27.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758021+0000", 2024-09-13T09:14:27.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001867446 2024-09-13T09:14:27.426 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.426 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.426 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.427 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29718 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:14:27.427 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:27.427 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.664541366, 2024-09-13T09:14:27.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0067812169999999, 2024-09-13T09:14:27.427 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.427 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:27.427 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:27.427 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:27.427 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:27.427 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29718 2024-09-13T09:14:27.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.427 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:27.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:27.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:27.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751585+0000", 2024-09-13T09:14:27.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2050000000000004e-06 2024-09-13T09:14:27.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751591+0000", 2024-09-13T09:14:27.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.37e-06 2024-09-13T09:14:27.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751597+0000", 2024-09-13T09:14:27.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1190000000000002e-06 2024-09-13T09:14:27.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.430 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.430 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753150+0000", 2024-09-13T09:14:27.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001552351 2024-09-13T09:14:27.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.430 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.430 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753164+0000", 2024-09-13T09:14:27.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4076e-05 2024-09-13T09:14:27.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.430 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.430 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.753307+0000", 2024-09-13T09:14:27.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0001434229999999 2024-09-13T09:14:27.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:27.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.753365+0000", 2024-09-13T09:14:27.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8014999999999999e-05 2024-09-13T09:14:27.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755501+0000", 2024-09-13T09:14:27.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021357260000000001 2024-09-13T09:14:27.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-09-13T09:14:27.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755558+0000", 2024-09-13T09:14:27.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7516000000000003e-05 2024-09-13T09:14:27.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756684+0000", 2024-09-13T09:14:27.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011259550000000001 2024-09-13T09:14:27.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756766+0000", 2024-09-13T09:14:27.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1755999999999994e-05 2024-09-13T09:14:27.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:27.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.757147+0000", 2024-09-13T09:14:27.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000381107 2024-09-13T09:14:27.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:27.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.757890+0000", 2024-09-13T09:14:27.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00074321499999999998 2024-09-13T09:14:27.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:27.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758178+0000", 2024-09-13T09:14:27.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00028755000000000002 2024-09-13T09:14:27.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:27.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758230+0000", 2024-09-13T09:14:27.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2317000000000001e-05 2024-09-13T09:14:27.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758362+0000", 2024-09-13T09:14:27.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013204100000000001 2024-09-13T09:14:27.434 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.434 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.434 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGPush(3.d 123/122 [PushOp(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29, version: 99'289, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29@99'289, size: 12582912, copy_subset: [0~12582912], clone_subset: {}, snapset: 29=[]:{29=[29]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:14:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.774123+0000", 2024-09-13T09:14:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.641999701, 2024-09-13T09:14:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051988820000000002, 2024-09-13T09:14:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.774123+0000", 2024-09-13T09:14:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.774123+0000", 2024-09-13T09:14:27.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.774122+0000", 2024-09-13T09:14:27.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777696+0000", 2024-09-13T09:14:27.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003573676 2024-09-13T09:14:27.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777708+0000", 2024-09-13T09:14:27.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2349e-05 2024-09-13T09:14:27.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777718+0000", 2024-09-13T09:14:27.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0013999999999999e-05 2024-09-13T09:14:27.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777761+0000", 2024-09-13T09:14:27.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3167999999999997e-05 2024-09-13T09:14:27.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.438 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.438 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777771+0000", 2024-09-13T09:14:27.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3600000000000002e-06 2024-09-13T09:14:27.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.438 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.438 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.779322+0000", 2024-09-13T09:14:27.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015509740000000001 2024-09-13T09:14:27.438 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.438 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.439 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.439 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29749 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [watch reconnect cookie 140383086477584 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-13T09:14:27.439 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:27.439 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.616702507999999, 2024-09-13T09:14:27.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037671060000000001, 2024-09-13T09:14:27.439 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.439 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:27.439 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:27.439 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:27.439 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:27.439 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29749 2024-09-13T09:14:27.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.440 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:27.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:27.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:27.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799422+0000", 2024-09-13T09:14:27.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:14:27.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799429+0000", 2024-09-13T09:14:27.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4670000000000001e-06 2024-09-13T09:14:27.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.442 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.442 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799434+0000", 2024-09-13T09:14:27.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0520000000000001e-06 2024-09-13T09:14:27.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.442 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.442 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.801379+0000", 2024-09-13T09:14:27.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019449599999999999 2024-09-13T09:14:27.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.442 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.442 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.801426+0000", 2024-09-13T09:14:27.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7391000000000001e-05 2024-09-13T09:14:27.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.443 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.803187+0000", 2024-09-13T09:14:27.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0017609870000000001 2024-09-13T09:14:27.443 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.443 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.443 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.443 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.3.0:2661 3.d e124/122 3:b5ba65cb:::rbd_data.1123fa58739e.0000000000000006:12 v 124'356, mlcod=123'351)", 2024-09-13T09:14:27.443 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:27.443 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.667975062, 2024-09-13T09:14:27.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034693089999999998, 2024-09-13T09:14:27.444 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.444 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:27.444 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.444 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:27.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.444 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:27.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:27.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748155+0000", 2024-09-13T09:14:27.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3769999999999996e-06 2024-09-13T09:14:27.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748163+0000", 2024-09-13T09:14:27.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0120000000000005e-06 2024-09-13T09:14:27.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748169+0000", 2024-09-13T09:14:27.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0610000000000003e-06 2024-09-13T09:14:27.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748695+0000", 2024-09-13T09:14:27.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052687199999999995 2024-09-13T09:14:27.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748716+0000", 2024-09-13T09:14:27.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0126e-05 2024-09-13T09:14:27.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:27.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.751587+0000", 2024-09-13T09:14:27.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028710839999999999 2024-09-13T09:14:27.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.751617+0000", 2024-09-13T09:14:27.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0096000000000001e-05 2024-09-13T09:14:27.447 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.448 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.448 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.448 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGPush(3.9 124/123 [PushOp(3:94c9dbee:::rbd_data.112339c5624d.0000000000000000:head, version: 88'1057, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:94c9dbee:::rbd_data.112339c5624d.0000000000000000:head@88'1057, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:0, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:14:27.448 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:17.795705+0000", 2024-09-13T09:14:27.448 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.6204179029999999, 2024-09-13T09:14:27.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0038339609999999999, 2024-09-13T09:14:27.448 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.448 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:27.448 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795705+0000", 2024-09-13T09:14:27.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795705+0000", 2024-09-13T09:14:27.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795704+0000", 2024-09-13T09:14:27.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.450 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795715+0000", 2024-09-13T09:14:27.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1025e-05 2024-09-13T09:14:27.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.450 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795727+0000", 2024-09-13T09:14:27.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2031000000000001e-05 2024-09-13T09:14:27.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.450 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795736+0000", 2024-09-13T09:14:27.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2220000000000008e-06 2024-09-13T09:14:27.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795816+0000", 2024-09-13T09:14:27.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9338000000000005e-05 2024-09-13T09:14:27.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795829+0000", 2024-09-13T09:14:27.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3677e-05 2024-09-13T09:14:27.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.799539+0000", 2024-09-13T09:14:27.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037092169999999999 2024-09-13T09:14:27.452 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.452 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.452 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.452 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.3.0:2692 3.d e125/122 3:b6fe9194:::rbd_data.112326fb43d3.000000000000002a:11 v 125'358, mlcod=124'354)", 2024-09-13T09:14:27.452 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:18.770322+0000", 2024-09-13T09:14:27.452 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.6458006510000001, 2024-09-13T09:14:27.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047215089999999996, 2024-09-13T09:14:27.452 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.452 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:27.452 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770322+0000", 2024-09-13T09:14:27.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770322+0000", 2024-09-13T09:14:27.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770321+0000", 2024-09-13T09:14:27.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:27.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770333+0000", 2024-09-13T09:14:27.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2014000000000001e-05 2024-09-13T09:14:27.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770346+0000", 2024-09-13T09:14:27.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2771e-05 2024-09-13T09:14:27.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770354+0000", 2024-09-13T09:14:27.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1249999999999993e-06 2024-09-13T09:14:27.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.455 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.455 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770459+0000", 2024-09-13T09:14:27.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010558200000000001 2024-09-13T09:14:27.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.455 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.455 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770483+0000", 2024-09-13T09:14:27.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3125e-05 2024-09-13T09:14:27.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.455 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:27.455 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.775015+0000", 2024-09-13T09:14:27.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045322449999999999 2024-09-13T09:14:27.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.456 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.775043+0000", 2024-09-13T09:14:27.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8532999999999999e-05 2024-09-13T09:14:27.456 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.456 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.456 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.456 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2445 3.9 e126/125 3:91fd9b46:::rbd_data.112326fb43d3.000000000000010f:11 v 126'1140, mlcod=126'1134)", 2024-09-13T09:14:27.456 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:27.456 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 7.6339443060000001, 2024-09-13T09:14:27.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0036771899999999999, 2024-09-13T09:14:27.457 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.457 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:27.457 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:27.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:27.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:27.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782184+0000", 2024-09-13T09:14:27.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5729999999999996e-06 2024-09-13T09:14:27.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782196+0000", 2024-09-13T09:14:27.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1817000000000001e-05 2024-09-13T09:14:27.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782202+0000", 2024-09-13T09:14:27.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:14:27.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.783326+0000", 2024-09-13T09:14:27.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011234509999999999 2024-09-13T09:14:27.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.783343+0000", 2024-09-13T09:14:27.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7504e-05 2024-09-13T09:14:27.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.460 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:27.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.785708+0000", 2024-09-13T09:14:27.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023648829999999999 2024-09-13T09:14:27.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.460 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.785855+0000", 2024-09-13T09:14:27.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014743199999999999 2024-09-13T09:14:27.460 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.460 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.460 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.461 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.1.0:2475 3.4 e126/125 3:2095a19f:::rbd_data.1123b428d19a.000000000000001b:16 v 126'630, mlcod=126'626)", 2024-09-13T09:14:27.461 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.784119+0000", 2024-09-13T09:14:27.461 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 7.6320038009999998, 2024-09-13T09:14:27.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033969479999999999, 2024-09-13T09:14:27.461 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.461 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:27.461 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784119+0000", 2024-09-13T09:14:27.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784119+0000", 2024-09-13T09:14:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784118+0000", 2024-09-13T09:14:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784126+0000", 2024-09-13T09:14:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6490000000000008e-06 2024-09-13T09:14:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784136+0000", 2024-09-13T09:14:27.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9699999999999994e-06 2024-09-13T09:14:27.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.463 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784141+0000", 2024-09-13T09:14:27.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8730000000000004e-06 2024-09-13T09:14:27.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.463 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784219+0000", 2024-09-13T09:14:27.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7650000000000004e-05 2024-09-13T09:14:27.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.463 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784239+0000", 2024-09-13T09:14:27.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9794999999999999e-05 2024-09-13T09:14:27.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.464 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:27.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.787481+0000", 2024-09-13T09:14:27.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0032426320000000001 2024-09-13T09:14:27.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.464 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.787516+0000", 2024-09-13T09:14:27.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4143000000000002e-05 2024-09-13T09:14:27.464 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.464 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.465 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.465 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2447 3.9 e126/125 3:9541a82a:::rbd_data.1123b428d19a.000000000000000a:16 v 126'1144, mlcod=126'1140)", 2024-09-13T09:14:27.465 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.793072+0000", 2024-09-13T09:14:27.465 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 7.6230505849999997, 2024-09-13T09:14:27.465 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046128150000000001, 2024-09-13T09:14:27.465 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.465 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:27.465 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793072+0000", 2024-09-13T09:14:27.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.466 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793072+0000", 2024-09-13T09:14:27.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.466 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793071+0000", 2024-09-13T09:14:27.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793077+0000", 2024-09-13T09:14:27.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1080000000000001e-06 2024-09-13T09:14:27.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.467 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793085+0000", 2024-09-13T09:14:27.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7309999999999999e-06 2024-09-13T09:14:27.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.467 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793091+0000", 2024-09-13T09:14:27.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6300000000000003e-06 2024-09-13T09:14:27.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.468 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.468 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793160+0000", 2024-09-13T09:14:27.468 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8681000000000002e-05 2024-09-13T09:14:27.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.468 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.468 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793178+0000", 2024-09-13T09:14:27.468 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8861000000000001e-05 2024-09-13T09:14:27.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.468 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:27.468 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.797650+0000", 2024-09-13T09:14:27.468 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004471193 2024-09-13T09:14:27.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.469 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.469 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.797685+0000", 2024-09-13T09:14:27.469 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5042000000000001e-05 2024-09-13T09:14:27.469 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.469 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.469 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.469 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop_reply(osd.2.0:98 3.18 e126/125 ondisk, result = 0)", 2024-09-13T09:14:27.469 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.794045+0000", 2024-09-13T09:14:27.469 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 7.6220777579999996, 2024-09-13T09:14:27.469 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050958949999999996, 2024-09-13T09:14:27.470 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.470 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:27.470 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794045+0000", 2024-09-13T09:14:27.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794045+0000", 2024-09-13T09:14:27.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794044+0000", 2024-09-13T09:14:27.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:27.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794049+0000", 2024-09-13T09:14:27.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8600000000000001e-06 2024-09-13T09:14:27.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794060+0000", 2024-09-13T09:14:27.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0801e-05 2024-09-13T09:14:27.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.472 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.472 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.472 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794068+0000", 2024-09-13T09:14:27.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1550000000000007e-06 2024-09-13T09:14:27.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.472 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.472 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.472 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.799120+0000", 2024-09-13T09:14:27.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050525040000000002 2024-09-13T09:14:27.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.472 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.472 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.472 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.799128+0000", 2024-09-13T09:14:27.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2819999999999996e-06 2024-09-13T09:14:27.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.473 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.473 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.473 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.799141+0000", 2024-09-13T09:14:27.473 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2098e-05 2024-09-13T09:14:27.473 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.473 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.473 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.473 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.473 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2450 3.9 e126/125 3:95b1a540:::rbd_data.1123e3c0a1e8.0000000000000030:18 v 126'1148, mlcod=126'1144)", 2024-09-13T09:14:27.473 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:27.473 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 7.614494122, 2024-09-13T09:14:27.473 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0035455719999999999, 2024-09-13T09:14:27.474 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.474 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:27.474 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.474 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:27.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.474 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:27.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801634+0000", 2024-09-13T09:14:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9539999999999998e-06 2024-09-13T09:14:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801646+0000", 2024-09-13T09:14:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1795e-05 2024-09-13T09:14:27.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801653+0000", 2024-09-13T09:14:27.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7259999999999996e-06 2024-09-13T09:14:27.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.803017+0000", 2024-09-13T09:14:27.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001363997 2024-09-13T09:14:27.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.803038+0000", 2024-09-13T09:14:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1294e-05 2024-09-13T09:14:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.805138+0000", 2024-09-13T09:14:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020996259999999998 2024-09-13T09:14:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.805174+0000", 2024-09-13T09:14:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5970999999999998e-05 2024-09-13T09:14:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2452 3.9 e126/125 3:9030a72f:::rbd_data.1123af309369.0000000000000023:1f v 126'1152, mlcod=126'1148)", 2024-09-13T09:14:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.812010+0000", 2024-09-13T09:14:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 7.6041127719999997, 2024-09-13T09:14:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033163540000000001, 2024-09-13T09:14:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.478 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812010+0000", 2024-09-13T09:14:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812010+0000", 2024-09-13T09:14:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812009+0000", 2024-09-13T09:14:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812093+0000", 2024-09-13T09:14:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4022000000000003e-05 2024-09-13T09:14:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812121+0000", 2024-09-13T09:14:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.826e-05 2024-09-13T09:14:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812128+0000", 2024-09-13T09:14:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6270000000000002e-06 2024-09-13T09:14:27.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812170+0000", 2024-09-13T09:14:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1785000000000001e-05 2024-09-13T09:14:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812202+0000", 2024-09-13T09:14:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2505999999999997e-05 2024-09-13T09:14:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.815226+0000", 2024-09-13T09:14:27.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003024185 2024-09-13T09:14:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.815326+0000", 2024-09-13T09:14:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9637999999999997e-05 2024-09-13T09:14:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2453 3.9 e126/125 3:94dbe637:::rbd_data.1123af309369.00000000000001dc:1f v 126'1154, mlcod=126'1148)", 2024-09-13T09:14:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.812843+0000", 2024-09-13T09:14:27.482 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 7.6032799119999996, 2024-09-13T09:14:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034843890000000001, 2024-09-13T09:14:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812843+0000", 2024-09-13T09:14:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812843+0000", 2024-09-13T09:14:27.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812842+0000", 2024-09-13T09:14:27.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:27.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812848+0000", 2024-09-13T09:14:27.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4189999999999999e-06 2024-09-13T09:14:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812859+0000", 2024-09-13T09:14:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0778000000000001e-05 2024-09-13T09:14:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812866+0000", 2024-09-13T09:14:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7619999999999998e-06 2024-09-13T09:14:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.813231+0000", 2024-09-13T09:14:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00036497299999999999 2024-09-13T09:14:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.813251+0000", 2024-09-13T09:14:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0511999999999999e-05 2024-09-13T09:14:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.816298+0000", 2024-09-13T09:14:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003046801 2024-09-13T09:14:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.816327+0000", 2024-09-13T09:14:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8943000000000001e-05 2024-09-13T09:14:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2455 3.9 e126/125 3:95f6e286:::rbd_data.1123af309369.00000000000000ba:1f v 126'1158, mlcod=126'1152)", 2024-09-13T09:14:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.817505+0000", 2024-09-13T09:14:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 7.5986174349999995, 2024-09-13T09:14:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034876989999999999, 2024-09-13T09:14:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.487 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:27.488 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817505+0000", 2024-09-13T09:14:27.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817505+0000", 2024-09-13T09:14:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817504+0000", 2024-09-13T09:14:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817512+0000", 2024-09-13T09:14:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1389999999999995e-06 2024-09-13T09:14:27.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817532+0000", 2024-09-13T09:14:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9089e-05 2024-09-13T09:14:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817538+0000", 2024-09-13T09:14:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5799999999999997e-06 2024-09-13T09:14:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.818301+0000", 2024-09-13T09:14:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00076256200000000005 2024-09-13T09:14:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.818318+0000", 2024-09-13T09:14:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7428999999999999e-05 2024-09-13T09:14:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820959+0000", 2024-09-13T09:14:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026407129999999998 2024-09-13T09:14:27.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820993+0000", 2024-09-13T09:14:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3912000000000003e-05 2024-09-13T09:14:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop_reply(osd.2.0:107 3.18 e126/125 ondisk, result = 0)", 2024-09-13T09:14:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.820584+0000", 2024-09-13T09:14:27.492 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 7.595538844, 2024-09-13T09:14:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045400709999999997, 2024-09-13T09:14:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820584+0000", 2024-09-13T09:14:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820584+0000", 2024-09-13T09:14:27.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820583+0000", 2024-09-13T09:14:27.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:27.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820588+0000", 2024-09-13T09:14:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7029999999999997e-06 2024-09-13T09:14:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820598+0000", 2024-09-13T09:14:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0179e-05 2024-09-13T09:14:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820604+0000", 2024-09-13T09:14:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:14:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825098+0000", 2024-09-13T09:14:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044948009999999997 2024-09-13T09:14:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825108+0000", 2024-09-13T09:14:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3759999999999997e-06 2024-09-13T09:14:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825124+0000", 2024-09-13T09:14:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5950000000000001e-05 2024-09-13T09:14:27.496 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2456 3.9 e126/125 3:93610890:::rbd_data.1123af309369.00000000000003af:1f v 126'1160, mlcod=126'1156)", 2024-09-13T09:14:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.821541+0000", 2024-09-13T09:14:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 7.5945814629999999, 2024-09-13T09:14:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003753915, 2024-09-13T09:14:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821541+0000", 2024-09-13T09:14:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821541+0000", 2024-09-13T09:14:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821540+0000", 2024-09-13T09:14:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821546+0000", 2024-09-13T09:14:27.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7370000000000003e-06 2024-09-13T09:14:27.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821604+0000", 2024-09-13T09:14:27.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8072000000000002e-05 2024-09-13T09:14:27.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821616+0000", 2024-09-13T09:14:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1956999999999999e-05 2024-09-13T09:14:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821711+0000", 2024-09-13T09:14:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5079000000000002e-05 2024-09-13T09:14:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821731+0000", 2024-09-13T09:14:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9661999999999999e-05 2024-09-13T09:14:27.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825263+0000", 2024-09-13T09:14:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0035322399999999999 2024-09-13T09:14:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825295+0000", 2024-09-13T09:14:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1813000000000002e-05 2024-09-13T09:14:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2457 3.9 e126/125 3:9313c095:::rbd_data.1123af309369.00000000000000be:1f v 126'1162, mlcod=126'1156)", 2024-09-13T09:14:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.822601+0000", 2024-09-13T09:14:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 7.5935219050000002, 2024-09-13T09:14:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003671147, 2024-09-13T09:14:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822601+0000", 2024-09-13T09:14:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822601+0000", 2024-09-13T09:14:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822600+0000", 2024-09-13T09:14:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822605+0000", 2024-09-13T09:14:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6979999999999997e-06 2024-09-13T09:14:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822612+0000", 2024-09-13T09:14:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8589999999999999e-06 2024-09-13T09:14:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822617+0000", 2024-09-13T09:14:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:14:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.823075+0000", 2024-09-13T09:14:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00045848500000000001 2024-09-13T09:14:27.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.823094+0000", 2024-09-13T09:14:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8740999999999999e-05 2024-09-13T09:14:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.826239+0000", 2024-09-13T09:14:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0031454709999999999 2024-09-13T09:14:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.826272+0000", 2024-09-13T09:14:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2330999999999999e-05 2024-09-13T09:14:27.505 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.506 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.506 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.506 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:30365 3.7 3:e7191d4c:::rbd_header.1123313a4365:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-13T09:14:27.506 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:22.797253+0000", 2024-09-13T09:14:27.506 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.6188695429999997, 2024-09-13T09:14:27.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0059535409999999997, 2024-09-13T09:14:27.506 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.506 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:27.506 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 30365 2024-09-13T09:14:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797253+0000", 2024-09-13T09:14:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797253+0000", 2024-09-13T09:14:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797251+0000", 2024-09-13T09:14:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-09-13T09:14:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797262+0000", 2024-09-13T09:14:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0993e-05 2024-09-13T09:14:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797281+0000", 2024-09-13T09:14:27.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8216999999999999e-05 2024-09-13T09:14:27.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797290+0000", 2024-09-13T09:14:27.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5230000000000001e-06 2024-09-13T09:14:27.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797339+0000", 2024-09-13T09:14:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8854999999999999e-05 2024-09-13T09:14:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797437+0000", 2024-09-13T09:14:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8105999999999999e-05 2024-09-13T09:14:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:14:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797842+0000", 2024-09-13T09:14:27.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00040497999999999999 2024-09-13T09:14:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.800594+0000", 2024-09-13T09:14:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002752193 2024-09-13T09:14:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.803126+0000", 2024-09-13T09:14:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025320070000000002 2024-09-13T09:14:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:27.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.803177+0000", 2024-09-13T09:14:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0513000000000002e-05 2024-09-13T09:14:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.803206+0000", 2024-09-13T09:14:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9747000000000002e-05 2024-09-13T09:14:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:30611 3.7 3:e0345511:::rbd_data.1123b259dc65.000000000000000b:head [delete] snapc e=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-13T09:14:27.512 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:23.893039+0000", 2024-09-13T09:14:27.513 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.523083594, 2024-09-13T09:14:27.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003465072, 2024-09-13T09:14:27.513 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.513 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:27.513 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:27.513 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:27.513 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:27.513 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 30611 2024-09-13T09:14:27.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.513 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893039+0000", 2024-09-13T09:14:27.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.514 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.514 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893039+0000", 2024-09-13T09:14:27.514 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.514 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.514 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893038+0000", 2024-09-13T09:14:27.514 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:27.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.514 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.514 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893043+0000", 2024-09-13T09:14:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3739999999999997e-06 2024-09-13T09:14:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893055+0000", 2024-09-13T09:14:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2138e-05 2024-09-13T09:14:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893083+0000", 2024-09-13T09:14:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7589999999999998e-05 2024-09-13T09:14:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893158+0000", 2024-09-13T09:14:27.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5318e-05 2024-09-13T09:14:27.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893343+0000", 2024-09-13T09:14:27.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000184747 2024-09-13T09:14:27.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:14:27.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893440+0000", 2024-09-13T09:14:27.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6751999999999998e-05 2024-09-13T09:14:27.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:27.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.896088+0000", 2024-09-13T09:14:27.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026486370000000001 2024-09-13T09:14:27.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:27.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.896422+0000", 2024-09-13T09:14:27.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000333526 2024-09-13T09:14:27.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:27.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.896467+0000", 2024-09-13T09:14:27.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5439000000000003e-05 2024-09-13T09:14:27.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.896504+0000", 2024-09-13T09:14:27.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6547000000000001e-05 2024-09-13T09:14:27.518 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.518 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.518 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.518 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.518 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.518 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:27.520 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:27.517+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:27.520 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:14:27.620 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:27.620+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:27.659 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op export --pgid 3.2 --file /var/log/ceph/exp.3.2.5 2024-09-13T09:14:27.667 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:27.667 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:27.668 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:27.668 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:27.676 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:14:27.720 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:27.718+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:27.782 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:27.783 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:27.783 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:27.783 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:27.783 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:27.792 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:14:27.834 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:27.832+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:27.922 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:27.920+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:27.945 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:27.945 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:27.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:27.945 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:27.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.945 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:27.945 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:27.945 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.413586243, 2024-09-13T09:14:27.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:14:27.945 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.945 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:27.945 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:27.946 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:27.946 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:27.946 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:14:27.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.946 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:27.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:27.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:14:27.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:14:27.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:14:27.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:14:27.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:14:27.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:14:27.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:14:27.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:14:27.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:14:27.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:14:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:14:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:14:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:14:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:14:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:14:27.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:14:27.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:14:27.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:14:27.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:14:27.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:14:27.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:14:27.951 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.951 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.951 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.951 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:27.951 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:27.951 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.413568208, 2024-09-13T09:14:27.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:14:27.951 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.951 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:27.951 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:27.951 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:27.952 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:27.952 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:14:27.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.952 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:27.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:27.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:27.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:14:27.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:14:27.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:14:27.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:14:27.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:14:27.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:14:27.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:14:27.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:14:27.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:14:27.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:14:27.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:14:27.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:14:27.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:14:27.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:14:27.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:14:27.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:14:27.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:14:27.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:14:27.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:14:27.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:14:27.957 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.957 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.957 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.957 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:27.957 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:27.957 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.41355153799999, 2024-09-13T09:14:27.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:14:27.957 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.957 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:27.957 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:27.957 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:27.957 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:14:27.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:14:27.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:27.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:14:27.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:14:27.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:14:27.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:14:27.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:14:27.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:14:27.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:14:27.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:14:27.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:14:27.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:14:27.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:14:27.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:14:27.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:14:27.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:14:27.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:14:27.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:14:27.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:14:27.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:14:27.962 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.962 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.962 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.963 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:27.963 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:27.963 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.413533906, 2024-09-13T09:14:27.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:14:27.963 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.963 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:27.963 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:27.963 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:27.963 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:27.963 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:14:27.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.963 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:27.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:27.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:27.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:14:27.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:14:27.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:14:27.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:14:27.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:14:27.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:14:27.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:14:27.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:14:27.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:14:27.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:14:27.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:14:27.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:14:27.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:14:27.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:14:27.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:14:27.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:14:27.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:14:27.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:14:27.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:14:27.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:14:27.968 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.968 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.968 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.968 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:27.969 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:27.969 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.413487601, 2024-09-13T09:14:27.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:14:27.969 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.969 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:27.969 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:27.969 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:27.969 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:27.969 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:14:27.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.969 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:27.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:27.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:14:27.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:14:27.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:14:27.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:14:27.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:14:27.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:14:27.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:14:27.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:14:27.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:14:27.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:14:27.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:14:27.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:14:27.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:14:27.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:14:27.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:14:27.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:14:27.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:14:27.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:14:27.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:14:27.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:14:27.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:14:27.974 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.974 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.975 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.975 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:27.975 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:27.975 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.41346230000001, 2024-09-13T09:14:27.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:14:27.975 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.975 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:27.975 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:27.975 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:14:27.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:14:27.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:14:27.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:14:27.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:14:27.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:14:27.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:14:27.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:14:27.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:14:27.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:14:27.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:14:27.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:14:27.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:14:27.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:14:27.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:14:27.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:14:27.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:14:27.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:14:27.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:14:27.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:14:27.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:14:27.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.981 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.981 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.981 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:27.981 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:27.981 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.413444071, 2024-09-13T09:14:27.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:14:27.981 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.981 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:27.981 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:27.982 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:27.982 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:27.982 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:14:27.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.982 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:27.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:27.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:27.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:14:27.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:14:27.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:14:27.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:14:27.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:14:27.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:14:27.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:14:27.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:14:27.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:14:27.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:14:27.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:14:27.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:14:27.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:14:27.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:14:27.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:14:27.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:14:27.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:14:27.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:14:27.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:14:27.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:14:27.987 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.987 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.987 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.987 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:27.987 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:27.987 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.413396752, 2024-09-13T09:14:27.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:14:27.987 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.988 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:27.988 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:27.988 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:27.988 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:27.988 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:14:27.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.988 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:27.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:27.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:27.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:27.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:14:27.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:14:27.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:14:27.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:14:27.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:14:27.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:14:27.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:14:27.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:14:27.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:14:27.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:14:27.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:14:27.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:14:27.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:14:27.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:14:27.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:14:27.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:14:27.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:14:27.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:14:27.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:14:27.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:14:27.993 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.993 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.993 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.993 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:27.993 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:27.994 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.412882514, 2024-09-13T09:14:27.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:14:27.994 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:27.994 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:27.994 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:27.994 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:27.994 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:27.994 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:14:27.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.994 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:27.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:27.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:27.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:27.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:27.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:27.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:27.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:14:27.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:27.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:27.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:14:27.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:14:27.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:27.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:14:27.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:14:27.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:27.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:14:27.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:14:27.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:14:27.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:14:27.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:14:27.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:14:27.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:14:27.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:14:27.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:27.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:14:27.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:14:27.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:27.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:14:27.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:14:27.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:27.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:14:27.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:14:27.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:27.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:14:27.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:14:27.999 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.999 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:27.999 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:27.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:27.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:27.999 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:27.999 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:27.999 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.412847984, 2024-09-13T09:14:27.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:14:28.000 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.000 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.000 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.000 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.000 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.000 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:14:28.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.000 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:28.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:28.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:28.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:14:28.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:14:28.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:14:28.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:14:28.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:14:28.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:14:28.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:14:28.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:14:28.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:14:28.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:14:28.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:14:28.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:14:28.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:14:28.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:14:28.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:14:28.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:14:28.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:14:28.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:14:28.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:14:28.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:14:28.005 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.005 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.005 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.005 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.005 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:28.006 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.412830077, 2024-09-13T09:14:28.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:14:28.006 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.006 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.006 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.006 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.006 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.006 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:14:28.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.006 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:28.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:28.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:28.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:14:28.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:14:28.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:14:28.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:14:28.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:14:28.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:14:28.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:14:28.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:14:28.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:14:28.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:14:28.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:14:28.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:14:28.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:14:28.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:14:28.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:14:28.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:14:28.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:14:28.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:14:28.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:14:28.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:14:28.011 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.011 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.011 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.012 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.012 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:28.012 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.403703229, 2024-09-13T09:14:28.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:14:28.012 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.012 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.012 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.012 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.012 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.012 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:14:28.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.012 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:28.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:28.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:28.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:14:28.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:14:28.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:14:28.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:14:28.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:14:28.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:14:28.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:14:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:14:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:14:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:14:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:14:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:14:28.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:14:28.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:14:28.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:14:28.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:14:28.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:14:28.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:14:28.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:14:28.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:14:28.017 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.017 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.017 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.017 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.017 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:28.017 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.403651649, 2024-09-13T09:14:28.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:14:28.018 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.018 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.018 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.018 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.018 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.018 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:14:28.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.018 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.018 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.018 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:28.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:28.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:14:28.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:14:28.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:14:28.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:14:28.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:14:28.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:14:28.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:14:28.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:14:28.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:14:28.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:14:28.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:14:28.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:14:28.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:14:28.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:14:28.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:14:28.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:14:28.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:14:28.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:14:28.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:14:28.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:14:28.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:14:28.023 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.023 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.023 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.024 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.024 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:28.024 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.403635373, 2024-09-13T09:14:28.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:14:28.024 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.024 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.024 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.024 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.024 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.024 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:14:28.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.024 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:28.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:28.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:14:28.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:14:28.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:14:28.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:14:28.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:14:28.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:14:28.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:14:28.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:14:28.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:14:28.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:14:28.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:14:28.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:14:28.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:14:28.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:14:28.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:14:28.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:14:28.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:14:28.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:14:28.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:14:28.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:14:28.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:14:28.029 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.029 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.029 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.030 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.030 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:28.030 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.403619599, 2024-09-13T09:14:28.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:14:28.030 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.030 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.030 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.030 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.030 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.030 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:14:28.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.030 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:28.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:28.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:14:28.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:14:28.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:14:28.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:14:28.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:14:28.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:14:28.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:14:28.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:14:28.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:14:28.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:14:28.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:14:28.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:14:28.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:14:28.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:14:28.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:14:28.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:14:28.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:14:28.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:14:28.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:14:28.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:14:28.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:14:28.035 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.035 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.035 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.036 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.036 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:28.036 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.403604724, 2024-09-13T09:14:28.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:14:28.036 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.036 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.036 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.036 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.036 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.036 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:14:28.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.036 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:28.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:28.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:28.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:14:28.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:14:28.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:14:28.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:14:28.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:14:28.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:14:28.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:14:28.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:14:28.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:14:28.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:14:28.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:14:28.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:14:28.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:14:28.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:14:28.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:14:28.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:14:28.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:14:28.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:14:28.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:14:28.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:14:28.041 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.041 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.041 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.041 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.042 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:28.042 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.403587262, 2024-09-13T09:14:28.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:14:28.042 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.042 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.042 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.042 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.042 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.042 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:14:28.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.042 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:28.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:28.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:14:28.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:14:28.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:14:28.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:14:28.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:14:28.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:14:28.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:14:28.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:14:28.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:14:28.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:14:28.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:14:28.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:14:28.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:14:28.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:14:28.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:14:28.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:14:28.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:14:28.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:14:28.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:14:28.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:14:28.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:14:28.047 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.047 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.047 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.047 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.047 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:28.047 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.40198663, 2024-09-13T09:14:28.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:14:28.048 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.048 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.048 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.048 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.048 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.048 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:14:28.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.048 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:28.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:28.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:14:28.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:14:28.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:14:28.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:14:28.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:14:28.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:14:28.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:14:28.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:14:28.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:14:28.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:14:28.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:14:28.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:14:28.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:14:28.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:14:28.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:14:28.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:14:28.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:14:28.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:14:28.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:14:28.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:14:28.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:14:28.053 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.053 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.053 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.053 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.053 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:28.053 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.401946579, 2024-09-13T09:14:28.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:14:28.053 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.053 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.053 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.053 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.053 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.053 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:14:28.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.054 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:28.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:28.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:14:28.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:14:28.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:14:28.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:14:28.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:14:28.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:14:28.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:14:28.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:14:28.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:14:28.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:14:28.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:14:28.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:14:28.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:14:28.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:14:28.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:14:28.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:14:28.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:14:28.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:14:28.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:14:28.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:14:28.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:14:28.058 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.058 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.058 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.059 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.059 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:28.059 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.401902816, 2024-09-13T09:14:28.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:14:28.059 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.059 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.059 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.059 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.059 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.059 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:14:28.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.059 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:28.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:28.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:28.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.061 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:14:28.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:14:28.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.061 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:14:28.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:14:28.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:14:28.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:14:28.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:14:28.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:14:28.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:14:28.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:14:28.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:14:28.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:14:28.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:14:28.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:14:28.064 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.064 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.064 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.064 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.064 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.064 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:28.066 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:14:28.066 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:28.064+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:28.167 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:28.166+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:28.214 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:28.214 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:28.214 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:28.214 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:28.222 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:14:28.268 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:28.266+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:28.328 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:28.328 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:28.337 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:14:28.369 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:28.368+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:28.444 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:28.445 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:28.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:28.445 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:28.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.445 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.445 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:28.445 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.939110481, 2024-09-13T09:14:28.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:14:28.445 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.445 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.445 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.445 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.446 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.446 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:14:28.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.446 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:28.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:28.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:14:28.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:14:28.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:14:28.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:14:28.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:14:28.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:14:28.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:14:28.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:14:28.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:14:28.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:14:28.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:14:28.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:14:28.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:14:28.449 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.449 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.449 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.449 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.450 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:28.450 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.939082797, 2024-09-13T09:14:28.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:14:28.450 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.450 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.450 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.450 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.450 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.450 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:14:28.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.450 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.450 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:28.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:28.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:28.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.452 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:14:28.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:14:28.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.452 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:14:28.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:14:28.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.452 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:14:28.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:14:28.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:14:28.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:14:28.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:14:28.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:14:28.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:14:28.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:14:28.454 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.454 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.454 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.454 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.454 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:28.454 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.939058456, 2024-09-13T09:14:28.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:14:28.454 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.454 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.454 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.455 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.455 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.455 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:14:28.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.455 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.455 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.455 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:28.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.455 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.455 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:28.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.456 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:14:28.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.456 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:14:28.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:14:28.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.456 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.456 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:14:28.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:14:28.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:14:28.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:14:28.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:14:28.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:14:28.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:14:28.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:14:28.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:14:28.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:14:28.458 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.458 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.458 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.458 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.459 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:28.459 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.939042136, 2024-09-13T09:14:28.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:14:28.459 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.459 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.459 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.459 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.459 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.459 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:14:28.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.459 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:28.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.460 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:28.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.460 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:28.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:14:28.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:28.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:14:28.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:14:28.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:14:28.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:14:28.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:14:28.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:14:28.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:14:28.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:14:28.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.462 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.462 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:14:28.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:14:28.463 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.463 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.463 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.463 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.463 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:28.463 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.93902572099999, 2024-09-13T09:14:28.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:14:28.463 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.463 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.463 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.463 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.464 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.464 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:14:28.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.464 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.464 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:28.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.464 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:28.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:28.465 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:14:28.465 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:14:28.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:14:28.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:14:28.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.466 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:14:28.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:14:28.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.466 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:14:28.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:14:28.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:14:28.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:14:28.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.467 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:14:28.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:14:28.467 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.467 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.467 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.467 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.468 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:28.468 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.939009613, 2024-09-13T09:14:28.468 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:14:28.468 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.468 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.468 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.468 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.468 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.468 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:14:28.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.468 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.468 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.468 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:28.469 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.469 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.469 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:28.469 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.469 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.469 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:28.469 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:14:28.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:14:28.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:14:28.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:14:28.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:14:28.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:14:28.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:14:28.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:14:28.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:14:28.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:14:28.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:14:28.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:14:28.472 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.472 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.472 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.472 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.472 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.472 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:28.472 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.938994052, 2024-09-13T09:14:28.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:14:28.472 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.472 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.472 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.472 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.472 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.473 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:14:28.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.473 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.473 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.473 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.473 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:28.473 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.473 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.473 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.473 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:28.473 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.474 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:28.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.474 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:14:28.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:14:28.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.474 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:14:28.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:14:28.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:14:28.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:14:28.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:14:28.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:14:28.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:14:28.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:14:28.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:14:28.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:14:28.476 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.476 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.476 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.476 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.476 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:28.477 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.938977214, 2024-09-13T09:14:28.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:14:28.477 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.477 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.477 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.477 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.477 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.477 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:14:28.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.477 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:28.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.478 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:28.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.478 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:14:28.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.478 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:14:28.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:14:28.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:14:28.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:14:28.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:14:28.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:14:28.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:14:28.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:14:28.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:14:28.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:14:28.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:14:28.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:14:28.481 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.481 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.481 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.481 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.481 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:28.481 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.938961222, 2024-09-13T09:14:28.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:14:28.481 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.481 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.482 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.482 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.482 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.482 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:14:28.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.482 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:28.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:28.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:14:28.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:14:28.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:14:28.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:14:28.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:14:28.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:14:28.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:14:28.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:14:28.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:14:28.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:14:28.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:14:28.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:14:28.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:14:28.485 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.485 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.485 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.486 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.486 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:28.486 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.93893372399999, 2024-09-13T09:14:28.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:14:28.486 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.486 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.486 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.486 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.486 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.486 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:14:28.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.486 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:28.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:28.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:28.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:14:28.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:14:28.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:14:28.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:14:28.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:14:28.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:14:28.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:14:28.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:14:28.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:14:28.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:14:28.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:14:28.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:14:28.490 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.490 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.490 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.490 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.490 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:28.491 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.938907865, 2024-09-13T09:14:28.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:14:28.491 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.491 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.491 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.491 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.491 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.491 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:14:28.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.491 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:28.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:28.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:28.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:14:28.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:14:28.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:14:28.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:14:28.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:14:28.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:14:28.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:14:28.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:14:28.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:14:28.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:14:28.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:14:28.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:14:28.495 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.495 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.495 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.495 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.495 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:28.495 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.938866375, 2024-09-13T09:14:28.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:14:28.496 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.496 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.496 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.496 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.496 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.496 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:14:28.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.496 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:28.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:28.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:14:28.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:14:28.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:14:28.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:14:28.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:14:28.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:14:28.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:14:28.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:14:28.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:14:28.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:14:28.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:14:28.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:14:28.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:14:28.500 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.500 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.500 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.500 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.500 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:28.500 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.938845137, 2024-09-13T09:14:28.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:14:28.500 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.500 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.500 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.500 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.500 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.501 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:14:28.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.501 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:28.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:28.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:28.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:14:28.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:14:28.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:14:28.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:14:28.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:14:28.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:14:28.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:14:28.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:14:28.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:14:28.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:14:28.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:14:28.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:14:28.504 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.504 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.504 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.504 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.504 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:28.504 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.938822337, 2024-09-13T09:14:28.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:14:28.505 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.505 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.505 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.505 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.505 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.505 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:14:28.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.505 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:28.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:28.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:14:28.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:14:28.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:14:28.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:14:28.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:14:28.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:14:28.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:14:28.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:14:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:14:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:14:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:14:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:14:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:14:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.93880588099999, 2024-09-13T09:14:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:14:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:14:28.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.510 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:28.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:28.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:28.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:14:28.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:14:28.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:14:28.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:14:28.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:14:28.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:14:28.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:14:28.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:14:28.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:14:28.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:14:28.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:14:28.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:14:28.513 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.513 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.513 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.513 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.513 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:28.514 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.938560744, 2024-09-13T09:14:28.514 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:14:28.514 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.514 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.514 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.514 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.514 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.514 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:14:28.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.514 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.514 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.514 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:28.514 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:28.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:14:28.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:14:28.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:14:28.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:14:28.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:14:28.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:14:28.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:14:28.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:14:28.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:14:28.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:14:28.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:14:28.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:14:28.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:14:28.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:14:28.517 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.936675185, 2024-09-13T09:14:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:14:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.518 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.519 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:14:28.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.519 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:28.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:28.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.520 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:14:28.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.520 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:14:28.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:14:28.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.520 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:14:28.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:14:28.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:14:28.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:14:28.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:14:28.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:14:28.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:14:28.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:14:28.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:14:28.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:14:28.522 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.522 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.522 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.523 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.523 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:28.523 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.936653819, 2024-09-13T09:14:28.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:14:28.523 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.523 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.523 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.523 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.523 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.523 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:14:28.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.523 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:28.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:28.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:28.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:14:28.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:14:28.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:14:28.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:14:28.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:14:28.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:14:28.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:14:28.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:14:28.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:14:28.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:14:28.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:14:28.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:14:28.527 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.527 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.527 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.527 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:28.527 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:28.527 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 105.936448451, 2024-09-13T09:14:28.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:14:28.527 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.528 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:28.528 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.528 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:28.528 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.528 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:14:28.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.528 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:28.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:28.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:14:28.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:14:28.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:14:28.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:14:28.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:14:28.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:14:28.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:14:28.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:14:28.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:14:28.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:14:28.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:14:28.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:14:28.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:14:28.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:14:28.531 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.531 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.531 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.532 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5272.0:1775 3.0 3:02a42fca:::rbd_data.11235782ee8d.0000000000000030:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2244608~40448] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:14:28.532 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:28.532 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 78.194906610000004, 2024-09-13T09:14:28.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.666843732, 2024-09-13T09:14:28.532 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:28.532 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:28.532 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:28.532 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5272", 2024-09-13T09:14:28.532 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4089902097", 2024-09-13T09:14:28.532 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1775 2024-09-13T09:14:28.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.533 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:28.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:28.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:28.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:28.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:28.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:28.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247780+0000", 2024-09-13T09:14:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247865+0000", 2024-09-13T09:14:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4359999999999999e-05 2024-09-13T09:14:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247876+0000", 2024-09-13T09:14:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1456e-05 2024-09-13T09:14:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247883+0000", 2024-09-13T09:14:28.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:14:28.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247926+0000", 2024-09-13T09:14:28.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3203e-05 2024-09-13T09:14:28.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247932+0000", 2024-09-13T09:14:28.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:14:28.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909639+0000", 2024-09-13T09:14:28.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6617069099999999 2024-09-13T09:14:28.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:28.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909828+0000", 2024-09-13T09:14:28.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000189075 2024-09-13T09:14:28.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:28.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911676+0000", 2024-09-13T09:14:28.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001847235 2024-09-13T09:14:28.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:28.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911765+0000", 2024-09-13T09:14:28.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9530000000000005e-05 2024-09-13T09:14:28.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:14:28.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911901+0000", 2024-09-13T09:14:28.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013603899999999999 2024-09-13T09:14:28.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:28.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.912889+0000", 2024-09-13T09:14:28.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098807400000000003 2024-09-13T09:14:28.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:28.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914438+0000", 2024-09-13T09:14:28.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015491070000000001 2024-09-13T09:14:28.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:28.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914505+0000", 2024-09-13T09:14:28.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:14:28.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:28.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:28.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:28.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914625+0000", 2024-09-13T09:14:28.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120108 2024-09-13T09:14:28.539 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.539 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.539 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.539 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:28.539 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:28.539 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:28.541 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:14:28.541 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:28.539+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:28.641 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:28.639+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:28.649 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:28.649 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:28.649 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:28.649 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:28.658 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:14:28.741 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:28.739+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:28.793 INFO:teuthology.orchestra.run.smithi129.stderr:Exporting 3.2 info 3.2( v 130'639 (0'0,130'639] local-lis/les=127/128 n=6 ec=21/21 lis/c=127/127 les/c/f=128/128/0 sis=127) 2024-09-13T09:14:28.801 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:409868d4:::rbd_data.1123b788e5d5.0000000000000039:head# 2024-09-13T09:14:28.801 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:4262c1e6:::rbd_data.1123ee68625c.0000000000000022:head# 2024-09-13T09:14:28.801 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:436e1b7d:::rbd_object_map.1123ffac6bc2:head# 2024-09-13T09:14:28.801 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:450ddae4:::rbd_data.1123ffac6bc2.0000000000000002:head# 2024-09-13T09:14:28.802 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:45165205:::rbd_data.1123b788e5d5.000000000000003b:head# 2024-09-13T09:14:28.802 INFO:teuthology.orchestra.run.smithi129.stderr:Read #3:4550e1e5:::rbd_id.image_client.0-clone1:head# 2024-09-13T09:14:28.802 INFO:teuthology.orchestra.run.smithi129.stderr:Export successful 2024-09-13T09:14:28.805 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:28.805 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:28.805 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:28.805 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:28.806 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:28.815 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:14:28.841 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:14:28.842 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:28.840+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:28.926 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:28.926 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:28.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:28.926 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:28.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.926 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:28.926 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:28.926 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.402720737, 2024-09-13T09:14:28.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:14:28.927 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:28.927 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:28.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:28.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:28.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.927 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:14:28.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.927 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:28.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:28.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:28.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:28.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:28.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:28.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:28.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:28.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:14:28.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:14:28.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:28.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:14:28.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:14:28.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:14:28.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:14:28.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:14:28.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:14:28.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:14:28.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:14:28.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:14:28.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:14:28.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:28.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:14:28.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:14:28.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.931 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:28.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.931 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:28.931 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:28.931 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.402671681, 2024-09-13T09:14:28.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:14:28.932 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:28.932 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:28.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:28.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:28.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.932 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:14:28.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.933 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:28.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:28.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:28.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:28.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:28.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:28.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:28.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:28.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:14:28.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:14:28.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:28.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:14:28.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:14:28.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:14:28.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:14:28.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:14:28.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:14:28.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:14:28.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:14:28.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:14:28.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:14:28.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:28.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:14:28.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:14:28.937 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.937 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:28.937 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.937 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:28.937 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:28.937 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.402631606, 2024-09-13T09:14:28.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:14:28.937 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:28.937 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:28.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:28.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:28.938 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.938 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:14:28.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.938 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:28.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:28.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:28.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:28.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:28.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:28.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:28.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:28.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:14:28.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:14:28.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:28.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:14:28.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:14:28.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:14:28.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:14:28.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:14:28.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:14:28.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:14:28.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:14:28.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:14:28.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:14:28.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:28.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:14:28.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:14:28.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.942 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:28.942 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.942 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:28.942 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:28.942 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.402601791, 2024-09-13T09:14:28.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:14:28.942 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:28.942 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:28.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:28.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:28.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.942 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:14:28.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.943 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:28.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:28.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:28.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:28.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:28.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:28.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:28.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:28.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:14:28.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:14:28.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:28.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:14:28.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:14:28.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:14:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:14:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:14:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:14:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:14:28.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:14:28.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:14:28.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:14:28.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:28.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:14:28.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:14:28.947 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.947 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:28.947 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.947 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:28.947 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:28.947 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.40257213, 2024-09-13T09:14:28.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:14:28.947 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:28.947 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:28.947 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:28.947 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:28.948 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.948 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:14:28.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.948 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:28.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:28.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:28.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:28.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:28.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:28.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:28.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:28.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:14:28.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:14:28.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:28.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:14:28.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:14:28.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:14:28.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:14:28.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:14:28.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:14:28.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:14:28.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:14:28.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:14:28.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:14:28.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:28.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:14:28.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:14:28.951 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.951 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:28.952 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.952 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:28.952 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:28.952 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.402542923, 2024-09-13T09:14:28.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:14:28.952 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:28.952 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:28.952 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:28.952 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:28.952 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.952 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:14:28.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.953 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:28.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:28.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:28.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:28.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:28.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:28.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:28.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:28.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:14:28.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:14:28.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:28.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:14:28.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:14:28.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:14:28.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:14:28.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:14:28.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:14:28.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:14:28.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:14:28.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:14:28.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:14:28.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:28.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:14:28.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:14:28.956 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.957 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:28.957 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.957 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:28.957 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:28.957 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.40248674199999, 2024-09-13T09:14:28.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:14:28.957 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:28.957 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:28.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:28.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:28.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.957 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:14:28.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.958 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:28.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:28.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:28.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:28.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:28.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:28.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:28.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:28.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:14:28.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:14:28.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:28.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:14:28.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:14:28.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:14:28.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:14:28.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:14:28.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:14:28.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:14:28.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:14:28.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:14:28.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:14:28.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:28.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:14:28.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:14:28.962 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.962 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:28.962 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.962 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:28.962 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:28.962 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.402433281, 2024-09-13T09:14:28.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:14:28.962 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:28.962 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:28.963 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:28.963 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:28.963 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.963 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:14:28.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.963 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:28.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:28.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:28.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:28.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:28.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:28.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:14:28.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:28.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:14:28.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:14:28.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:28.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:14:28.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:14:28.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:14:28.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:14:28.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:14:28.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:14:28.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:14:28.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:14:28.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:14:28.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:14:28.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:28.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:14:28.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:14:28.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.967 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:28.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.967 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:28.967 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:28.967 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.40237645099999, 2024-09-13T09:14:28.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:14:28.968 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:28.968 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:28.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:28.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:28.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.968 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:14:28.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.968 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:28.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:28.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:28.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:28.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:28.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:28.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:14:28.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:28.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:14:28.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:14:28.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:28.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:14:28.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:14:28.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:14:28.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:14:28.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:14:28.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:14:28.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:14:28.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:14:28.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:14:28.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:14:28.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:28.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:14:28.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:14:28.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.972 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:28.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.973 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:28.973 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:28.973 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.402325424, 2024-09-13T09:14:28.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:14:28.973 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:28.973 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:28.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:28.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:28.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.973 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:14:28.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.973 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:28.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:28.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:28.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:28.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:28.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:28.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:14:28.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:28.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:14:28.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:14:28.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:28.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:14:28.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:14:28.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:14:28.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:14:28.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:14:28.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:14:28.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:14:28.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:14:28.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:14:28.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:14:28.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:28.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:14:28.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:14:28.977 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.977 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:28.977 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.977 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:28.977 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:28.978 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.40228423400001, 2024-09-13T09:14:28.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:14:28.978 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:28.978 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:28.978 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:28.978 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:28.978 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.978 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:14:28.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.978 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:28.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:28.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:28.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:28.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:28.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:28.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:28.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:28.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:14:28.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:14:28.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:28.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:14:28.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:14:28.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:14:28.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:14:28.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:14:28.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:14:28.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:14:28.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:14:28.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:14:28.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:14:28.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:28.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:14:28.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:14:28.982 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.983 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:28.983 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.983 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:28.983 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:28.983 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.402242118, 2024-09-13T09:14:28.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:14:28.983 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:28.983 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:28.983 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:28.983 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:28.983 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.984 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:14:28.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.984 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:28.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:28.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:28.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:28.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:28.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:28.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:28.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:28.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:28.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:14:28.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:14:28.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:28.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:14:28.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:14:28.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:14:28.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:14:28.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:14:28.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:14:28.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:14:28.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:14:28.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:14:28.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:14:28.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:28.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:14:28.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:14:28.987 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.988 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:28.988 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.988 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:28.988 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:28.988 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.402198239, 2024-09-13T09:14:28.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:14:28.988 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:28.988 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:28.988 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:28.988 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:28.988 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:14:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:14:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:14:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:14:28.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:14:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:14:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:14:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:14:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:14:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:14:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:14:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:14:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:14:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:14:28.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:14:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.40215232200001, 2024-09-13T09:14:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:14:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:28.993 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:14:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:28.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:14:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:14:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:14:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:28.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:14:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:14:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:14:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:14:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:14:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:14:28.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:14:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:14:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:14:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:14:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:14:28.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:14:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.402102551, 2024-09-13T09:14:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:14:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:28.998 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:14:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:28.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:14:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:14:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:14:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:14:29.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:14:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:14:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:14:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:14:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:14:29.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:14:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:14:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:14:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:14:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:14:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.40205714699999, 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:14:29.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:14:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:14:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:14:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:14:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:14:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:14:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:14:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:14:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:14:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:14:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:14:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:14:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:14:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.402006509, 2024-09-13T09:14:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:14:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:29.007 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:14:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:29.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:14:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:14:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:14:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:14:29.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:14:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:14:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:14:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:14:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:14:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:29.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:14:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:14:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:14:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:14:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:14:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:14:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.011 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.401950666, 2024-09-13T09:14:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:14:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:14:29.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:14:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:14:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:14:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:14:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:14:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:14:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:14:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:14:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:14:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:14:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:14:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:14:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:14:29.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:14:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.40189374400001, 2024-09-13T09:14:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:14:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.017 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:14:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:14:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:14:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:14:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:14:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:14:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:14:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:14:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:14:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:14:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:14:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:14:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:14:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:14:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:14:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 106.401840181, 2024-09-13T09:14:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:14:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:14:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.023 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:14:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:14:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:14:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:14:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:14:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:14:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:14:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:14:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:14:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:14:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:14:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:14:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:14:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:14:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.028 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.028 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:29.030 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:14:29.030 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:29.027+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:29.131 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:29.129+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:29.133 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:29.133 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:29.133 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:29.133 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:29.142 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:14:29.232 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:29.230+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:29.244 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:29.244 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:29.244 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:29.244 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:29.244 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:29.254 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:14:29.321 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --force --op remove --pgid 3.2 2024-09-13T09:14:29.360 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:29.360 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:29.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:29.361 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:29.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.361 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:29.361 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:29.361 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.831595254000007, 2024-09-13T09:14:29.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:14:29.361 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.361 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.361 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.361 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:29.361 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:29.361 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:14:29.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.362 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:29.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:29.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:14:29.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:14:29.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:14:29.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:14:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:14:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:14:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:14:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:14:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:14:29.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:14:29.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:14:29.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:14:29.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:14:29.365 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.365 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.365 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.366 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:29.366 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:29.366 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.831577555999999, 2024-09-13T09:14:29.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:14:29.366 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.366 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.366 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.366 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:29.366 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:29.366 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:14:29.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.366 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:29.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:29.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:29.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:14:29.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:14:29.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:14:29.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:14:29.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:14:29.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:14:29.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:14:29.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:14:29.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:14:29.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:14:29.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:14:29.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:14:29.370 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.370 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.370 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.370 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:29.370 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:29.370 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.831556941000002, 2024-09-13T09:14:29.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:14:29.370 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.370 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.370 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:29.371 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:29.371 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:14:29.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.371 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:29.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:29.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:29.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:14:29.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:14:29.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:14:29.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:14:29.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:14:29.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:14:29.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:14:29.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:14:29.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:14:29.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:14:29.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:14:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:14:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.831537984999997, 2024-09-13T09:14:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:14:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.375 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:29.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:29.376 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:14:29.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.376 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:29.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:29.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:29.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:14:29.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:14:29.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:14:29.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:14:29.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:14:29.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:14:29.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:14:29.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:14:29.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:14:29.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:14:29.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:14:29.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:14:29.379 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.380 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.380 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.380 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:29.380 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:29.380 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.831520726999997, 2024-09-13T09:14:29.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:14:29.380 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.380 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.380 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.380 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:29.380 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:29.380 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:14:29.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.381 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:29.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:29.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:29.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:14:29.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:14:29.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:14:29.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:14:29.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:14:29.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:14:29.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:14:29.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:14:29.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:14:29.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:14:29.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:14:29.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:14:29.384 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.384 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.384 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.385 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:29.385 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:29.385 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.831500712999997, 2024-09-13T09:14:29.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:14:29.385 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.385 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.385 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.385 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:29.385 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:29.385 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:14:29.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.386 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:29.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:29.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:29.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:14:29.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:14:29.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:14:29.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:14:29.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:14:29.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:14:29.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:14:29.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:14:29.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:14:29.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:14:29.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:14:29.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:14:29.389 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.389 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.389 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.389 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:29.390 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:29.390 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.831475131000005, 2024-09-13T09:14:29.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:14:29.390 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.390 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.390 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.390 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:29.390 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:29.390 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:14:29.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.390 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:29.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:29.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:14:29.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:14:29.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:14:29.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:14:29.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:14:29.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:14:29.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:14:29.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:14:29.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:14:29.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:14:29.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:14:29.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:14:29.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:14:29.394 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.394 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.395 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.395 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:29.395 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:29.395 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.831455847000001, 2024-09-13T09:14:29.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:14:29.395 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.395 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:29.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:29.395 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:14:29.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.396 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:29.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:29.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:29.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:14:29.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:14:29.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:14:29.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:14:29.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:14:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:14:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:14:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:14:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:14:29.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:14:29.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:14:29.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:14:29.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.399 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.399 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:29.399 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:29.400 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.831438590999994, 2024-09-13T09:14:29.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:14:29.400 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.400 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:29.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:29.400 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:14:29.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.400 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:29.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:29.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:29.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:14:29.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:14:29.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:14:29.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:14:29.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:14:29.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:14:29.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:14:29.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:14:29.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:14:29.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:14:29.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:14:29.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:14:29.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.404 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.405 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.405 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:29.405 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:29.405 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.831419111000002, 2024-09-13T09:14:29.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:14:29.406 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.406 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:29.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:29.406 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:14:29.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.406 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:29.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:29.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:14:29.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:14:29.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:14:29.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:14:29.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:14:29.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:14:29.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:14:29.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:14:29.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:14:29.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:14:29.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:14:29.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:14:29.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:14:29.409 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.409 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.409 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.410 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:29.410 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:29.410 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.831402182000005, 2024-09-13T09:14:29.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:14:29.410 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.410 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:29.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:29.410 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:14:29.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.410 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:29.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:29.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:14:29.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:14:29.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:14:29.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:14:29.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:14:29.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:14:29.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:14:29.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:14:29.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:14:29.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:14:29.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:14:29.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:14:29.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:14:29.414 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.414 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.414 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.414 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:29.414 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:29.414 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.831383032999994, 2024-09-13T09:14:29.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:14:29.414 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.414 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:29.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:29.415 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:14:29.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.415 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:29.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:29.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:29.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:14:29.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:14:29.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:14:29.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:14:29.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:14:29.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:14:29.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:14:29.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:14:29.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:14:29.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:14:29.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:14:29.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:14:29.418 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.418 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.419 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:29.419 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:29.419 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.831364894000004, 2024-09-13T09:14:29.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:14:29.419 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.419 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:29.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:29.419 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:14:29.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.420 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:29.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:29.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:29.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:14:29.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:14:29.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:14:29.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:14:29.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:14:29.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:14:29.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:14:29.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:14:29.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:14:29.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:14:29.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:14:29.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:14:29.423 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.423 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.423 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.423 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:29.423 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:29.423 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.830954399000007, 2024-09-13T09:14:29.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:14:29.423 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.423 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.424 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.424 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:29.424 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:29.424 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:14:29.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.424 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:29.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:29.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:29.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:14:29.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:14:29.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:14:29.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:14:29.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:14:29.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:14:29.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:14:29.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:14:29.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:14:29.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:14:29.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:14:29.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:14:29.427 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.427 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.427 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.428 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:29.428 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:29.428 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 73.830931497999998, 2024-09-13T09:14:29.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:14:29.428 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.428 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:29.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:29.428 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:14:29.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.428 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:29.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:29.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:29.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:14:29.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:14:29.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:14:29.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:14:29.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:14:29.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:14:29.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:14:29.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:14:29.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:14:29.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:14:29.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:14:29.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:14:29.432 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.432 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.432 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.432 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:30141 3.9 3:908b9f7f:::rbd_data.112326fb43d3.0000000000000001:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-13T09:14:29.432 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:14:18.853518+0000", 2024-09-13T09:14:29.432 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.503931010000001, 2024-09-13T09:14:29.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.92492906200000002, 2024-09-13T09:14:29.432 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.432 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.432 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.433 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:29.433 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:29.433 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 30141 2024-09-13T09:14:29.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.433 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853518+0000", 2024-09-13T09:14:29.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853518+0000", 2024-09-13T09:14:29.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853516+0000", 2024-09-13T09:14:29.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:14:29.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853527+0000", 2024-09-13T09:14:29.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1420000000000001e-05 2024-09-13T09:14:29.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853556+0000", 2024-09-13T09:14:29.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8496e-05 2024-09-13T09:14:29.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853569+0000", 2024-09-13T09:14:29.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3446e-05 2024-09-13T09:14:29.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853691+0000", 2024-09-13T09:14:29.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012215500000000001 2024-09-13T09:14:29.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:29.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853715+0000", 2024-09-13T09:14:29.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3266000000000001e-05 2024-09-13T09:14:29.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.775199+0000", 2024-09-13T09:14:29.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.92148457100000003 2024-09-13T09:14:29.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.775330+0000", 2024-09-13T09:14:29.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013118200000000001 2024-09-13T09:14:29.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:14:29.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.775498+0000", 2024-09-13T09:14:29.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016740200000000001 2024-09-13T09:14:29.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:29.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.777064+0000", 2024-09-13T09:14:29.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015657469999999999 2024-09-13T09:14:29.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:29.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778397+0000", 2024-09-13T09:14:29.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00133325 2024-09-13T09:14:29.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:29.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778426+0000", 2024-09-13T09:14:29.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9612e-05 2024-09-13T09:14:29.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778447+0000", 2024-09-13T09:14:29.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0494999999999999e-05 2024-09-13T09:14:29.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.438 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.439 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:30143 3.9 3:93c2f121:::rbd_data.112326fb43d3.0000000000000003:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-13T09:14:29.439 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:14:18.853590+0000", 2024-09-13T09:14:29.439 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.503858804, 2024-09-13T09:14:29.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.92495849799999996, 2024-09-13T09:14:29.439 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.439 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:29.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:29.439 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 30143 2024-09-13T09:14:29.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.439 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853590+0000", 2024-09-13T09:14:29.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853590+0000", 2024-09-13T09:14:29.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853589+0000", 2024-09-13T09:14:29.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:14:29.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853595+0000", 2024-09-13T09:14:29.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:14:29.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853616+0000", 2024-09-13T09:14:29.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1313000000000001e-05 2024-09-13T09:14:29.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853630+0000", 2024-09-13T09:14:29.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3370000000000001e-05 2024-09-13T09:14:29.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853801+0000", 2024-09-13T09:14:29.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00017078899999999999 2024-09-13T09:14:29.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:29.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853823+0000", 2024-09-13T09:14:29.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2557000000000001e-05 2024-09-13T09:14:29.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.776396+0000", 2024-09-13T09:14:29.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.92257259800000002 2024-09-13T09:14:29.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.776478+0000", 2024-09-13T09:14:29.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2528999999999995e-05 2024-09-13T09:14:29.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:14:29.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.776559+0000", 2024-09-13T09:14:29.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1043000000000003e-05 2024-09-13T09:14:29.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:29.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778313+0000", 2024-09-13T09:14:29.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0017531440000000001 2024-09-13T09:14:29.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:29.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778479+0000", 2024-09-13T09:14:29.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016687000000000001 2024-09-13T09:14:29.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:29.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778508+0000", 2024-09-13T09:14:29.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8311999999999999e-05 2024-09-13T09:14:29.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778549+0000", 2024-09-13T09:14:29.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0907e-05 2024-09-13T09:14:29.445 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.445 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.445 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.445 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:30877 3.e 3:72416bbe:::rbd_header.112323636db9:head [call rbd.snapshot_remove in=18b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e133)", 2024-09-13T09:14:29.446 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:14:26.823896+0000", 2024-09-13T09:14:29.446 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.5335532499999998, 2024-09-13T09:14:29.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0092431399999999, 2024-09-13T09:14:29.446 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.446 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.446 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.446 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:29.446 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:29.446 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 30877 2024-09-13T09:14:29.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.446 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823896+0000", 2024-09-13T09:14:29.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823896+0000", 2024-09-13T09:14:29.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823894+0000", 2024-09-13T09:14:29.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:14:29.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823904+0000", 2024-09-13T09:14:29.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0881e-05 2024-09-13T09:14:29.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823923+0000", 2024-09-13T09:14:29.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8105999999999999e-05 2024-09-13T09:14:29.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823934+0000", 2024-09-13T09:14:29.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1559999999999999e-05 2024-09-13T09:14:29.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.824008+0000", 2024-09-13T09:14:29.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3452000000000001e-05 2024-09-13T09:14:29.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:14:29.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.824016+0000", 2024-09-13T09:14:29.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5609999999999999e-06 2024-09-13T09:14:29.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.827969+0000", 2024-09-13T09:14:29.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003952817 2024-09-13T09:14:29.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:29.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.827985+0000", 2024-09-13T09:14:29.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6314e-05 2024-09-13T09:14:29.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.831801+0000", 2024-09-13T09:14:29.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0038155980000001 2024-09-13T09:14:29.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.832061+0000", 2024-09-13T09:14:29.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025987599999999998 2024-09-13T09:14:29.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:29.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833014+0000", 2024-09-13T09:14:29.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00095324299999999999 2024-09-13T09:14:29.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:29.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833055+0000", 2024-09-13T09:14:29.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0895000000000001e-05 2024-09-13T09:14:29.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833139+0000", 2024-09-13T09:14:29.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3898000000000002e-05 2024-09-13T09:14:29.452 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.452 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.452 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.452 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:30879 3.e 3:72416bbe:::rbd_header.112323636db9:head [watch reconnect cookie 140383088041072 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-13T09:14:29.452 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:14:26.823954+0000", 2024-09-13T09:14:29.453 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.5334953919999998, 2024-09-13T09:14:29.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0099145549999999, 2024-09-13T09:14:29.453 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.453 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.453 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.453 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:29.453 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:29.453 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 30879 2024-09-13T09:14:29.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.453 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823954+0000", 2024-09-13T09:14:29.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823954+0000", 2024-09-13T09:14:29.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823953+0000", 2024-09-13T09:14:29.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823956+0000", 2024-09-13T09:14:29.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:14:29.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823963+0000", 2024-09-13T09:14:29.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0890000000000002e-06 2024-09-13T09:14:29.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823969+0000", 2024-09-13T09:14:29.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5690000000000004e-06 2024-09-13T09:14:29.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.824079+0000", 2024-09-13T09:14:29.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000110147 2024-09-13T09:14:29.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:14:29.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.824092+0000", 2024-09-13T09:14:29.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2927e-05 2024-09-13T09:14:29.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.828011+0000", 2024-09-13T09:14:29.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039192890000000003 2024-09-13T09:14:29.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:29.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.828018+0000", 2024-09-13T09:14:29.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2620000000000001e-06 2024-09-13T09:14:29.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.832844+0000", 2024-09-13T09:14:29.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.004825871 2024-09-13T09:14:29.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for rw locks", 2024-09-13T09:14:29.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.832937+0000", 2024-09-13T09:14:29.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2279000000000002e-05 2024-09-13T09:14:29.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833169+0000", 2024-09-13T09:14:29.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000232716 2024-09-13T09:14:29.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833215+0000", 2024-09-13T09:14:29.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5939000000000002e-05 2024-09-13T09:14:29.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833868+0000", 2024-09-13T09:14:29.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00065288399999999995 2024-09-13T09:14:29.459 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.459 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.459 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.459 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:30880 3.e 3:72416bbe:::rbd_header.112323636db9:head [watch ping cookie 140383088041072 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-13T09:14:29.459 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:14:27.104897+0000", 2024-09-13T09:14:29.459 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.2525523359999999, 2024-09-13T09:14:29.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.72873839699999998, 2024-09-13T09:14:29.459 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:29.459 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:29.460 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:29.460 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:29.460 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:29.460 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 30880 2024-09-13T09:14:29.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.460 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:29.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:29.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104897+0000", 2024-09-13T09:14:29.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:29.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104897+0000", 2024-09-13T09:14:29.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:29.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:29.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104896+0000", 2024-09-13T09:14:29.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:14:29.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:29.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104901+0000", 2024-09-13T09:14:29.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1290000000000004e-06 2024-09-13T09:14:29.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:29.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104908+0000", 2024-09-13T09:14:29.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.63e-06 2024-09-13T09:14:29.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104915+0000", 2024-09-13T09:14:29.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8849999999999999e-06 2024-09-13T09:14:29.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104977+0000", 2024-09-13T09:14:29.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1896999999999994e-05 2024-09-13T09:14:29.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:29.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104987+0000", 2024-09-13T09:14:29.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5259999999999999e-06 2024-09-13T09:14:29.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.832958+0000", 2024-09-13T09:14:29.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.72797099099999996 2024-09-13T09:14:29.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for rw locks", 2024-09-13T09:14:29.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833006+0000", 2024-09-13T09:14:29.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8177000000000003e-05 2024-09-13T09:14:29.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:29.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833457+0000", 2024-09-13T09:14:29.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000451213 2024-09-13T09:14:29.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:29.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833525+0000", 2024-09-13T09:14:29.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8194999999999993e-05 2024-09-13T09:14:29.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:29.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:29.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:29.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833635+0000", 2024-09-13T09:14:29.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010948 2024-09-13T09:14:29.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.465 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:29.465 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:29.465 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:29.467 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:29.358+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:29.468 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:14:29.510 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:29.508+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:29.570 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:29.568+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:29.578 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:29.578 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:29.578 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:29.578 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:29.587 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:14:29.672 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:29.670+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:29.693 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:29.693 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:29.702 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:14:29.773 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:29.772+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:29.835 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-09-13T09:14:29.861 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29717 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [watch reconnect cookie 140383290562704 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:14:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.107301024, 2024-09-13T09:14:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.006472936, 2024-09-13T09:14:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29717 2024-09-13T09:14:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751547+0000", 2024-09-13T09:14:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751552+0000", 2024-09-13T09:14:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:14:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751564+0000", 2024-09-13T09:14:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1793e-05 2024-09-13T09:14:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751571+0000", 2024-09-13T09:14:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2660000000000004e-06 2024-09-13T09:14:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753102+0000", 2024-09-13T09:14:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001531085 2024-09-13T09:14:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753115+0000", 2024-09-13T09:14:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3039e-05 2024-09-13T09:14:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.752858+0000", 2024-09-13T09:14:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.999742715 2024-09-13T09:14:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.753010+0000", 2024-09-13T09:14:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015227799999999999 2024-09-13T09:14:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755125+0000", 2024-09-13T09:14:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021150029999999998 2024-09-13T09:14:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-09-13T09:14:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755235+0000", 2024-09-13T09:14:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000109775 2024-09-13T09:14:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756059+0000", 2024-09-13T09:14:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00082370900000000005 2024-09-13T09:14:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756154+0000", 2024-09-13T09:14:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4945999999999995e-05 2024-09-13T09:14:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758021+0000", 2024-09-13T09:14:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001867446 2024-09-13T09:14:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29718 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:14:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.107268159, 2024-09-13T09:14:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0067812169999999, 2024-09-13T09:14:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29718 2024-09-13T09:14:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751585+0000", 2024-09-13T09:14:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2050000000000004e-06 2024-09-13T09:14:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751591+0000", 2024-09-13T09:14:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.37e-06 2024-09-13T09:14:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751597+0000", 2024-09-13T09:14:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1190000000000002e-06 2024-09-13T09:14:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753150+0000", 2024-09-13T09:14:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001552351 2024-09-13T09:14:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753164+0000", 2024-09-13T09:14:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4076e-05 2024-09-13T09:14:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.753307+0000", 2024-09-13T09:14:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0001434229999999 2024-09-13T09:14:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.753365+0000", 2024-09-13T09:14:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8014999999999999e-05 2024-09-13T09:14:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755501+0000", 2024-09-13T09:14:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021357260000000001 2024-09-13T09:14:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-09-13T09:14:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755558+0000", 2024-09-13T09:14:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7516000000000003e-05 2024-09-13T09:14:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756684+0000", 2024-09-13T09:14:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011259550000000001 2024-09-13T09:14:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756766+0000", 2024-09-13T09:14:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1755999999999994e-05 2024-09-13T09:14:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.757147+0000", 2024-09-13T09:14:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000381107 2024-09-13T09:14:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.757890+0000", 2024-09-13T09:14:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00074321499999999998 2024-09-13T09:14:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758178+0000", 2024-09-13T09:14:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00028755000000000002 2024-09-13T09:14:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758230+0000", 2024-09-13T09:14:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2317000000000001e-05 2024-09-13T09:14:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758362+0000", 2024-09-13T09:14:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013204100000000001 2024-09-13T09:14:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGPush(3.d 123/122 [PushOp(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29, version: 99'289, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29@99'289, size: 12582912, copy_subset: [0~12582912], clone_subset: {}, snapset: 29=[]:{29=[29]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:14:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.774123+0000", 2024-09-13T09:14:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.084726494, 2024-09-13T09:14:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051988820000000002, 2024-09-13T09:14:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.774123+0000", 2024-09-13T09:14:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.774123+0000", 2024-09-13T09:14:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.774122+0000", 2024-09-13T09:14:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777696+0000", 2024-09-13T09:14:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003573676 2024-09-13T09:14:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777708+0000", 2024-09-13T09:14:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2349e-05 2024-09-13T09:14:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777718+0000", 2024-09-13T09:14:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0013999999999999e-05 2024-09-13T09:14:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777761+0000", 2024-09-13T09:14:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3167999999999997e-05 2024-09-13T09:14:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777771+0000", 2024-09-13T09:14:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3600000000000002e-06 2024-09-13T09:14:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.779322+0000", 2024-09-13T09:14:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015509740000000001 2024-09-13T09:14:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29749 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [watch reconnect cookie 140383086477584 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-13T09:14:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.059429301, 2024-09-13T09:14:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037671060000000001, 2024-09-13T09:14:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29749 2024-09-13T09:14:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799422+0000", 2024-09-13T09:14:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:14:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799429+0000", 2024-09-13T09:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4670000000000001e-06 2024-09-13T09:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799434+0000", 2024-09-13T09:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0520000000000001e-06 2024-09-13T09:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.801379+0000", 2024-09-13T09:14:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019449599999999999 2024-09-13T09:14:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.801426+0000", 2024-09-13T09:14:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7391000000000001e-05 2024-09-13T09:14:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.803187+0000", 2024-09-13T09:14:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0017609870000000001 2024-09-13T09:14:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.3.0:2661 3.d e124/122 3:b5ba65cb:::rbd_data.1123fa58739e.0000000000000006:12 v 124'356, mlcod=123'351)", 2024-09-13T09:14:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.110701855, 2024-09-13T09:14:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034693089999999998, 2024-09-13T09:14:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748155+0000", 2024-09-13T09:14:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3769999999999996e-06 2024-09-13T09:14:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748163+0000", 2024-09-13T09:14:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0120000000000005e-06 2024-09-13T09:14:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748169+0000", 2024-09-13T09:14:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0610000000000003e-06 2024-09-13T09:14:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748695+0000", 2024-09-13T09:14:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052687199999999995 2024-09-13T09:14:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748716+0000", 2024-09-13T09:14:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0126e-05 2024-09-13T09:14:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.751587+0000", 2024-09-13T09:14:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028710839999999999 2024-09-13T09:14:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.751617+0000", 2024-09-13T09:14:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0096000000000001e-05 2024-09-13T09:14:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGPush(3.9 124/123 [PushOp(3:94c9dbee:::rbd_data.112339c5624d.0000000000000000:head, version: 88'1057, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:94c9dbee:::rbd_data.112339c5624d.0000000000000000:head@88'1057, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:0, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:14:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:17.795705+0000", 2024-09-13T09:14:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.063144696, 2024-09-13T09:14:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0038339609999999999, 2024-09-13T09:14:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795705+0000", 2024-09-13T09:14:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795705+0000", 2024-09-13T09:14:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795704+0000", 2024-09-13T09:14:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795715+0000", 2024-09-13T09:14:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1025e-05 2024-09-13T09:14:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795727+0000", 2024-09-13T09:14:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2031000000000001e-05 2024-09-13T09:14:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795736+0000", 2024-09-13T09:14:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2220000000000008e-06 2024-09-13T09:14:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795816+0000", 2024-09-13T09:14:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9338000000000005e-05 2024-09-13T09:14:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795829+0000", 2024-09-13T09:14:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3677e-05 2024-09-13T09:14:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.799539+0000", 2024-09-13T09:14:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037092169999999999 2024-09-13T09:14:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.3.0:2692 3.d e125/122 3:b6fe9194:::rbd_data.112326fb43d3.000000000000002a:11 v 125'358, mlcod=124'354)", 2024-09-13T09:14:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:18.770322+0000", 2024-09-13T09:14:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.088527444, 2024-09-13T09:14:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047215089999999996, 2024-09-13T09:14:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770322+0000", 2024-09-13T09:14:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770322+0000", 2024-09-13T09:14:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770321+0000", 2024-09-13T09:14:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770333+0000", 2024-09-13T09:14:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2014000000000001e-05 2024-09-13T09:14:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770346+0000", 2024-09-13T09:14:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2771e-05 2024-09-13T09:14:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770354+0000", 2024-09-13T09:14:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1249999999999993e-06 2024-09-13T09:14:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770459+0000", 2024-09-13T09:14:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010558200000000001 2024-09-13T09:14:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770483+0000", 2024-09-13T09:14:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3125e-05 2024-09-13T09:14:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.775015+0000", 2024-09-13T09:14:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045322449999999999 2024-09-13T09:14:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.775043+0000", 2024-09-13T09:14:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8532999999999999e-05 2024-09-13T09:14:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2445 3.9 e126/125 3:91fd9b46:::rbd_data.112326fb43d3.000000000000010f:11 v 126'1140, mlcod=126'1134)", 2024-09-13T09:14:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.076671099, 2024-09-13T09:14:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0036771899999999999, 2024-09-13T09:14:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782184+0000", 2024-09-13T09:14:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5729999999999996e-06 2024-09-13T09:14:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782196+0000", 2024-09-13T09:14:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1817000000000001e-05 2024-09-13T09:14:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782202+0000", 2024-09-13T09:14:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:14:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.783326+0000", 2024-09-13T09:14:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011234509999999999 2024-09-13T09:14:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.783343+0000", 2024-09-13T09:14:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7504e-05 2024-09-13T09:14:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.785708+0000", 2024-09-13T09:14:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023648829999999999 2024-09-13T09:14:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.785855+0000", 2024-09-13T09:14:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014743199999999999 2024-09-13T09:14:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.1.0:2475 3.4 e126/125 3:2095a19f:::rbd_data.1123b428d19a.000000000000001b:16 v 126'630, mlcod=126'626)", 2024-09-13T09:14:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.784119+0000", 2024-09-13T09:14:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.074730594, 2024-09-13T09:14:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033969479999999999, 2024-09-13T09:14:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784119+0000", 2024-09-13T09:14:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784119+0000", 2024-09-13T09:14:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784118+0000", 2024-09-13T09:14:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784126+0000", 2024-09-13T09:14:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6490000000000008e-06 2024-09-13T09:14:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784136+0000", 2024-09-13T09:14:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9699999999999994e-06 2024-09-13T09:14:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784141+0000", 2024-09-13T09:14:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8730000000000004e-06 2024-09-13T09:14:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784219+0000", 2024-09-13T09:14:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7650000000000004e-05 2024-09-13T09:14:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784239+0000", 2024-09-13T09:14:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9794999999999999e-05 2024-09-13T09:14:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.787481+0000", 2024-09-13T09:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0032426320000000001 2024-09-13T09:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.787516+0000", 2024-09-13T09:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4143000000000002e-05 2024-09-13T09:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2447 3.9 e126/125 3:9541a82a:::rbd_data.1123b428d19a.000000000000000a:16 v 126'1144, mlcod=126'1140)", 2024-09-13T09:14:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.793072+0000", 2024-09-13T09:14:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.065777378, 2024-09-13T09:14:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046128150000000001, 2024-09-13T09:14:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793072+0000", 2024-09-13T09:14:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793072+0000", 2024-09-13T09:14:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793071+0000", 2024-09-13T09:14:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793077+0000", 2024-09-13T09:14:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1080000000000001e-06 2024-09-13T09:14:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793085+0000", 2024-09-13T09:14:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7309999999999999e-06 2024-09-13T09:14:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793091+0000", 2024-09-13T09:14:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6300000000000003e-06 2024-09-13T09:14:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793160+0000", 2024-09-13T09:14:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8681000000000002e-05 2024-09-13T09:14:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793178+0000", 2024-09-13T09:14:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8861000000000001e-05 2024-09-13T09:14:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.797650+0000", 2024-09-13T09:14:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004471193 2024-09-13T09:14:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.797685+0000", 2024-09-13T09:14:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5042000000000001e-05 2024-09-13T09:14:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop_reply(osd.2.0:98 3.18 e126/125 ondisk, result = 0)", 2024-09-13T09:14:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.794045+0000", 2024-09-13T09:14:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.064804551, 2024-09-13T09:14:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050958949999999996, 2024-09-13T09:14:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794045+0000", 2024-09-13T09:14:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794045+0000", 2024-09-13T09:14:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794044+0000", 2024-09-13T09:14:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794049+0000", 2024-09-13T09:14:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8600000000000001e-06 2024-09-13T09:14:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794060+0000", 2024-09-13T09:14:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0801e-05 2024-09-13T09:14:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794068+0000", 2024-09-13T09:14:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1550000000000007e-06 2024-09-13T09:14:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.799120+0000", 2024-09-13T09:14:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050525040000000002 2024-09-13T09:14:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.799128+0000", 2024-09-13T09:14:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2819999999999996e-06 2024-09-13T09:14:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.799141+0000", 2024-09-13T09:14:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2098e-05 2024-09-13T09:14:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2450 3.9 e126/125 3:95b1a540:::rbd_data.1123e3c0a1e8.0000000000000030:18 v 126'1148, mlcod=126'1144)", 2024-09-13T09:14:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.057220915, 2024-09-13T09:14:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0035455719999999999, 2024-09-13T09:14:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801634+0000", 2024-09-13T09:14:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9539999999999998e-06 2024-09-13T09:14:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801646+0000", 2024-09-13T09:14:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1795e-05 2024-09-13T09:14:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801653+0000", 2024-09-13T09:14:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7259999999999996e-06 2024-09-13T09:14:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.803017+0000", 2024-09-13T09:14:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001363997 2024-09-13T09:14:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.803038+0000", 2024-09-13T09:14:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1294e-05 2024-09-13T09:14:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.805138+0000", 2024-09-13T09:14:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020996259999999998 2024-09-13T09:14:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.805174+0000", 2024-09-13T09:14:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5970999999999998e-05 2024-09-13T09:14:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2452 3.9 e126/125 3:9030a72f:::rbd_data.1123af309369.0000000000000023:1f v 126'1152, mlcod=126'1148)", 2024-09-13T09:14:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.812010+0000", 2024-09-13T09:14:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.046839565000001, 2024-09-13T09:14:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033163540000000001, 2024-09-13T09:14:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812010+0000", 2024-09-13T09:14:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812010+0000", 2024-09-13T09:14:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812009+0000", 2024-09-13T09:14:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812093+0000", 2024-09-13T09:14:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4022000000000003e-05 2024-09-13T09:14:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812121+0000", 2024-09-13T09:14:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.826e-05 2024-09-13T09:14:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812128+0000", 2024-09-13T09:14:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6270000000000002e-06 2024-09-13T09:14:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812170+0000", 2024-09-13T09:14:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1785000000000001e-05 2024-09-13T09:14:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812202+0000", 2024-09-13T09:14:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2505999999999997e-05 2024-09-13T09:14:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.815226+0000", 2024-09-13T09:14:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003024185 2024-09-13T09:14:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.815326+0000", 2024-09-13T09:14:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9637999999999997e-05 2024-09-13T09:14:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2453 3.9 e126/125 3:94dbe637:::rbd_data.1123af309369.00000000000001dc:1f v 126'1154, mlcod=126'1148)", 2024-09-13T09:14:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.812843+0000", 2024-09-13T09:14:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.046006705, 2024-09-13T09:14:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034843890000000001, 2024-09-13T09:14:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812843+0000", 2024-09-13T09:14:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812843+0000", 2024-09-13T09:14:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812842+0000", 2024-09-13T09:14:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812848+0000", 2024-09-13T09:14:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4189999999999999e-06 2024-09-13T09:14:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812859+0000", 2024-09-13T09:14:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0778000000000001e-05 2024-09-13T09:14:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812866+0000", 2024-09-13T09:14:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7619999999999998e-06 2024-09-13T09:14:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.813231+0000", 2024-09-13T09:14:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00036497299999999999 2024-09-13T09:14:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.813251+0000", 2024-09-13T09:14:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0511999999999999e-05 2024-09-13T09:14:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.816298+0000", 2024-09-13T09:14:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003046801 2024-09-13T09:14:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.816327+0000", 2024-09-13T09:14:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8943000000000001e-05 2024-09-13T09:14:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2455 3.9 e126/125 3:95f6e286:::rbd_data.1123af309369.00000000000000ba:1f v 126'1158, mlcod=126'1152)", 2024-09-13T09:14:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.817505+0000", 2024-09-13T09:14:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.041344228, 2024-09-13T09:14:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034876989999999999, 2024-09-13T09:14:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817505+0000", 2024-09-13T09:14:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817505+0000", 2024-09-13T09:14:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817504+0000", 2024-09-13T09:14:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817512+0000", 2024-09-13T09:14:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1389999999999995e-06 2024-09-13T09:14:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817532+0000", 2024-09-13T09:14:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9089e-05 2024-09-13T09:14:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817538+0000", 2024-09-13T09:14:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5799999999999997e-06 2024-09-13T09:14:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.818301+0000", 2024-09-13T09:14:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00076256200000000005 2024-09-13T09:14:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.818318+0000", 2024-09-13T09:14:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7428999999999999e-05 2024-09-13T09:14:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820959+0000", 2024-09-13T09:14:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026407129999999998 2024-09-13T09:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820993+0000", 2024-09-13T09:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3912000000000003e-05 2024-09-13T09:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop_reply(osd.2.0:107 3.18 e126/125 ondisk, result = 0)", 2024-09-13T09:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.820584+0000", 2024-09-13T09:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.038265637, 2024-09-13T09:14:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045400709999999997, 2024-09-13T09:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820584+0000", 2024-09-13T09:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820584+0000", 2024-09-13T09:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820583+0000", 2024-09-13T09:14:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820588+0000", 2024-09-13T09:14:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7029999999999997e-06 2024-09-13T09:14:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820598+0000", 2024-09-13T09:14:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0179e-05 2024-09-13T09:14:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820604+0000", 2024-09-13T09:14:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:14:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825098+0000", 2024-09-13T09:14:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044948009999999997 2024-09-13T09:14:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825108+0000", 2024-09-13T09:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3759999999999997e-06 2024-09-13T09:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825124+0000", 2024-09-13T09:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5950000000000001e-05 2024-09-13T09:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2456 3.9 e126/125 3:93610890:::rbd_data.1123af309369.00000000000003af:1f v 126'1160, mlcod=126'1156)", 2024-09-13T09:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.821541+0000", 2024-09-13T09:14:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.037308255999999, 2024-09-13T09:14:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003753915, 2024-09-13T09:14:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821541+0000", 2024-09-13T09:14:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821541+0000", 2024-09-13T09:14:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821540+0000", 2024-09-13T09:14:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821546+0000", 2024-09-13T09:14:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7370000000000003e-06 2024-09-13T09:14:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821604+0000", 2024-09-13T09:14:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8072000000000002e-05 2024-09-13T09:14:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821616+0000", 2024-09-13T09:14:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1956999999999999e-05 2024-09-13T09:14:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821711+0000", 2024-09-13T09:14:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5079000000000002e-05 2024-09-13T09:14:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821731+0000", 2024-09-13T09:14:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9661999999999999e-05 2024-09-13T09:14:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825263+0000", 2024-09-13T09:14:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0035322399999999999 2024-09-13T09:14:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825295+0000", 2024-09-13T09:14:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1813000000000002e-05 2024-09-13T09:14:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2457 3.9 e126/125 3:9313c095:::rbd_data.1123af309369.00000000000000be:1f v 126'1162, mlcod=126'1156)", 2024-09-13T09:14:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.822601+0000", 2024-09-13T09:14:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.036248698, 2024-09-13T09:14:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003671147, 2024-09-13T09:14:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822601+0000", 2024-09-13T09:14:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822601+0000", 2024-09-13T09:14:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822600+0000", 2024-09-13T09:14:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822605+0000", 2024-09-13T09:14:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6979999999999997e-06 2024-09-13T09:14:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822612+0000", 2024-09-13T09:14:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8589999999999999e-06 2024-09-13T09:14:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822617+0000", 2024-09-13T09:14:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:14:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.823075+0000", 2024-09-13T09:14:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00045848500000000001 2024-09-13T09:14:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.823094+0000", 2024-09-13T09:14:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8740999999999999e-05 2024-09-13T09:14:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.826239+0000", 2024-09-13T09:14:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0031454709999999999 2024-09-13T09:14:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.826272+0000", 2024-09-13T09:14:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2330999999999999e-05 2024-09-13T09:14:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:30365 3.7 3:e7191d4c:::rbd_header.1123313a4365:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-13T09:14:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:22.797253+0000", 2024-09-13T09:14:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 7.061596336, 2024-09-13T09:14:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0059535409999999997, 2024-09-13T09:14:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 30365 2024-09-13T09:14:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797253+0000", 2024-09-13T09:14:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797253+0000", 2024-09-13T09:14:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797251+0000", 2024-09-13T09:14:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-09-13T09:14:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797262+0000", 2024-09-13T09:14:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0993e-05 2024-09-13T09:14:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797281+0000", 2024-09-13T09:14:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8216999999999999e-05 2024-09-13T09:14:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797290+0000", 2024-09-13T09:14:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5230000000000001e-06 2024-09-13T09:14:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797339+0000", 2024-09-13T09:14:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8854999999999999e-05 2024-09-13T09:14:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797437+0000", 2024-09-13T09:14:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8105999999999999e-05 2024-09-13T09:14:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:14:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797842+0000", 2024-09-13T09:14:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00040497999999999999 2024-09-13T09:14:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.800594+0000", 2024-09-13T09:14:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002752193 2024-09-13T09:14:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.803126+0000", 2024-09-13T09:14:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025320070000000002 2024-09-13T09:14:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.803177+0000", 2024-09-13T09:14:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0513000000000002e-05 2024-09-13T09:14:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.803206+0000", 2024-09-13T09:14:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9747000000000002e-05 2024-09-13T09:14:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:30611 3.7 3:e0345511:::rbd_data.1123b259dc65.000000000000000b:head [delete] snapc e=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-13T09:14:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:23.893039+0000", 2024-09-13T09:14:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.9658103870000003, 2024-09-13T09:14:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003465072, 2024-09-13T09:14:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 30611 2024-09-13T09:14:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893039+0000", 2024-09-13T09:14:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893039+0000", 2024-09-13T09:14:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893038+0000", 2024-09-13T09:14:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893043+0000", 2024-09-13T09:14:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3739999999999997e-06 2024-09-13T09:14:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893055+0000", 2024-09-13T09:14:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2138e-05 2024-09-13T09:14:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893083+0000", 2024-09-13T09:14:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7589999999999998e-05 2024-09-13T09:14:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893158+0000", 2024-09-13T09:14:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5318e-05 2024-09-13T09:14:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893343+0000", 2024-09-13T09:14:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000184747 2024-09-13T09:14:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:14:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893440+0000", 2024-09-13T09:14:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6751999999999998e-05 2024-09-13T09:14:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.896088+0000", 2024-09-13T09:14:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026486370000000001 2024-09-13T09:14:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.896422+0000", 2024-09-13T09:14:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000333526 2024-09-13T09:14:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.896467+0000", 2024-09-13T09:14:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5439000000000003e-05 2024-09-13T09:14:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.896504+0000", 2024-09-13T09:14:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6547000000000001e-05 2024-09-13T09:14:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:29.958 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:29.959 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:29.957+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:29.959 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:14:30.060 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:30.058+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:30.107 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:30.107 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:30.115 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:14:30.161 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:30.159+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:30.261 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:30.260+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:30.267 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:30.268 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:30.268 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:30.268 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:30.268 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:30.278 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:14:30.362 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:30.361+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:30.395 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.863932252, 2024-09-13T09:14:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:14:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:14:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:14:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:14:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:14:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:14:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:14:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:14:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:14:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:14:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:14:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:14:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:14:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:14:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:14:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:14:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:14:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:14:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:14:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:14:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:14:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:14:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:14:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.863914217, 2024-09-13T09:14:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:14:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:14:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:14:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:14:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:14:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:14:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:14:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:14:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:14:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:14:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:14:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:14:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:14:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:14:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:14:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:14:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:14:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:14:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:14:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:14:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:14:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:14:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.86389754699999, 2024-09-13T09:14:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:14:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:14:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:14:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:14:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:14:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:14:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:14:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:14:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:14:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:14:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:14:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:14:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:14:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:14:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:14:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:14:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:14:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:14:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:14:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:14:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:14:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:14:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.863879915, 2024-09-13T09:14:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:14:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:14:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:14:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:14:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:14:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:14:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:14:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:14:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:14:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:14:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:14:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:14:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:14:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:14:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:14:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:14:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:14:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:14:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:14:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:14:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:14:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:14:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.86383361, 2024-09-13T09:14:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:14:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:14:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:14:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:14:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:14:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:14:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:14:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:14:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:14:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:14:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:14:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:14:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:14:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:14:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:14:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:14:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:14:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:14:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:14:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:14:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:14:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:14:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:14:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.86380830900001, 2024-09-13T09:14:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:14:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:14:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:14:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:14:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:14:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:14:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:14:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:14:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:14:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:14:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:14:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:14:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:14:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:14:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:14:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:14:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:14:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:14:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:14:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:14:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:14:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:14:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:14:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.86379008, 2024-09-13T09:14:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:14:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:14:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:14:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:14:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:14:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:14:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:14:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:14:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:14:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:14:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:14:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:14:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:14:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:14:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:14:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:14:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:14:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:14:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:14:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:14:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:14:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:14:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.863742761, 2024-09-13T09:14:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:14:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:14:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:14:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:14:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:14:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:14:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:14:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:14:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:14:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:14:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:14:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:14:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:14:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:14:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:14:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:14:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:14:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:14:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:14:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:14:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:14:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:14:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.863228523, 2024-09-13T09:14:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:14:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:14:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:14:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:14:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:14:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:14:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:14:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:14:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:14:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:14:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:14:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:14:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:14:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:14:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:14:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:14:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:14:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:14:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:14:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:14:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:14:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:14:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:14:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.863193993, 2024-09-13T09:14:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:14:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:14:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:14:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:14:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:14:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:14:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:14:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:14:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:14:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:14:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:14:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:14:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:14:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:14:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:14:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:14:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:14:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:14:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:14:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:14:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:14:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:14:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.863176086, 2024-09-13T09:14:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:14:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:14:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:14:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:14:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:14:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:14:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:14:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:14:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:14:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:14:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:14:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:14:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:14:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:14:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:14:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:14:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:14:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:14:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:14:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:14:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:14:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:14:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.854049238, 2024-09-13T09:14:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:14:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:14:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:14:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:14:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:14:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:14:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:14:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:14:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:14:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:14:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:14:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:14:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:14:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:14:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:14:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:14:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:14:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:14:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:14:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:14:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:14:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:14:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.853997658, 2024-09-13T09:14:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:14:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:14:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:14:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:14:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:14:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:14:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:14:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:14:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:14:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:14:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:14:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:14:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:14:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:14:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:14:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:14:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:14:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:14:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:14:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:14:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:14:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:14:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:14:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.853981382, 2024-09-13T09:14:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:14:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:14:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:14:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:14:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:14:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:14:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:14:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:14:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:14:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:14:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:14:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:14:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:14:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:14:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:14:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:14:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:14:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:14:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:14:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:14:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:14:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:14:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:14:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.853965608, 2024-09-13T09:14:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:14:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:14:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:14:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:14:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:14:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:14:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:14:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:14:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:14:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:14:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:14:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:14:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:14:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:14:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:14:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:14:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:14:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:14:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:14:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:14:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:14:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:14:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:14:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.853950733, 2024-09-13T09:14:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:14:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:14:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:14:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:14:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:14:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:14:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:14:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:14:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:14:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:14:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:14:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:14:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:14:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:14:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:14:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:14:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:14:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:14:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:14:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:14:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:14:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:14:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.853933271, 2024-09-13T09:14:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:14:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:14:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:14:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:14:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:14:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:14:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:14:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:14:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:14:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:14:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:14:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:14:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:14:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:14:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:14:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:14:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:14:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:14:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:14:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:14:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:14:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:14:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:14:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.852332639, 2024-09-13T09:14:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:14:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:14:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:14:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:14:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:14:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:14:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:14:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:14:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:14:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:14:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:14:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:14:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:14:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:14:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:14:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:14:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:14:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:14:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:14:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:14:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:14:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:14:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:14:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.852292588, 2024-09-13T09:14:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:14:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:14:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:14:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:14:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:14:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:14:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:14:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:14:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:14:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:14:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:14:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:14:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:14:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:14:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:14:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:14:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:14:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:14:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:14:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:14:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:14:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:14:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:14:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 107.852248825, 2024-09-13T09:14:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:14:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:14:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:14:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:14:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:14:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:14:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:14:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:14:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:14:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:14:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:14:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:14:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:14:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:14:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:14:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:14:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:14:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:14:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:14:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:14:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:14:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:14:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.514 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:30.515 INFO:teuthology.orchestra.run.smithi129.stdout: marking collection for removal 2024-09-13T09:14:30.515 INFO:teuthology.orchestra.run.smithi129.stdout:setting '_remove' omap key 2024-09-13T09:14:30.516 INFO:teuthology.orchestra.run.smithi129.stdout:finish_remove_pgs 3.2_head removing 3.2 2024-09-13T09:14:30.516 INFO:teuthology.orchestra.run.smithi129.stdout:Remove successful 2024-09-13T09:14:30.516 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:30.514+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:30.516 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:14:30.616 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:30.614+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:30.664 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:30.664 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:30.672 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:14:30.757 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:30.755+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:30.818 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:30.816+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:30.819 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:30.820 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:30.820 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:30.820 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:30.820 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:30.828 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:14:30.918 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:30.917+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:30.939 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.2.5 2024-09-13T09:14:30.977 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:30.978 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:30.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:30.978 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:30.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.978 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20306 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6166528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.978 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:30.978 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.472110904, 2024-09-13T09:14:30.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050539153000000003, 2024-09-13T09:14:30.978 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.978 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.978 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.978 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20306 2024-09-13T09:14:30.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.979 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:30.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503578+0000", 2024-09-13T09:14:30.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503577+0000", 2024-09-13T09:14:30.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503584+0000", 2024-09-13T09:14:30.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0149999999999998e-06 2024-09-13T09:14:30.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503591+0000", 2024-09-13T09:14:30.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6789999999999999e-06 2024-09-13T09:14:30.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503600+0000", 2024-09-13T09:14:30.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4450000000000002e-06 2024-09-13T09:14:30.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554026+0000", 2024-09-13T09:14:30.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050425800999999999 2024-09-13T09:14:30.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554066+0000", 2024-09-13T09:14:30.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9808000000000002e-05 2024-09-13T09:14:30.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554117+0000", 2024-09-13T09:14:30.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0794000000000002e-05 2024-09-13T09:14:30.982 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.982 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.982 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.982 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20307 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6189056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.982 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:30.982 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.47208322, 2024-09-13T09:14:30.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050620236999999998, 2024-09-13T09:14:30.983 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.983 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.983 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.983 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.983 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.983 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20307 2024-09-13T09:14:30.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.983 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:30.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:30.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503605+0000", 2024-09-13T09:14:30.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503607+0000", 2024-09-13T09:14:30.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4490000000000002e-06 2024-09-13T09:14:30.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503615+0000", 2024-09-13T09:14:30.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3039999999999999e-06 2024-09-13T09:14:30.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503625+0000", 2024-09-13T09:14:30.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:14:30.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554140+0000", 2024-09-13T09:14:30.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050515160000000003 2024-09-13T09:14:30.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554179+0000", 2024-09-13T09:14:30.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8605000000000001e-05 2024-09-13T09:14:30.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554225+0000", 2024-09-13T09:14:30.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6825e-05 2024-09-13T09:14:30.987 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.987 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.987 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.987 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20308 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6211584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.987 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:30.987 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.472058879, 2024-09-13T09:14:30.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050709265000000003, 2024-09-13T09:14:30.987 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.987 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.987 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.988 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.988 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.988 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20308 2024-09-13T09:14:30.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.988 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:30.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503630+0000", 2024-09-13T09:14:30.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503629+0000", 2024-09-13T09:14:30.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503632+0000", 2024-09-13T09:14:30.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.368e-06 2024-09-13T09:14:30.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503638+0000", 2024-09-13T09:14:30.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.224e-06 2024-09-13T09:14:30.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503642+0000", 2024-09-13T09:14:30.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2629999999999997e-06 2024-09-13T09:14:30.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554248+0000", 2024-09-13T09:14:30.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050605794000000003 2024-09-13T09:14:30.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554287+0000", 2024-09-13T09:14:30.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8989999999999998e-05 2024-09-13T09:14:30.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554339+0000", 2024-09-13T09:14:30.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2061999999999997e-05 2024-09-13T09:14:30.991 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.991 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.991 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.991 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20309 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6234112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.992 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:30.992 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.472042559, 2024-09-13T09:14:30.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050800769000000003, 2024-09-13T09:14:30.992 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.992 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.992 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.992 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.992 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.992 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20309 2024-09-13T09:14:30.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.992 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:30.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:30.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503646+0000", 2024-09-13T09:14:30.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503648+0000", 2024-09-13T09:14:30.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:30.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503654+0000", 2024-09-13T09:14:30.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0390000000000001e-06 2024-09-13T09:14:30.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503658+0000", 2024-09-13T09:14:30.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5129999999999998e-06 2024-09-13T09:14:30.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:30.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554361+0000", 2024-09-13T09:14:30.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050703204000000002 2024-09-13T09:14:30.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:30.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554401+0000", 2024-09-13T09:14:30.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9084000000000001e-05 2024-09-13T09:14:30.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:30.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554447+0000", 2024-09-13T09:14:30.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6105e-05 2024-09-13T09:14:30.996 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.996 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:30.996 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:30.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.996 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20310 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6256640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.472026144, 2024-09-13T09:14:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050897513999999998, 2024-09-13T09:14:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20310 2024-09-13T09:14:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:30.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503662+0000", 2024-09-13T09:14:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503664+0000", 2024-09-13T09:14:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.283e-06 2024-09-13T09:14:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503670+0000", 2024-09-13T09:14:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1279999999999996e-06 2024-09-13T09:14:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503674+0000", 2024-09-13T09:14:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1180000000000002e-06 2024-09-13T09:14:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:30.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554471+0000", 2024-09-13T09:14:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050796270999999997 2024-09-13T09:14:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554510+0000", 2024-09-13T09:14:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9375000000000002e-05 2024-09-13T09:14:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554560+0000", 2024-09-13T09:14:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9712000000000001e-05 2024-09-13T09:14:31.000 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20311 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.472010036, 2024-09-13T09:14:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050987693000000001, 2024-09-13T09:14:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.001 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20311 2024-09-13T09:14:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503678+0000", 2024-09-13T09:14:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503680+0000", 2024-09-13T09:14:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.903e-06 2024-09-13T09:14:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503686+0000", 2024-09-13T09:14:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:14:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503690+0000", 2024-09-13T09:14:31.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1790000000000001e-06 2024-09-13T09:14:31.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554583+0000", 2024-09-13T09:14:31.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050893401999999997 2024-09-13T09:14:31.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554620+0000", 2024-09-13T09:14:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7200999999999998e-05 2024-09-13T09:14:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554666+0000", 2024-09-13T09:14:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5760000000000002e-05 2024-09-13T09:14:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20393 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5244928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.471994475, 2024-09-13T09:14:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051084592999999998, 2024-09-13T09:14:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20393 2024-09-13T09:14:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503694+0000", 2024-09-13T09:14:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503696+0000", 2024-09-13T09:14:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1789999999999998e-06 2024-09-13T09:14:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503701+0000", 2024-09-13T09:14:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:14:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503706+0000", 2024-09-13T09:14:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:14:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554689+0000", 2024-09-13T09:14:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050983839000000003 2024-09-13T09:14:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554729+0000", 2024-09-13T09:14:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9706999999999998e-05 2024-09-13T09:14:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554779+0000", 2024-09-13T09:14:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9453000000000002e-05 2024-09-13T09:14:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.009 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20394 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5267456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.47197763699999, 2024-09-13T09:14:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051196744000000002, 2024-09-13T09:14:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20394 2024-09-13T09:14:31.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503711+0000", 2024-09-13T09:14:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:31.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503710+0000", 2024-09-13T09:14:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503712+0000", 2024-09-13T09:14:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04e-06 2024-09-13T09:14:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503718+0000", 2024-09-13T09:14:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:14:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503723+0000", 2024-09-13T09:14:31.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:14:31.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554802+0000", 2024-09-13T09:14:31.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051079503999999998 2024-09-13T09:14:31.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554861+0000", 2024-09-13T09:14:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8603000000000001e-05 2024-09-13T09:14:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554908+0000", 2024-09-13T09:14:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6879999999999998e-05 2024-09-13T09:14:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20395 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5289984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:31.014 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.47196164499999, 2024-09-13T09:14:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051299089999999999, 2024-09-13T09:14:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20395 2024-09-13T09:14:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503727+0000", 2024-09-13T09:14:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503726+0000", 2024-09-13T09:14:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503735+0000", 2024-09-13T09:14:31.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6489999999999997e-06 2024-09-13T09:14:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503744+0000", 2024-09-13T09:14:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0489999999999993e-06 2024-09-13T09:14:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503749+0000", 2024-09-13T09:14:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-09-13T09:14:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554933+0000", 2024-09-13T09:14:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051183764999999999 2024-09-13T09:14:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.554974+0000", 2024-09-13T09:14:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1006e-05 2024-09-13T09:14:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555026+0000", 2024-09-13T09:14:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2000999999999999e-05 2024-09-13T09:14:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.018 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20396 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5312512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.471934147, 2024-09-13T09:14:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051377776999999999, 2024-09-13T09:14:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20396 2024-09-13T09:14:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.019 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503754+0000", 2024-09-13T09:14:31.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503756+0000", 2024-09-13T09:14:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5670000000000002e-06 2024-09-13T09:14:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503763+0000", 2024-09-13T09:14:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2060000000000004e-06 2024-09-13T09:14:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503775+0000", 2024-09-13T09:14:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2797e-05 2024-09-13T09:14:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555049+0000", 2024-09-13T09:14:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051274005999999997 2024-09-13T09:14:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555088+0000", 2024-09-13T09:14:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8226000000000003e-05 2024-09-13T09:14:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555132+0000", 2024-09-13T09:14:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.439e-05 2024-09-13T09:14:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20397 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5335040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.47190828799999, 2024-09-13T09:14:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051463363999999998, 2024-09-13T09:14:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:31.023 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20397 2024-09-13T09:14:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:31.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503780+0000", 2024-09-13T09:14:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503782+0000", 2024-09-13T09:14:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.43e-06 2024-09-13T09:14:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503788+0000", 2024-09-13T09:14:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-09-13T09:14:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503793+0000", 2024-09-13T09:14:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4889999999999997e-06 2024-09-13T09:14:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555155+0000", 2024-09-13T09:14:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051362170999999998 2024-09-13T09:14:31.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555194+0000", 2024-09-13T09:14:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8986999999999998e-05 2024-09-13T09:14:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555244+0000", 2024-09-13T09:14:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9704999999999999e-05 2024-09-13T09:14:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.028 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20398 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5357568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:31.028 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:31.028 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.47186679799999, 2024-09-13T09:14:31.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051527569000000002, 2024-09-13T09:14:31.028 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:31.028 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:31.028 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:31.028 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:31.028 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.028 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20398 2024-09-13T09:14:31.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.028 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:31.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:31.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:31.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:31.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503822+0000", 2024-09-13T09:14:31.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:31.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503821+0000", 2024-09-13T09:14:31.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:31.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503824+0000", 2024-09-13T09:14:31.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.932e-06 2024-09-13T09:14:31.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:31.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503831+0000", 2024-09-13T09:14:31.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:14:31.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503837+0000", 2024-09-13T09:14:31.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.395e-06 2024-09-13T09:14:31.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555267+0000", 2024-09-13T09:14:31.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051429892999999997 2024-09-13T09:14:31.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555305+0000", 2024-09-13T09:14:31.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8209e-05 2024-09-13T09:14:31.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555349+0000", 2024-09-13T09:14:31.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4372000000000001e-05 2024-09-13T09:14:31.032 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.032 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.032 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.032 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20399 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5380096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:31.032 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:31.032 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.47184556000001, 2024-09-13T09:14:31.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051619542999999997, 2024-09-13T09:14:31.032 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:31.032 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:31.032 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:31.032 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:31.033 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.033 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20399 2024-09-13T09:14:31.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.033 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:31.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:31.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:31.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:31.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:31.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:31.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503843+0000", 2024-09-13T09:14:31.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:31.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503849+0000", 2024-09-13T09:14:31.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:14:31.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:31.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503856+0000", 2024-09-13T09:14:31.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:14:31.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503861+0000", 2024-09-13T09:14:31.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0370000000000002e-06 2024-09-13T09:14:31.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555373+0000", 2024-09-13T09:14:31.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.05151149 2024-09-13T09:14:31.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555413+0000", 2024-09-13T09:14:31.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0059999999999999e-05 2024-09-13T09:14:31.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.036 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.036 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555462+0000", 2024-09-13T09:14:31.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9531999999999999e-05 2024-09-13T09:14:31.036 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.036 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.036 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.036 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20400 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5402624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:31.036 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:31.036 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.47182275999999, 2024-09-13T09:14:31.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051704973000000001, 2024-09-13T09:14:31.037 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:31.037 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:31.037 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:31.037 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:31.037 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.037 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20400 2024-09-13T09:14:31.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.037 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:31.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:31.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:31.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:31.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503866+0000", 2024-09-13T09:14:31.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:31.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503865+0000", 2024-09-13T09:14:31.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:31.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503868+0000", 2024-09-13T09:14:31.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2800000000000002e-06 2024-09-13T09:14:31.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:31.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503874+0000", 2024-09-13T09:14:31.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:14:31.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503878+0000", 2024-09-13T09:14:31.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:14:31.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555486+0000", 2024-09-13T09:14:31.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051607627000000003 2024-09-13T09:14:31.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555525+0000", 2024-09-13T09:14:31.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.981e-05 2024-09-13T09:14:31.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555571+0000", 2024-09-13T09:14:31.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5253999999999997e-05 2024-09-13T09:14:31.040 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.040 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.040 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.041 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20401 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5425152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:31.041 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:31.041 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.471806304, 2024-09-13T09:14:31.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051798578999999997, 2024-09-13T09:14:31.041 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:31.041 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:31.041 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:31.041 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:31.041 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.041 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20401 2024-09-13T09:14:31.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.041 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:31.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:31.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:31.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:31.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:31.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:31.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503882+0000", 2024-09-13T09:14:31.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:31.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503884+0000", 2024-09-13T09:14:31.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:14:31.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:31.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503890+0000", 2024-09-13T09:14:31.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6189999999999999e-06 2024-09-13T09:14:31.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503894+0000", 2024-09-13T09:14:31.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:14:31.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555592+0000", 2024-09-13T09:14:31.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051698243999999997 2024-09-13T09:14:31.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555631+0000", 2024-09-13T09:14:31.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8656999999999999e-05 2024-09-13T09:14:31.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555681+0000", 2024-09-13T09:14:31.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9892999999999998e-05 2024-09-13T09:14:31.045 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.045 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.045 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.046 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20402 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:31.046 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:31.046 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.471561167, 2024-09-13T09:14:31.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051659098000000001, 2024-09-13T09:14:31.046 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:31.046 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:31.046 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:31.046 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:31.046 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.046 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20402 2024-09-13T09:14:31.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.046 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:31.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504127+0000", 2024-09-13T09:14:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.503898+0000", 2024-09-13T09:14:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9997702 2024-09-13T09:14:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504131+0000", 2024-09-13T09:14:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023328499999999999 2024-09-13T09:14:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504141+0000", 2024-09-13T09:14:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0202e-05 2024-09-13T09:14:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.504147+0000", 2024-09-13T09:14:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0129999999999999e-06 2024-09-13T09:14:31.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555704+0000", 2024-09-13T09:14:31.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.051557139000000002 2024-09-13T09:14:31.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555742+0000", 2024-09-13T09:14:31.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8099000000000002e-05 2024-09-13T09:14:31.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.555786+0000", 2024-09-13T09:14:31.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.409e-05 2024-09-13T09:14:31.050 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.050 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.050 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.050 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20409 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5605376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:31.050 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:31.050 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.469675608, 2024-09-13T09:14:31.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050553506999999998, 2024-09-13T09:14:31.050 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:31.050 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:31.050 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:31.050 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:31.050 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.051 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20409 2024-09-13T09:14:31.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.051 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:31.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:31.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:31.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:31.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506013+0000", 2024-09-13T09:14:31.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:31.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506012+0000", 2024-09-13T09:14:31.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:31.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506018+0000", 2024-09-13T09:14:31.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7790000000000001e-06 2024-09-13T09:14:31.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:31.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506025+0000", 2024-09-13T09:14:31.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.533e-06 2024-09-13T09:14:31.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506030+0000", 2024-09-13T09:14:31.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:14:31.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556478+0000", 2024-09-13T09:14:31.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050448422 2024-09-13T09:14:31.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556517+0000", 2024-09-13T09:14:31.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8670000000000001e-05 2024-09-13T09:14:31.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556566+0000", 2024-09-13T09:14:31.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9577000000000003e-05 2024-09-13T09:14:31.054 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.054 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.054 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.054 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20410 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5627904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:31.054 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:31.054 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.469654242, 2024-09-13T09:14:31.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050637412, 2024-09-13T09:14:31.054 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20410 2024-09-13T09:14:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:31.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:31.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:31.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506034+0000", 2024-09-13T09:14:31.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:31.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506036+0000", 2024-09-13T09:14:31.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.198e-06 2024-09-13T09:14:31.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:31.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506042+0000", 2024-09-13T09:14:31.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7520000000000002e-06 2024-09-13T09:14:31.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506053+0000", 2024-09-13T09:14:31.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1528999999999999e-05 2024-09-13T09:14:31.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556590+0000", 2024-09-13T09:14:31.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050536244000000001 2024-09-13T09:14:31.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556627+0000", 2024-09-13T09:14:31.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7221000000000002e-05 2024-09-13T09:14:31.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556672+0000", 2024-09-13T09:14:31.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4848999999999997e-05 2024-09-13T09:14:31.058 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.058 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.058 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.058 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20411 3.19 3:9f9eae7f:::rbd_data.1123d4249472.0000000000000002:10 [read 5650432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:31.058 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:31.059 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 108.46944887399999, 2024-09-13T09:14:31.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050544089, 2024-09-13T09:14:31.059 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:31.059 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:31.059 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:31.059 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:31.059 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.059 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20411 2024-09-13T09:14:31.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.059 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:31.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:31.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:31.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:31.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506240+0000", 2024-09-13T09:14:31.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:31.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506058+0000", 2024-09-13T09:14:31.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9998188 2024-09-13T09:14:31.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:31.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506245+0000", 2024-09-13T09:14:31.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018646 2024-09-13T09:14:31.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:31.061 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506257+0000", 2024-09-13T09:14:31.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2187999999999999e-05 2024-09-13T09:14:31.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.061 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.506265+0000", 2024-09-13T09:14:31.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:14:31.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556695+0000", 2024-09-13T09:14:31.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.050430062999999997 2024-09-13T09:14:31.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556734+0000", 2024-09-13T09:14:31.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:14:31.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.062 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.062 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.556784+0000", 2024-09-13T09:14:31.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9307000000000001e-05 2024-09-13T09:14:31.062 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.062 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.063 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.063 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.5272.0:1775 3.0 3:02a42fca:::rbd_data.11235782ee8d.0000000000000030:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2244608~40448] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:14:31.063 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:31.063 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 80.727907032999994, 2024-09-13T09:14:31.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.666843732, 2024-09-13T09:14:31.063 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:31.063 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:31.063 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:31.063 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.5272", 2024-09-13T09:14:31.063 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4089902097", 2024-09-13T09:14:31.063 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1775 2024-09-13T09:14:31.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.064 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:31.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:31.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:31.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:31.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247781+0000", 2024-09-13T09:14:31.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:31.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:31.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247780+0000", 2024-09-13T09:14:31.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:31.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:31.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247865+0000", 2024-09-13T09:14:31.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4359999999999999e-05 2024-09-13T09:14:31.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:31.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247876+0000", 2024-09-13T09:14:31.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1456e-05 2024-09-13T09:14:31.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247883+0000", 2024-09-13T09:14:31.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:14:31.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247926+0000", 2024-09-13T09:14:31.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3203e-05 2024-09-13T09:14:31.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:10.247932+0000", 2024-09-13T09:14:31.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:14:31.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909639+0000", 2024-09-13T09:14:31.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6617069099999999 2024-09-13T09:14:31.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:31.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.909828+0000", 2024-09-13T09:14:31.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000189075 2024-09-13T09:14:31.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:31.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911676+0000", 2024-09-13T09:14:31.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001847235 2024-09-13T09:14:31.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:31.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911765+0000", 2024-09-13T09:14:31.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9530000000000005e-05 2024-09-13T09:14:31.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 5", 2024-09-13T09:14:31.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.911901+0000", 2024-09-13T09:14:31.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013603899999999999 2024-09-13T09:14:31.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.068 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:31.068 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.912889+0000", 2024-09-13T09:14:31.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00098807400000000003 2024-09-13T09:14:31.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:31.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914438+0000", 2024-09-13T09:14:31.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015491070000000001 2024-09-13T09:14:31.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:31.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914505+0000", 2024-09-13T09:14:31.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:14:31.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:31.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:31.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:31.070 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:13:11.914625+0000", 2024-09-13T09:14:31.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000120108 2024-09-13T09:14:31.070 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.070 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.070 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.070 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:31.070 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:31.070 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:31.072 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:14:31.072 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:31.069+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:31.173 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:31.171+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:31.181 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:31.181 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:31.181 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:31.181 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:31.190 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:14:31.273 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:31.271+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:31.339 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:31.340 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:31.340 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:31.340 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:31.340 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:31.349 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:14:31.393 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:31.391+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:31.446 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:14:31.449 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:14:31.449 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:14:31.461 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19587 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.937453658, 2024-09-13T09:14:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0129272119999999, 2024-09-13T09:14:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19587 2024-09-13T09:14:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521168+0000", 2024-09-13T09:14:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521170+0000", 2024-09-13T09:14:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.733e-06 2024-09-13T09:14:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521174+0000", 2024-09-13T09:14:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3579999999999996e-06 2024-09-13T09:14:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521177+0000", 2024-09-13T09:14:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9079999999999999e-06 2024-09-13T09:14:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547707+0000", 2024-09-13T09:14:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026529697000000001 2024-09-13T09:14:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547711+0000", 2024-09-13T09:14:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0690000000000003e-06 2024-09-13T09:14:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534088+0000", 2024-09-13T09:14:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98637694300000001 2024-09-13T09:14:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534096+0000", 2024-09-13T09:14:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7849999999999992e-06 2024-09-13T09:14:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19588 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.937404602, 2024-09-13T09:14:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012902776, 2024-09-13T09:14:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19588 2024-09-13T09:14:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521217+0000", 2024-09-13T09:14:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521219+0000", 2024-09-13T09:14:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:14:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521224+0000", 2024-09-13T09:14:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1509999999999997e-06 2024-09-13T09:14:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521227+0000", 2024-09-13T09:14:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.287e-06 2024-09-13T09:14:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547795+0000", 2024-09-13T09:14:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026567989 2024-09-13T09:14:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547809+0000", 2024-09-13T09:14:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4676e-05 2024-09-13T09:14:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534113+0000", 2024-09-13T09:14:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986303449 2024-09-13T09:14:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534120+0000", 2024-09-13T09:14:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:14:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19589 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.937364527, 2024-09-13T09:14:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128864550000001, 2024-09-13T09:14:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19589 2024-09-13T09:14:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521257+0000", 2024-09-13T09:14:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521259+0000", 2024-09-13T09:14:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.962e-06 2024-09-13T09:14:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521264+0000", 2024-09-13T09:14:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:14:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521266+0000", 2024-09-13T09:14:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7930000000000002e-06 2024-09-13T09:14:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547846+0000", 2024-09-13T09:14:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026579535000000001 2024-09-13T09:14:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547852+0000", 2024-09-13T09:14:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-06 2024-09-13T09:14:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534137+0000", 2024-09-13T09:14:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628549300000001 2024-09-13T09:14:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534144+0000", 2024-09-13T09:14:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:14:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19590 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.93733471199999, 2024-09-13T09:14:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128803770000001, 2024-09-13T09:14:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19590 2024-09-13T09:14:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521287+0000", 2024-09-13T09:14:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521291+0000", 2024-09-13T09:14:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.766e-06 2024-09-13T09:14:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521295+0000", 2024-09-13T09:14:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8079999999999998e-06 2024-09-13T09:14:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521298+0000", 2024-09-13T09:14:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5260000000000002e-06 2024-09-13T09:14:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547892+0000", 2024-09-13T09:14:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02659398 2024-09-13T09:14:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547898+0000", 2024-09-13T09:14:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1870000000000002e-06 2024-09-13T09:14:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534161+0000", 2024-09-13T09:14:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626253500000005 2024-09-13T09:14:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534168+0000", 2024-09-13T09:14:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:14:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19591 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.937305051, 2024-09-13T09:14:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128754040000001, 2024-09-13T09:14:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19591 2024-09-13T09:14:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521317+0000", 2024-09-13T09:14:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521319+0000", 2024-09-13T09:14:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.897e-06 2024-09-13T09:14:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521323+0000", 2024-09-13T09:14:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:14:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521326+0000", 2024-09-13T09:14:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.146e-06 2024-09-13T09:14:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547940+0000", 2024-09-13T09:14:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026613695999999999 2024-09-13T09:14:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547947+0000", 2024-09-13T09:14:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8619999999999997e-06 2024-09-13T09:14:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534186+0000", 2024-09-13T09:14:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623897100000002 2024-09-13T09:14:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534192+0000", 2024-09-13T09:14:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7379999999999997e-06 2024-09-13T09:14:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19592 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.937275844, 2024-09-13T09:14:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012870438, 2024-09-13T09:14:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19592 2024-09-13T09:14:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521346+0000", 2024-09-13T09:14:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521348+0000", 2024-09-13T09:14:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.996e-06 2024-09-13T09:14:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521352+0000", 2024-09-13T09:14:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:14:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521355+0000", 2024-09-13T09:14:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0489999999999999e-06 2024-09-13T09:14:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547977+0000", 2024-09-13T09:14:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026621694000000001 2024-09-13T09:14:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.547980+0000", 2024-09-13T09:14:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0299999999999998e-06 2024-09-13T09:14:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534210+0000", 2024-09-13T09:14:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623008300000004 2024-09-13T09:14:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534217+0000", 2024-09-13T09:14:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9090000000000003e-06 2024-09-13T09:14:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19593 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.93721966299999, 2024-09-13T09:14:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128429489999999, 2024-09-13T09:14:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19593 2024-09-13T09:14:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521402+0000", 2024-09-13T09:14:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521403+0000", 2024-09-13T09:14:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4479999999999999e-06 2024-09-13T09:14:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521407+0000", 2024-09-13T09:14:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8859999999999997e-06 2024-09-13T09:14:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521410+0000", 2024-09-13T09:14:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.553e-06 2024-09-13T09:14:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548000+0000", 2024-09-13T09:14:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026589741 2024-09-13T09:14:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548005+0000", 2024-09-13T09:14:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:14:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534238+0000", 2024-09-13T09:14:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623317600000004 2024-09-13T09:14:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534245+0000", 2024-09-13T09:14:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1579999999999999e-06 2024-09-13T09:14:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19594 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.937166202, 2024-09-13T09:14:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0128176440000001, 2024-09-13T09:14:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19594 2024-09-13T09:14:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521456+0000", 2024-09-13T09:14:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521455+0000", 2024-09-13T09:14:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521458+0000", 2024-09-13T09:14:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0839999999999999e-06 2024-09-13T09:14:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521462+0000", 2024-09-13T09:14:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:14:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521465+0000", 2024-09-13T09:14:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.615e-06 2024-09-13T09:14:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548026+0000", 2024-09-13T09:14:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026561194 2024-09-13T09:14:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548029+0000", 2024-09-13T09:14:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:14:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534265+0000", 2024-09-13T09:14:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98623640499999998 2024-09-13T09:14:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534273+0000", 2024-09-13T09:14:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0639999999999994e-06 2024-09-13T09:14:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19595 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.93710937199999, 2024-09-13T09:14:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127964279999999, 2024-09-13T09:14:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19595 2024-09-13T09:14:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521513+0000", 2024-09-13T09:14:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521512+0000", 2024-09-13T09:14:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521514+0000", 2024-09-13T09:14:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.86e-06 2024-09-13T09:14:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521518+0000", 2024-09-13T09:14:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0520000000000003e-06 2024-09-13T09:14:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521521+0000", 2024-09-13T09:14:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1420000000000002e-06 2024-09-13T09:14:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548045+0000", 2024-09-13T09:14:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026524082000000001 2024-09-13T09:14:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548048+0000", 2024-09-13T09:14:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7010000000000001e-06 2024-09-13T09:14:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534303+0000", 2024-09-13T09:14:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986254874 2024-09-13T09:14:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534309+0000", 2024-09-13T09:14:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:14:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19596 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.937058345, 2024-09-13T09:14:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127661269999999, 2024-09-13T09:14:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19596 2024-09-13T09:14:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521564+0000", 2024-09-13T09:14:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521563+0000", 2024-09-13T09:14:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521565+0000", 2024-09-13T09:14:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9240000000000001e-06 2024-09-13T09:14:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521569+0000", 2024-09-13T09:14:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0509999999999998e-06 2024-09-13T09:14:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521573+0000", 2024-09-13T09:14:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6049999999999998e-06 2024-09-13T09:14:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548065+0000", 2024-09-13T09:14:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026492056 2024-09-13T09:14:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548068+0000", 2024-09-13T09:14:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:14:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534322+0000", 2024-09-13T09:14:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625342100000002 2024-09-13T09:14:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534330+0000", 2024-09-13T09:14:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:14:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19597 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.93701715500001, 2024-09-13T09:14:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127404289999999, 2024-09-13T09:14:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19597 2024-09-13T09:14:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521605+0000", 2024-09-13T09:14:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521606+0000", 2024-09-13T09:14:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.547e-06 2024-09-13T09:14:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521610+0000", 2024-09-13T09:14:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9269999999999998e-06 2024-09-13T09:14:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521613+0000", 2024-09-13T09:14:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5110000000000002e-06 2024-09-13T09:14:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548085+0000", 2024-09-13T09:14:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026472822 2024-09-13T09:14:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548089+0000", 2024-09-13T09:14:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5899999999999999e-06 2024-09-13T09:14:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534341+0000", 2024-09-13T09:14:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625197899999995 2024-09-13T09:14:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534345+0000", 2024-09-13T09:14:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2919999999999997e-06 2024-09-13T09:14:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19598 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.936975039, 2024-09-13T09:14:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012721695, 2024-09-13T09:14:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19598 2024-09-13T09:14:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521647+0000", 2024-09-13T09:14:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521648+0000", 2024-09-13T09:14:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3659999999999999e-06 2024-09-13T09:14:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521652+0000", 2024-09-13T09:14:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-06 2024-09-13T09:14:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521655+0000", 2024-09-13T09:14:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5490000000000001e-06 2024-09-13T09:14:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548107+0000", 2024-09-13T09:14:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026452099 2024-09-13T09:14:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548110+0000", 2024-09-13T09:14:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4029999999999999e-06 2024-09-13T09:14:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534359+0000", 2024-09-13T09:14:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.986248758 2024-09-13T09:14:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534369+0000", 2024-09-13T09:14:31.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:14:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19599 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.93693116, 2024-09-13T09:14:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0127035049999999, 2024-09-13T09:14:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19599 2024-09-13T09:14:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521691+0000", 2024-09-13T09:14:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521692+0000", 2024-09-13T09:14:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.418e-06 2024-09-13T09:14:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521696+0000", 2024-09-13T09:14:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8789999999999996e-06 2024-09-13T09:14:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521698+0000", 2024-09-13T09:14:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5799999999999999e-06 2024-09-13T09:14:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548129+0000", 2024-09-13T09:14:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026430204999999998 2024-09-13T09:14:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548131+0000", 2024-09-13T09:14:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7e-06 2024-09-13T09:14:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534386+0000", 2024-09-13T09:14:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625454499999998 2024-09-13T09:14:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534394+0000", 2024-09-13T09:14:31.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4139999999999994e-06 2024-09-13T09:14:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19600 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.93688524300001, 2024-09-13T09:14:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012675285, 2024-09-13T09:14:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19600 2024-09-13T09:14:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521737+0000", 2024-09-13T09:14:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521736+0000", 2024-09-13T09:14:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521738+0000", 2024-09-13T09:14:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8470000000000001e-06 2024-09-13T09:14:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521742+0000", 2024-09-13T09:14:31.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:14:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521746+0000", 2024-09-13T09:14:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:14:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548149+0000", 2024-09-13T09:14:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026403638 2024-09-13T09:14:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548152+0000", 2024-09-13T09:14:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:14:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534407+0000", 2024-09-13T09:14:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625543100000002 2024-09-13T09:14:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534412+0000", 2024-09-13T09:14:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:14:31.531 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19601 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.936835472, 2024-09-13T09:14:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0126467960000001, 2024-09-13T09:14:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.532 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19601 2024-09-13T09:14:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521786+0000", 2024-09-13T09:14:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521788+0000", 2024-09-13T09:14:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:14:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521792+0000", 2024-09-13T09:14:31.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8809999999999998e-06 2024-09-13T09:14:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521795+0000", 2024-09-13T09:14:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6589999999999999e-06 2024-09-13T09:14:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548169+0000", 2024-09-13T09:14:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026374473999999998 2024-09-13T09:14:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548172+0000", 2024-09-13T09:14:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:14:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534429+0000", 2024-09-13T09:14:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98625671000000004 2024-09-13T09:14:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534433+0000", 2024-09-13T09:14:31.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:14:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19602 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.93679006799999, 2024-09-13T09:14:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012636841, 2024-09-13T09:14:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.537 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.538 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.538 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19602 2024-09-13T09:14:31.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.538 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:31.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:31.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521832+0000", 2024-09-13T09:14:31.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:31.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521838+0000", 2024-09-13T09:14:31.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1770000000000003e-06 2024-09-13T09:14:31.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521842+0000", 2024-09-13T09:14:31.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:14:31.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521845+0000", 2024-09-13T09:14:31.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7300000000000001e-06 2024-09-13T09:14:31.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548188+0000", 2024-09-13T09:14:31.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026343437000000001 2024-09-13T09:14:31.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548191+0000", 2024-09-13T09:14:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6230000000000001e-06 2024-09-13T09:14:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534460+0000", 2024-09-13T09:14:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626892600000005 2024-09-13T09:14:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534469+0000", 2024-09-13T09:14:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7860000000000002e-06 2024-09-13T09:14:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19603 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.93673943, 2024-09-13T09:14:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012618451, 2024-09-13T09:14:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.542 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19603 2024-09-13T09:14:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521883+0000", 2024-09-13T09:14:31.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521882+0000", 2024-09-13T09:14:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521884+0000", 2024-09-13T09:14:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5340000000000001e-06 2024-09-13T09:14:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521888+0000", 2024-09-13T09:14:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0169999999999998e-06 2024-09-13T09:14:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521891+0000", 2024-09-13T09:14:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.852e-06 2024-09-13T09:14:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548208+0000", 2024-09-13T09:14:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026317364999999999 2024-09-13T09:14:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548211+0000", 2024-09-13T09:14:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-13T09:14:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534493+0000", 2024-09-13T09:14:31.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98628235500000005 2024-09-13T09:14:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534501+0000", 2024-09-13T09:14:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9049999999999997e-06 2024-09-13T09:14:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19604 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.936683587, 2024-09-13T09:14:31.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012577109, 2024-09-13T09:14:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19604 2024-09-13T09:14:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521938+0000", 2024-09-13T09:14:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521940+0000", 2024-09-13T09:14:31.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.516e-06 2024-09-13T09:14:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521944+0000", 2024-09-13T09:14:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9960000000000004e-06 2024-09-13T09:14:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521946+0000", 2024-09-13T09:14:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.604e-06 2024-09-13T09:14:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548232+0000", 2024-09-13T09:14:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026286130000000001 2024-09-13T09:14:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548235+0000", 2024-09-13T09:14:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:14:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534511+0000", 2024-09-13T09:14:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627648999999995 2024-09-13T09:14:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534515+0000", 2024-09-13T09:14:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0389999999999998e-06 2024-09-13T09:14:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19605 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.93662666500001, 2024-09-13T09:14:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012541157, 2024-09-13T09:14:31.552 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19605 2024-09-13T09:14:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521995+0000", 2024-09-13T09:14:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967296 2024-09-13T09:14:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.521997+0000", 2024-09-13T09:14:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.5090000000000001e-06 2024-09-13T09:14:31.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522000+0000", 2024-09-13T09:14:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8870000000000002e-06 2024-09-13T09:14:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522003+0000", 2024-09-13T09:14:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:14:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548253+0000", 2024-09-13T09:14:31.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026249850000000002 2024-09-13T09:14:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548256+0000", 2024-09-13T09:14:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:14:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534529+0000", 2024-09-13T09:14:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98627310499999998 2024-09-13T09:14:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534536+0000", 2024-09-13T09:14:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7710000000000006e-06 2024-09-13T09:14:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4903.0:19606 3.15 3.ba631d95 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:14:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 108.936573102, 2024-09-13T09:14:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.012502432, 2024-09-13T09:14:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.557 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-13T09:14:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4903", 2024-09-13T09:14:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 19606 2024-09-13T09:14:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522049+0000", 2024-09-13T09:14:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522051+0000", 2024-09-13T09:14:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1689999999999999e-06 2024-09-13T09:14:31.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522054+0000", 2024-09-13T09:14:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.084e-06 2024-09-13T09:14:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.522065+0000", 2024-09-13T09:14:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1123e-05 2024-09-13T09:14:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548276+0000", 2024-09-13T09:14:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026210639000000001 2024-09-13T09:14:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:42.548278+0000", 2024-09-13T09:14:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:14:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534547+0000", 2024-09-13T09:14:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.98626845600000002 2024-09-13T09:14:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:12:43.534551+0000", 2024-09-13T09:14:31.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4689999999999999e-06 2024-09-13T09:14:31.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.562 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.562 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.562 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:31.564 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:14:31.564 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:31.561+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:31.665 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:31.662+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:31.708 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:31.708 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:31.708 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:31.708 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:31.717 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:14:31.765 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:31.764+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:31.821 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:31.821 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:31.821 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-13T09:14:31.821 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-13T09:14:31.821 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:31.829 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:14:31.853 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:14:31.897 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:31.895+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:31.967 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:31.966+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:31.975 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-13T09:14:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-13T09:14:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-13T09:14:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2193 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.447169887000001, 2024-09-13T09:14:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064237548000000005, 2024-09-13T09:14:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2193 2024-09-13T09:14:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525854+0000", 2024-09-13T09:14:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525853+0000", 2024-09-13T09:14:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525855+0000", 2024-09-13T09:14:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-13T09:14:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525863+0000", 2024-09-13T09:14:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:14:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525867+0000", 2024-09-13T09:14:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.42e-06 2024-09-13T09:14:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.589922+0000", 2024-09-13T09:14:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064055239999999999 2024-09-13T09:14:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590015+0000", 2024-09-13T09:14:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2732999999999995e-05 2024-09-13T09:14:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590091+0000", 2024-09-13T09:14:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6271000000000002e-05 2024-09-13T09:14:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2195 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 933888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.447152188999993, 2024-09-13T09:14:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064373643999999994, 2024-09-13T09:14:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2195 2024-09-13T09:14:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525871+0000", 2024-09-13T09:14:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525873+0000", 2024-09-13T09:14:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:14:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525883+0000", 2024-09-13T09:14:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:14:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525888+0000", 2024-09-13T09:14:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5639999999999998e-06 2024-09-13T09:14:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590124+0000", 2024-09-13T09:14:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064236646999999994 2024-09-13T09:14:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590178+0000", 2024-09-13T09:14:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3841999999999997e-05 2024-09-13T09:14:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590245+0000", 2024-09-13T09:14:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6663999999999997e-05 2024-09-13T09:14:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2196 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 942080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.447131573999997, 2024-09-13T09:14:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064525259000000001, 2024-09-13T09:14:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2196 2024-09-13T09:14:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525892+0000", 2024-09-13T09:14:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525894+0000", 2024-09-13T09:14:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:14:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525902+0000", 2024-09-13T09:14:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.439e-06 2024-09-13T09:14:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525907+0000", 2024-09-13T09:14:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-13T09:14:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590284+0000", 2024-09-13T09:14:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064376937999999995 2024-09-13T09:14:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590351+0000", 2024-09-13T09:14:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7292000000000006e-05 2024-09-13T09:14:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590417+0000", 2024-09-13T09:14:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6348999999999998e-05 2024-09-13T09:14:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2198 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 950272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.447112618000006, 2024-09-13T09:14:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064790898, 2024-09-13T09:14:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2198 2024-09-13T09:14:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525911+0000", 2024-09-13T09:14:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525913+0000", 2024-09-13T09:14:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:14:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525919+0000", 2024-09-13T09:14:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:14:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525924+0000", 2024-09-13T09:14:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5229999999999997e-06 2024-09-13T09:14:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590446+0000", 2024-09-13T09:14:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064522170000000004 2024-09-13T09:14:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590519+0000", 2024-09-13T09:14:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2558999999999994e-05 2024-09-13T09:14:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590702+0000", 2024-09-13T09:14:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018325899999999999 2024-09-13T09:14:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2200 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.447095360000006, 2024-09-13T09:14:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065072160000000004, 2024-09-13T09:14:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2200 2024-09-13T09:14:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525928+0000", 2024-09-13T09:14:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525930+0000", 2024-09-13T09:14:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0499999999999999e-06 2024-09-13T09:14:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525939+0000", 2024-09-13T09:14:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7150000000000004e-06 2024-09-13T09:14:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525944+0000", 2024-09-13T09:14:31.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:14:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590722+0000", 2024-09-13T09:14:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.064778200999999994 2024-09-13T09:14:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.590769+0000", 2024-09-13T09:14:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7051000000000001e-05 2024-09-13T09:14:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:31.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591000+0000", 2024-09-13T09:14:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00023136000000000001 2024-09-13T09:14:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2202 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 966656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.447075346000005, 2024-09-13T09:14:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065206792999999999, 2024-09-13T09:14:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:31.999 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2202 2024-09-13T09:14:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:32.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525948+0000", 2024-09-13T09:14:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525955+0000", 2024-09-13T09:14:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3889999999999999e-06 2024-09-13T09:14:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525964+0000", 2024-09-13T09:14:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0389999999999993e-06 2024-09-13T09:14:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525969+0000", 2024-09-13T09:14:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:14:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591020+0000", 2024-09-13T09:14:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065051048 2024-09-13T09:14:32.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591094+0000", 2024-09-13T09:14:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3355000000000005e-05 2024-09-13T09:14:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591155+0000", 2024-09-13T09:14:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1446999999999996e-05 2024-09-13T09:14:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2203 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 974848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.447049763999999, 2024-09-13T09:14:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065398898999999996, 2024-09-13T09:14:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2203 2024-09-13T09:14:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.004 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525974+0000", 2024-09-13T09:14:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:32.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525973+0000", 2024-09-13T09:14:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525976+0000", 2024-09-13T09:14:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2560000000000001e-06 2024-09-13T09:14:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525984+0000", 2024-09-13T09:14:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1820000000000006e-06 2024-09-13T09:14:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525989+0000", 2024-09-13T09:14:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:14:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591193+0000", 2024-09-13T09:14:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065204113999999994 2024-09-13T09:14:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591282+0000", 2024-09-13T09:14:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8819000000000005e-05 2024-09-13T09:14:32.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591373+0000", 2024-09-13T09:14:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1085e-05 2024-09-13T09:14:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2205 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.447030479999995, 2024-09-13T09:14:32.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065513648999999993, 2024-09-13T09:14:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2205 2024-09-13T09:14:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525993+0000", 2024-09-13T09:14:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.525995+0000", 2024-09-13T09:14:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:14:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526001+0000", 2024-09-13T09:14:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0440000000000001e-06 2024-09-13T09:14:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526006+0000", 2024-09-13T09:14:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7319999999999998e-06 2024-09-13T09:14:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591399+0000", 2024-09-13T09:14:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065393387999999997 2024-09-13T09:14:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591445+0000", 2024-09-13T09:14:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5640999999999999e-05 2024-09-13T09:14:32.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591507+0000", 2024-09-13T09:14:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1823999999999996e-05 2024-09-13T09:14:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2208 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.447013224000003, 2024-09-13T09:14:32.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065786990000000004, 2024-09-13T09:14:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2208 2024-09-13T09:14:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526010+0000", 2024-09-13T09:14:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:32.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526012+0000", 2024-09-13T09:14:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:14:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526020+0000", 2024-09-13T09:14:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1599999999999998e-06 2024-09-13T09:14:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526026+0000", 2024-09-13T09:14:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:14:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591543+0000", 2024-09-13T09:14:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065517369000000006 2024-09-13T09:14:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591685+0000", 2024-09-13T09:14:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000142118 2024-09-13T09:14:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591797+0000", 2024-09-13T09:14:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000112259 2024-09-13T09:14:32.017 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2209 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 999424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.446993743999997, 2024-09-13T09:14:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066027186000000002, 2024-09-13T09:14:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:32.018 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2209 2024-09-13T09:14:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526030+0000", 2024-09-13T09:14:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:32.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526029+0000", 2024-09-13T09:14:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526032+0000", 2024-09-13T09:14:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1390000000000002e-06 2024-09-13T09:14:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526038+0000", 2024-09-13T09:14:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0920000000000003e-06 2024-09-13T09:14:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526042+0000", 2024-09-13T09:14:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.656e-06 2024-09-13T09:14:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591824+0000", 2024-09-13T09:14:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.065782004000000005 2024-09-13T09:14:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.591870+0000", 2024-09-13T09:14:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5602999999999997e-05 2024-09-13T09:14:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592057+0000", 2024-09-13T09:14:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018707000000000001 2024-09-13T09:14:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.022 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2210 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1007616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.446976814999999, 2024-09-13T09:14:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066324365999999996, 2024-09-13T09:14:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:32.023 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2210 2024-09-13T09:14:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526047+0000", 2024-09-13T09:14:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526046+0000", 2024-09-13T09:14:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526048+0000", 2024-09-13T09:14:32.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1019999999999999e-06 2024-09-13T09:14:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526056+0000", 2024-09-13T09:14:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.542e-06 2024-09-13T09:14:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526061+0000", 2024-09-13T09:14:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2249999999999999e-06 2024-09-13T09:14:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592093+0000", 2024-09-13T09:14:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066031426000000004 2024-09-13T09:14:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592173+0000", 2024-09-13T09:14:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0241999999999995e-05 2024-09-13T09:14:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592371+0000", 2024-09-13T09:14:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019818900000000001 2024-09-13T09:14:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.027 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2212 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1015808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.446957666000003, 2024-09-13T09:14:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066586027000000006, 2024-09-13T09:14:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:32.028 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2212 2024-09-13T09:14:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526066+0000", 2024-09-13T09:14:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526068+0000", 2024-09-13T09:14:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-13T09:14:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526075+0000", 2024-09-13T09:14:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2490000000000004e-06 2024-09-13T09:14:32.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526080+0000", 2024-09-13T09:14:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7609999999999998e-06 2024-09-13T09:14:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592402+0000", 2024-09-13T09:14:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066322123999999996 2024-09-13T09:14:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592468+0000", 2024-09-13T09:14:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6341999999999996e-05 2024-09-13T09:14:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592652+0000", 2024-09-13T09:14:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000183757 2024-09-13T09:14:32.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2213 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1024000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.446939526999998, 2024-09-13T09:14:32.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066902850999999999, 2024-09-13T09:14:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2213 2024-09-13T09:14:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:32.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526084+0000", 2024-09-13T09:14:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526086+0000", 2024-09-13T09:14:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.159e-06 2024-09-13T09:14:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526480+0000", 2024-09-13T09:14:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00039422799999999999 2024-09-13T09:14:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526487+0000", 2024-09-13T09:14:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:14:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592713+0000", 2024-09-13T09:14:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066225698999999999 2024-09-13T09:14:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592817+0000", 2024-09-13T09:14:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010385700000000001 2024-09-13T09:14:32.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.592987+0000", 2024-09-13T09:14:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016990000000000001 2024-09-13T09:14:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2215 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.446529032000001, 2024-09-13T09:14:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066734399999999999, 2024-09-13T09:14:32.038 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2215 2024-09-13T09:14:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526494+0000", 2024-09-13T09:14:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526498+0000", 2024-09-13T09:14:32.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1289999999999998e-06 2024-09-13T09:14:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526507+0000", 2024-09-13T09:14:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5009999999999993e-06 2024-09-13T09:14:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526512+0000", 2024-09-13T09:14:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:14:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593014+0000", 2024-09-13T09:14:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066502097999999996 2024-09-13T09:14:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593060+0000", 2024-09-13T09:14:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.545e-05 2024-09-13T09:14:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593229+0000", 2024-09-13T09:14:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000169075 2024-09-13T09:14:32.042 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.5347.0:2217 3.3 3:c65e14ee:::rbd_data.11239759ab94.0000000000000002:1b [read 1040384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:14:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 76.446506131000007, 2024-09-13T09:14:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.067048664999999993, 2024-09-13T09:14:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.5347", 2024-09-13T09:14:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/4157380093", 2024-09-13T09:14:32.043 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2217 2024-09-13T09:14:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526517+0000", 2024-09-13T09:14:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526519+0000", 2024-09-13T09:14:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.435e-06 2024-09-13T09:14:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526525+0000", 2024-09-13T09:14:32.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3920000000000002e-06 2024-09-13T09:14:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.526529+0000", 2024-09-13T09:14:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:14:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593260+0000", 2024-09-13T09:14:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.066730435000000005 2024-09-13T09:14:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593343+0000", 2024-09-13T09:14:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3008000000000002e-05 2024-09-13T09:14:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:13:15.593566+0000", 2024-09-13T09:14:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000223537 2024-09-13T09:14:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.047 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:30141 3.9 3:908b9f7f:::rbd_data.112326fb43d3.0000000000000001:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-13T09:14:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:14:18.853518+0000", 2024-09-13T09:14:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.119505643, 2024-09-13T09:14:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.92492906200000002, 2024-09-13T09:14:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 30141 2024-09-13T09:14:32.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853518+0000", 2024-09-13T09:14:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853518+0000", 2024-09-13T09:14:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853516+0000", 2024-09-13T09:14:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:14:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853527+0000", 2024-09-13T09:14:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1420000000000001e-05 2024-09-13T09:14:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853556+0000", 2024-09-13T09:14:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8496e-05 2024-09-13T09:14:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853569+0000", 2024-09-13T09:14:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3446e-05 2024-09-13T09:14:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853691+0000", 2024-09-13T09:14:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012215500000000001 2024-09-13T09:14:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853715+0000", 2024-09-13T09:14:32.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3266000000000001e-05 2024-09-13T09:14:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.775199+0000", 2024-09-13T09:14:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.92148457100000003 2024-09-13T09:14:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.775330+0000", 2024-09-13T09:14:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013118200000000001 2024-09-13T09:14:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:14:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.775498+0000", 2024-09-13T09:14:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016740200000000001 2024-09-13T09:14:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.777064+0000", 2024-09-13T09:14:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0015657469999999999 2024-09-13T09:14:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778397+0000", 2024-09-13T09:14:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00133325 2024-09-13T09:14:32.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778426+0000", 2024-09-13T09:14:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9612e-05 2024-09-13T09:14:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778447+0000", 2024-09-13T09:14:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0494999999999999e-05 2024-09-13T09:14:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:30143 3.9 3:93c2f121:::rbd_data.112326fb43d3.0000000000000003:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-13T09:14:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:14:18.853590+0000", 2024-09-13T09:14:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.119433437, 2024-09-13T09:14:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.92495849799999996, 2024-09-13T09:14:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 30143 2024-09-13T09:14:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:32.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853590+0000", 2024-09-13T09:14:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853590+0000", 2024-09-13T09:14:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853589+0000", 2024-09-13T09:14:32.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:14:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853595+0000", 2024-09-13T09:14:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:14:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853616+0000", 2024-09-13T09:14:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1313000000000001e-05 2024-09-13T09:14:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853630+0000", 2024-09-13T09:14:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3370000000000001e-05 2024-09-13T09:14:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853801+0000", 2024-09-13T09:14:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00017078899999999999 2024-09-13T09:14:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:18.853823+0000", 2024-09-13T09:14:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2557000000000001e-05 2024-09-13T09:14:32.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.776396+0000", 2024-09-13T09:14:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.92257259800000002 2024-09-13T09:14:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.776478+0000", 2024-09-13T09:14:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2528999999999995e-05 2024-09-13T09:14:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-13T09:14:32.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.776559+0000", 2024-09-13T09:14:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1043000000000003e-05 2024-09-13T09:14:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778313+0000", 2024-09-13T09:14:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0017531440000000001 2024-09-13T09:14:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778479+0000", 2024-09-13T09:14:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016687000000000001 2024-09-13T09:14:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778508+0000", 2024-09-13T09:14:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8311999999999999e-05 2024-09-13T09:14:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:19.778549+0000", 2024-09-13T09:14:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0907e-05 2024-09-13T09:14:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:30877 3.e 3:72416bbe:::rbd_header.112323636db9:head [call rbd.snapshot_remove in=18b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e133)", 2024-09-13T09:14:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:14:26.823896+0000", 2024-09-13T09:14:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.1491278830000002, 2024-09-13T09:14:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0092431399999999, 2024-09-13T09:14:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 30877 2024-09-13T09:14:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:32.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823896+0000", 2024-09-13T09:14:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823896+0000", 2024-09-13T09:14:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823894+0000", 2024-09-13T09:14:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999981 2024-09-13T09:14:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823904+0000", 2024-09-13T09:14:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0881e-05 2024-09-13T09:14:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823923+0000", 2024-09-13T09:14:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8105999999999999e-05 2024-09-13T09:14:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823934+0000", 2024-09-13T09:14:32.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1559999999999999e-05 2024-09-13T09:14:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.824008+0000", 2024-09-13T09:14:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3452000000000001e-05 2024-09-13T09:14:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op must wait for map", 2024-09-13T09:14:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.824016+0000", 2024-09-13T09:14:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5609999999999999e-06 2024-09-13T09:14:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.827969+0000", 2024-09-13T09:14:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003952817 2024-09-13T09:14:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.827985+0000", 2024-09-13T09:14:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6314e-05 2024-09-13T09:14:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.831801+0000", 2024-09-13T09:14:32.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0038155980000001 2024-09-13T09:14:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.832061+0000", 2024-09-13T09:14:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00025987599999999998 2024-09-13T09:14:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-13T09:14:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833014+0000", 2024-09-13T09:14:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00095324299999999999 2024-09-13T09:14:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-13T09:14:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833055+0000", 2024-09-13T09:14:32.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0895000000000001e-05 2024-09-13T09:14:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833139+0000", 2024-09-13T09:14:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3898000000000002e-05 2024-09-13T09:14:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:30879 3.e 3:72416bbe:::rbd_header.112323636db9:head [watch reconnect cookie 140383088041072 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-13T09:14:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:14:26.823954+0000", 2024-09-13T09:14:32.068 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.1490700250000003, 2024-09-13T09:14:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0099145549999999, 2024-09-13T09:14:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 30879 2024-09-13T09:14:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823954+0000", 2024-09-13T09:14:32.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823954+0000", 2024-09-13T09:14:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823953+0000", 2024-09-13T09:14:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823956+0000", 2024-09-13T09:14:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:14:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823963+0000", 2024-09-13T09:14:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0890000000000002e-06 2024-09-13T09:14:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.823969+0000", 2024-09-13T09:14:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5690000000000004e-06 2024-09-13T09:14:32.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.824079+0000", 2024-09-13T09:14:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000110147 2024-09-13T09:14:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting_for_map not empty", 2024-09-13T09:14:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.824092+0000", 2024-09-13T09:14:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2927e-05 2024-09-13T09:14:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.828011+0000", 2024-09-13T09:14:32.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0039192890000000003 2024-09-13T09:14:32.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:32.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:26.828018+0000", 2024-09-13T09:14:32.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2620000000000001e-06 2024-09-13T09:14:32.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.832844+0000", 2024-09-13T09:14:32.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.004825871 2024-09-13T09:14:32.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for rw locks", 2024-09-13T09:14:32.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.832937+0000", 2024-09-13T09:14:32.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2279000000000002e-05 2024-09-13T09:14:32.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833169+0000", 2024-09-13T09:14:32.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000232716 2024-09-13T09:14:32.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:32.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833215+0000", 2024-09-13T09:14:32.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5939000000000002e-05 2024-09-13T09:14:32.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:32.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833868+0000", 2024-09-13T09:14:32.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00065288399999999995 2024-09-13T09:14:32.075 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.075 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.075 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.075 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4387.0:30880 3.e 3:72416bbe:::rbd_header.112323636db9:head [watch ping cookie 140383088041072 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-13T09:14:32.075 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-13T09:14:27.104897+0000", 2024-09-13T09:14:32.075 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.8681269690000004, 2024-09-13T09:14:32.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.72873839699999998, 2024-09-13T09:14:32.075 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-13T09:14:32.075 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-13T09:14:32.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-13T09:14:32.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4387", 2024-09-13T09:14:32.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:32.076 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 30880 2024-09-13T09:14:32.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.076 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-13T09:14:32.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-13T09:14:32.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104897+0000", 2024-09-13T09:14:32.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-13T09:14:32.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104897+0000", 2024-09-13T09:14:32.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-13T09:14:32.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-13T09:14:32.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104896+0000", 2024-09-13T09:14:32.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4294967295.999999 2024-09-13T09:14:32.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-13T09:14:32.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104901+0000", 2024-09-13T09:14:32.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1290000000000004e-06 2024-09-13T09:14:32.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-13T09:14:32.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104908+0000", 2024-09-13T09:14:32.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.63e-06 2024-09-13T09:14:32.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104915+0000", 2024-09-13T09:14:32.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8849999999999999e-06 2024-09-13T09:14:32.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104977+0000", 2024-09-13T09:14:32.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1896999999999994e-05 2024-09-13T09:14:32.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for peered", 2024-09-13T09:14:32.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.104987+0000", 2024-09-13T09:14:32.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5259999999999999e-06 2024-09-13T09:14:32.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.832958+0000", 2024-09-13T09:14:32.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.72797099099999996 2024-09-13T09:14:32.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for rw locks", 2024-09-13T09:14:32.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833006+0000", 2024-09-13T09:14:32.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8177000000000003e-05 2024-09-13T09:14:32.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-13T09:14:32.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833457+0000", 2024-09-13T09:14:32.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000451213 2024-09-13T09:14:32.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-13T09:14:32.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833525+0000", 2024-09-13T09:14:32.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8194999999999993e-05 2024-09-13T09:14:32.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-13T09:14:32.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-13T09:14:32.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-13T09:14:32.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-13T09:14:27.833635+0000", 2024-09-13T09:14:32.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010948 2024-09-13T09:14:32.081 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.081 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.081 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.081 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-13T09:14:32.081 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-13T09:14:32.081 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:32.083 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:32.080+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:32.083 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:14:32.177 INFO:teuthology.orchestra.run.smithi129.stdout:get_pg_num_history pg_num_history pg_num_history(e131 pg_nums {1={13=1},2={16=8},3={21=32}} deleted_pools ) 2024-09-13T09:14:32.177 INFO:teuthology.orchestra.run.smithi129.stdout:Importing pgid 3.2 2024-09-13T09:14:32.178 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:409868d4:::rbd_data.1123b788e5d5.0000000000000039:head# 2024-09-13T09:14:32.178 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 17=[]:{} 2024-09-13T09:14:32.178 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:4262c1e6:::rbd_data.1123ee68625c.0000000000000022:head# 2024-09-13T09:14:32.178 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 14=[]:{} 2024-09-13T09:14:32.178 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:436e1b7d:::rbd_object_map.1123ffac6bc2:head# 2024-09-13T09:14:32.178 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 0=[]:{} 2024-09-13T09:14:32.179 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:450ddae4:::rbd_data.1123ffac6bc2.0000000000000002:head# 2024-09-13T09:14:32.179 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 0=[]:{} 2024-09-13T09:14:32.180 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:45165205:::rbd_data.1123b788e5d5.000000000000003b:head# 2024-09-13T09:14:32.180 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 17=[]:{} 2024-09-13T09:14:32.180 INFO:teuthology.orchestra.run.smithi129.stdout:Write #3:4550e1e5:::rbd_id.image_client.0-clone1:head# 2024-09-13T09:14:32.180 INFO:teuthology.orchestra.run.smithi129.stdout:snapset 0=[]:{} 2024-09-13T09:14:32.180 INFO:teuthology.orchestra.run.smithi129.stdout:write_pg epoch 131 info 3.2( v 130'639 (0'0,130'639] local-lis/les=127/128 n=6 ec=21/21 lis/c=127/127 les/c/f=128/128/0 sis=127) 2024-09-13T09:14:32.183 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:32.181+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:32.185 INFO:teuthology.orchestra.run.smithi129.stdout:Import successful 2024-09-13T09:14:32.191 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:32.191 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:32.191 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:32.191 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:32.200 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:14:32.260 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-09-13T09:14:32.320 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:32.318+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:32.361 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:32.361 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:32.361 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:32.361 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:32.362 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:32.371 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:14:32.386 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:32.384+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:32.486 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:32.484+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:32.520 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:32.520 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:32.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:32.520 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:32.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.521 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29717 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [watch reconnect cookie 140383290562704 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:14:32.521 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:32.521 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.766729233, 2024-09-13T09:14:32.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.006472936, 2024-09-13T09:14:32.521 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.521 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:32.521 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:32.521 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:32.521 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:32.521 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29717 2024-09-13T09:14:32.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.521 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:32.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751548+0000", 2024-09-13T09:14:32.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751547+0000", 2024-09-13T09:14:32.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:32.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751552+0000", 2024-09-13T09:14:32.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:14:32.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751564+0000", 2024-09-13T09:14:32.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1793e-05 2024-09-13T09:14:32.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751571+0000", 2024-09-13T09:14:32.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2660000000000004e-06 2024-09-13T09:14:32.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753102+0000", 2024-09-13T09:14:32.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001531085 2024-09-13T09:14:32.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:32.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753115+0000", 2024-09-13T09:14:32.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3039e-05 2024-09-13T09:14:32.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.752858+0000", 2024-09-13T09:14:32.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.999742715 2024-09-13T09:14:32.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:32.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.753010+0000", 2024-09-13T09:14:32.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015227799999999999 2024-09-13T09:14:32.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755125+0000", 2024-09-13T09:14:32.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021150029999999998 2024-09-13T09:14:32.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-09-13T09:14:32.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755235+0000", 2024-09-13T09:14:32.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000109775 2024-09-13T09:14:32.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756059+0000", 2024-09-13T09:14:32.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00082370900000000005 2024-09-13T09:14:32.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756154+0000", 2024-09-13T09:14:32.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4945999999999995e-05 2024-09-13T09:14:32.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758021+0000", 2024-09-13T09:14:32.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001867446 2024-09-13T09:14:32.527 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.527 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.527 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.527 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29718 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:14:32.527 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:32.527 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.766696368000002, 2024-09-13T09:14:32.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0067812169999999, 2024-09-13T09:14:32.527 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.528 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.528 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:32.528 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:32.528 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:32.528 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29718 2024-09-13T09:14:32.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.528 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:32.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:32.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751581+0000", 2024-09-13T09:14:32.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751585+0000", 2024-09-13T09:14:32.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2050000000000004e-06 2024-09-13T09:14:32.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751591+0000", 2024-09-13T09:14:32.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.37e-06 2024-09-13T09:14:32.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.751597+0000", 2024-09-13T09:14:32.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1190000000000002e-06 2024-09-13T09:14:32.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753150+0000", 2024-09-13T09:14:32.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001552351 2024-09-13T09:14:32.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:32.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:15.753164+0000", 2024-09-13T09:14:32.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4076e-05 2024-09-13T09:14:32.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.753307+0000", 2024-09-13T09:14:32.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0001434229999999 2024-09-13T09:14:32.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-09-13T09:14:32.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.753365+0000", 2024-09-13T09:14:32.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8014999999999999e-05 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755501+0000", 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021357260000000001 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.755558+0000", 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7516000000000003e-05 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756684+0000", 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011259550000000001 2024-09-13T09:14:32.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.756766+0000", 2024-09-13T09:14:32.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1755999999999994e-05 2024-09-13T09:14:32.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 3", 2024-09-13T09:14:32.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.757147+0000", 2024-09-13T09:14:32.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000381107 2024-09-13T09:14:32.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:32.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.757890+0000", 2024-09-13T09:14:32.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00074321499999999998 2024-09-13T09:14:32.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:32.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758178+0000", 2024-09-13T09:14:32.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00028755000000000002 2024-09-13T09:14:32.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:32.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758230+0000", 2024-09-13T09:14:32.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2317000000000001e-05 2024-09-13T09:14:32.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.758362+0000", 2024-09-13T09:14:32.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013204100000000001 2024-09-13T09:14:32.536 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.536 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.536 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.537 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGPush(3.d 123/122 [PushOp(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29, version: 99'289, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:b16abff6:::rbd_data.11231303fdb5.0000000000000001:29@99'289, size: 12582912, copy_subset: [0~12582912], clone_subset: {}, snapset: 29=[]:{29=[29]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:14:32.537 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.774123+0000", 2024-09-13T09:14:32.537 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.744154703, 2024-09-13T09:14:32.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051988820000000002, 2024-09-13T09:14:32.537 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.537 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:32.537 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.774123+0000", 2024-09-13T09:14:32.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.774123+0000", 2024-09-13T09:14:32.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.774122+0000", 2024-09-13T09:14:32.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777696+0000", 2024-09-13T09:14:32.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003573676 2024-09-13T09:14:32.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777708+0000", 2024-09-13T09:14:32.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2349e-05 2024-09-13T09:14:32.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777718+0000", 2024-09-13T09:14:32.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0013999999999999e-05 2024-09-13T09:14:32.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777761+0000", 2024-09-13T09:14:32.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3167999999999997e-05 2024-09-13T09:14:32.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.777771+0000", 2024-09-13T09:14:32.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3600000000000002e-06 2024-09-13T09:14:32.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.779322+0000", 2024-09-13T09:14:32.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015509740000000001 2024-09-13T09:14:32.541 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.541 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.541 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.541 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:29749 3.1e 3:78028c3c:::rbd_header.1123fb8ad2ac:head [watch reconnect cookie 140383086477584 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-13T09:14:32.541 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:32.541 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.718857509999999, 2024-09-13T09:14:32.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037671060000000001, 2024-09-13T09:14:32.541 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.541 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:32.541 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:32.541 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:32.541 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:32.542 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 29749 2024-09-13T09:14:32.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.542 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:32.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:32.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.543 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.543 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799420+0000", 2024-09-13T09:14:32.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.543 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.543 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799422+0000", 2024-09-13T09:14:32.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6790000000000001e-06 2024-09-13T09:14:32.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.543 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.543 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799429+0000", 2024-09-13T09:14:32.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4670000000000001e-06 2024-09-13T09:14:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.799434+0000", 2024-09-13T09:14:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0520000000000001e-06 2024-09-13T09:14:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.801379+0000", 2024-09-13T09:14:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019449599999999999 2024-09-13T09:14:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.801426+0000", 2024-09-13T09:14:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7391000000000001e-05 2024-09-13T09:14:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:16.803187+0000", 2024-09-13T09:14:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0017609870000000001 2024-09-13T09:14:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.3.0:2661 3.d e124/122 3:b5ba65cb:::rbd_data.1123fa58739e.0000000000000006:12 v 124'356, mlcod=123'351)", 2024-09-13T09:14:32.545 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.770130064, 2024-09-13T09:14:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034693089999999998, 2024-09-13T09:14:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748147+0000", 2024-09-13T09:14:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748155+0000", 2024-09-13T09:14:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3769999999999996e-06 2024-09-13T09:14:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748163+0000", 2024-09-13T09:14:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0120000000000005e-06 2024-09-13T09:14:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748169+0000", 2024-09-13T09:14:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0610000000000003e-06 2024-09-13T09:14:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748695+0000", 2024-09-13T09:14:32.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052687199999999995 2024-09-13T09:14:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.748716+0000", 2024-09-13T09:14:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0126e-05 2024-09-13T09:14:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.751587+0000", 2024-09-13T09:14:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028710839999999999 2024-09-13T09:14:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.751617+0000", 2024-09-13T09:14:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0096000000000001e-05 2024-09-13T09:14:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGPush(3.9 124/123 [PushOp(3:94c9dbee:::rbd_data.112339c5624d.0000000000000000:head, version: 88'1057, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:94c9dbee:::rbd_data.112339c5624d.0000000000000000:head@88'1057, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:0, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:14:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:17.795705+0000", 2024-09-13T09:14:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.722572905, 2024-09-13T09:14:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0038339609999999999, 2024-09-13T09:14:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:32.550 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795705+0000", 2024-09-13T09:14:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795705+0000", 2024-09-13T09:14:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795704+0000", 2024-09-13T09:14:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795715+0000", 2024-09-13T09:14:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1025e-05 2024-09-13T09:14:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795727+0000", 2024-09-13T09:14:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2031000000000001e-05 2024-09-13T09:14:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795736+0000", 2024-09-13T09:14:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2220000000000008e-06 2024-09-13T09:14:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795816+0000", 2024-09-13T09:14:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9338000000000005e-05 2024-09-13T09:14:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.795829+0000", 2024-09-13T09:14:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3677e-05 2024-09-13T09:14:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:17.799539+0000", 2024-09-13T09:14:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037092169999999999 2024-09-13T09:14:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.3.0:2692 3.d e125/122 3:b6fe9194:::rbd_data.112326fb43d3.000000000000002a:11 v 125'358, mlcod=124'354)", 2024-09-13T09:14:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:18.770322+0000", 2024-09-13T09:14:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.747955653, 2024-09-13T09:14:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047215089999999996, 2024-09-13T09:14:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.554 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.555 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770322+0000", 2024-09-13T09:14:32.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770322+0000", 2024-09-13T09:14:32.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770321+0000", 2024-09-13T09:14:32.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:32.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770333+0000", 2024-09-13T09:14:32.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2014000000000001e-05 2024-09-13T09:14:32.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770346+0000", 2024-09-13T09:14:32.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2771e-05 2024-09-13T09:14:32.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770354+0000", 2024-09-13T09:14:32.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1249999999999993e-06 2024-09-13T09:14:32.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770459+0000", 2024-09-13T09:14:32.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010558200000000001 2024-09-13T09:14:32.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.770483+0000", 2024-09-13T09:14:32.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3125e-05 2024-09-13T09:14:32.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:32.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.775015+0000", 2024-09-13T09:14:32.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045322449999999999 2024-09-13T09:14:32.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:18.775043+0000", 2024-09-13T09:14:32.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8532999999999999e-05 2024-09-13T09:14:32.558 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.558 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.558 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.558 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2445 3.9 e126/125 3:91fd9b46:::rbd_data.112326fb43d3.000000000000010f:11 v 126'1140, mlcod=126'1134)", 2024-09-13T09:14:32.559 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:32.559 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.736099308, 2024-09-13T09:14:32.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0036771899999999999, 2024-09-13T09:14:32.559 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.559 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.559 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:32.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:32.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782178+0000", 2024-09-13T09:14:32.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782184+0000", 2024-09-13T09:14:32.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5729999999999996e-06 2024-09-13T09:14:32.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782196+0000", 2024-09-13T09:14:32.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1817000000000001e-05 2024-09-13T09:14:32.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.782202+0000", 2024-09-13T09:14:32.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:14:32.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.783326+0000", 2024-09-13T09:14:32.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011234509999999999 2024-09-13T09:14:32.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.783343+0000", 2024-09-13T09:14:32.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7504e-05 2024-09-13T09:14:32.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:32.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.785708+0000", 2024-09-13T09:14:32.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023648829999999999 2024-09-13T09:14:32.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.785855+0000", 2024-09-13T09:14:32.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014743199999999999 2024-09-13T09:14:32.563 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.563 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.563 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.563 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.1.0:2475 3.4 e126/125 3:2095a19f:::rbd_data.1123b428d19a.000000000000001b:16 v 126'630, mlcod=126'626)", 2024-09-13T09:14:32.564 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.784119+0000", 2024-09-13T09:14:32.564 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.734158803, 2024-09-13T09:14:32.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033969479999999999, 2024-09-13T09:14:32.564 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.564 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.564 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784119+0000", 2024-09-13T09:14:32.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784119+0000", 2024-09-13T09:14:32.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784118+0000", 2024-09-13T09:14:32.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:32.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784126+0000", 2024-09-13T09:14:32.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6490000000000008e-06 2024-09-13T09:14:32.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784136+0000", 2024-09-13T09:14:32.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9699999999999994e-06 2024-09-13T09:14:32.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784141+0000", 2024-09-13T09:14:32.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8730000000000004e-06 2024-09-13T09:14:32.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784219+0000", 2024-09-13T09:14:32.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7650000000000004e-05 2024-09-13T09:14:32.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.784239+0000", 2024-09-13T09:14:32.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9794999999999999e-05 2024-09-13T09:14:32.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:32.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.787481+0000", 2024-09-13T09:14:32.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0032426320000000001 2024-09-13T09:14:32.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.787516+0000", 2024-09-13T09:14:32.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4143000000000002e-05 2024-09-13T09:14:32.567 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.568 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.568 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.568 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2447 3.9 e126/125 3:9541a82a:::rbd_data.1123b428d19a.000000000000000a:16 v 126'1144, mlcod=126'1140)", 2024-09-13T09:14:32.568 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.793072+0000", 2024-09-13T09:14:32.568 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.725205587, 2024-09-13T09:14:32.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046128150000000001, 2024-09-13T09:14:32.568 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.568 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.568 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793072+0000", 2024-09-13T09:14:32.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793072+0000", 2024-09-13T09:14:32.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793071+0000", 2024-09-13T09:14:32.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793077+0000", 2024-09-13T09:14:32.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1080000000000001e-06 2024-09-13T09:14:32.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793085+0000", 2024-09-13T09:14:32.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7309999999999999e-06 2024-09-13T09:14:32.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793091+0000", 2024-09-13T09:14:32.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6300000000000003e-06 2024-09-13T09:14:32.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793160+0000", 2024-09-13T09:14:32.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8681000000000002e-05 2024-09-13T09:14:32.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.793178+0000", 2024-09-13T09:14:32.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8861000000000001e-05 2024-09-13T09:14:32.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:32.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.797650+0000", 2024-09-13T09:14:32.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004471193 2024-09-13T09:14:32.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.797685+0000", 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5042000000000001e-05 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop_reply(osd.2.0:98 3.18 e126/125 ondisk, result = 0)", 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.794045+0000", 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.72423276, 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050958949999999996, 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794045+0000", 2024-09-13T09:14:32.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794045+0000", 2024-09-13T09:14:32.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794044+0000", 2024-09-13T09:14:32.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:32.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794049+0000", 2024-09-13T09:14:32.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8600000000000001e-06 2024-09-13T09:14:32.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794060+0000", 2024-09-13T09:14:32.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0801e-05 2024-09-13T09:14:32.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.794068+0000", 2024-09-13T09:14:32.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1550000000000007e-06 2024-09-13T09:14:32.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.799120+0000", 2024-09-13T09:14:32.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050525040000000002 2024-09-13T09:14:32.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.799128+0000", 2024-09-13T09:14:32.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2819999999999996e-06 2024-09-13T09:14:32.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.799141+0000", 2024-09-13T09:14:32.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2098e-05 2024-09-13T09:14:32.576 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.576 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.576 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.576 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2450 3.9 e126/125 3:95b1a540:::rbd_data.1123e3c0a1e8.0000000000000030:18 v 126'1148, mlcod=126'1144)", 2024-09-13T09:14:32.576 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:32.576 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.716649124, 2024-09-13T09:14:32.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0035455719999999999, 2024-09-13T09:14:32.576 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.576 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.576 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:32.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:32.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801628+0000", 2024-09-13T09:14:32.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:32.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801634+0000", 2024-09-13T09:14:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9539999999999998e-06 2024-09-13T09:14:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801646+0000", 2024-09-13T09:14:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1795e-05 2024-09-13T09:14:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.801653+0000", 2024-09-13T09:14:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7259999999999996e-06 2024-09-13T09:14:32.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.803017+0000", 2024-09-13T09:14:32.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001363997 2024-09-13T09:14:32.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.803038+0000", 2024-09-13T09:14:32.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1294e-05 2024-09-13T09:14:32.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:32.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.805138+0000", 2024-09-13T09:14:32.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020996259999999998 2024-09-13T09:14:32.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.805174+0000", 2024-09-13T09:14:32.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5970999999999998e-05 2024-09-13T09:14:32.580 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.580 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.580 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.580 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2452 3.9 e126/125 3:9030a72f:::rbd_data.1123af309369.0000000000000023:1f v 126'1152, mlcod=126'1148)", 2024-09-13T09:14:32.580 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.812010+0000", 2024-09-13T09:14:32.580 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.706267774000001, 2024-09-13T09:14:32.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0033163540000000001, 2024-09-13T09:14:32.581 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.581 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.581 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812010+0000", 2024-09-13T09:14:32.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812010+0000", 2024-09-13T09:14:32.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812009+0000", 2024-09-13T09:14:32.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812093+0000", 2024-09-13T09:14:32.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4022000000000003e-05 2024-09-13T09:14:32.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812121+0000", 2024-09-13T09:14:32.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.826e-05 2024-09-13T09:14:32.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812128+0000", 2024-09-13T09:14:32.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6270000000000002e-06 2024-09-13T09:14:32.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812170+0000", 2024-09-13T09:14:32.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1785000000000001e-05 2024-09-13T09:14:32.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812202+0000", 2024-09-13T09:14:32.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2505999999999997e-05 2024-09-13T09:14:32.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:32.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.815226+0000", 2024-09-13T09:14:32.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003024185 2024-09-13T09:14:32.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.815326+0000", 2024-09-13T09:14:32.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9637999999999997e-05 2024-09-13T09:14:32.584 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.584 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.584 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.585 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2453 3.9 e126/125 3:94dbe637:::rbd_data.1123af309369.00000000000001dc:1f v 126'1154, mlcod=126'1148)", 2024-09-13T09:14:32.585 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.812843+0000", 2024-09-13T09:14:32.585 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.705434914, 2024-09-13T09:14:32.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034843890000000001, 2024-09-13T09:14:32.585 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.585 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.585 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812843+0000", 2024-09-13T09:14:32.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812843+0000", 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812842+0000", 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812848+0000", 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4189999999999999e-06 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812859+0000", 2024-09-13T09:14:32.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0778000000000001e-05 2024-09-13T09:14:32.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.812866+0000", 2024-09-13T09:14:32.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7619999999999998e-06 2024-09-13T09:14:32.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.813231+0000", 2024-09-13T09:14:32.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00036497299999999999 2024-09-13T09:14:32.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.813251+0000", 2024-09-13T09:14:32.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0511999999999999e-05 2024-09-13T09:14:32.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:32.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.816298+0000", 2024-09-13T09:14:32.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003046801 2024-09-13T09:14:32.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.816327+0000", 2024-09-13T09:14:32.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8943000000000001e-05 2024-09-13T09:14:32.588 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.588 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.588 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.589 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2455 3.9 e126/125 3:95f6e286:::rbd_data.1123af309369.00000000000000ba:1f v 126'1158, mlcod=126'1152)", 2024-09-13T09:14:32.589 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.817505+0000", 2024-09-13T09:14:32.589 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.700772436999999, 2024-09-13T09:14:32.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034876989999999999, 2024-09-13T09:14:32.589 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.589 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.589 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817505+0000", 2024-09-13T09:14:32.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817505+0000", 2024-09-13T09:14:32.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817504+0000", 2024-09-13T09:14:32.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:32.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817512+0000", 2024-09-13T09:14:32.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1389999999999995e-06 2024-09-13T09:14:32.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817532+0000", 2024-09-13T09:14:32.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9089e-05 2024-09-13T09:14:32.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.817538+0000", 2024-09-13T09:14:32.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5799999999999997e-06 2024-09-13T09:14:32.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.818301+0000", 2024-09-13T09:14:32.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00076256200000000005 2024-09-13T09:14:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.818318+0000", 2024-09-13T09:14:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7428999999999999e-05 2024-09-13T09:14:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820959+0000", 2024-09-13T09:14:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026407129999999998 2024-09-13T09:14:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820993+0000", 2024-09-13T09:14:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3912000000000003e-05 2024-09-13T09:14:32.593 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.593 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.593 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.593 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop_reply(osd.2.0:107 3.18 e126/125 ondisk, result = 0)", 2024-09-13T09:14:32.593 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.820584+0000", 2024-09-13T09:14:32.593 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.697693846, 2024-09-13T09:14:32.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045400709999999997, 2024-09-13T09:14:32.593 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.593 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:32.593 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820584+0000", 2024-09-13T09:14:32.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820584+0000", 2024-09-13T09:14:32.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820583+0000", 2024-09-13T09:14:32.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:32.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820588+0000", 2024-09-13T09:14:32.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7029999999999997e-06 2024-09-13T09:14:32.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820598+0000", 2024-09-13T09:14:32.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0179e-05 2024-09-13T09:14:32.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.820604+0000", 2024-09-13T09:14:32.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7849999999999997e-06 2024-09-13T09:14:32.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825098+0000", 2024-09-13T09:14:32.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044948009999999997 2024-09-13T09:14:32.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825108+0000", 2024-09-13T09:14:32.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3759999999999997e-06 2024-09-13T09:14:32.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825124+0000", 2024-09-13T09:14:32.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5950000000000001e-05 2024-09-13T09:14:32.596 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.597 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.597 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.597 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2456 3.9 e126/125 3:93610890:::rbd_data.1123af309369.00000000000003af:1f v 126'1160, mlcod=126'1156)", 2024-09-13T09:14:32.597 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.821541+0000", 2024-09-13T09:14:32.597 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.696736465000001, 2024-09-13T09:14:32.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003753915, 2024-09-13T09:14:32.597 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.597 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.597 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821541+0000", 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821541+0000", 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821540+0000", 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821546+0000", 2024-09-13T09:14:32.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7370000000000003e-06 2024-09-13T09:14:32.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821604+0000", 2024-09-13T09:14:32.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8072000000000002e-05 2024-09-13T09:14:32.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821616+0000", 2024-09-13T09:14:32.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1956999999999999e-05 2024-09-13T09:14:32.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821711+0000", 2024-09-13T09:14:32.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5079000000000002e-05 2024-09-13T09:14:32.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.821731+0000", 2024-09-13T09:14:32.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9661999999999999e-05 2024-09-13T09:14:32.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:32.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825263+0000", 2024-09-13T09:14:32.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0035322399999999999 2024-09-13T09:14:32.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.825295+0000", 2024-09-13T09:14:32.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1813000000000002e-05 2024-09-13T09:14:32.601 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.601 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.601 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.602 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_repop(osd.4.0:2457 3.9 e126/125 3:9313c095:::rbd_data.1123af309369.00000000000000be:1f v 126'1162, mlcod=126'1156)", 2024-09-13T09:14:32.602 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:19.822601+0000", 2024-09-13T09:14:32.602 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.695676906999999, 2024-09-13T09:14:32.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003671147, 2024-09-13T09:14:32.602 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.602 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.602 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822601+0000", 2024-09-13T09:14:32.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822601+0000", 2024-09-13T09:14:32.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822600+0000", 2024-09-13T09:14:32.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:32.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822605+0000", 2024-09-13T09:14:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6979999999999997e-06 2024-09-13T09:14:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822612+0000", 2024-09-13T09:14:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8589999999999999e-06 2024-09-13T09:14:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.822617+0000", 2024-09-13T09:14:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:14:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.823075+0000", 2024-09-13T09:14:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00045848500000000001 2024-09-13T09:14:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.823094+0000", 2024-09-13T09:14:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8740999999999999e-05 2024-09-13T09:14:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:32.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.826239+0000", 2024-09-13T09:14:32.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0031454709999999999 2024-09-13T09:14:32.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:19.826272+0000", 2024-09-13T09:14:32.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2330999999999999e-05 2024-09-13T09:14:32.606 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.606 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.606 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.606 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:30365 3.7 3:e7191d4c:::rbd_header.1123313a4365:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-13T09:14:32.606 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:22.797253+0000", 2024-09-13T09:14:32.607 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.7210245450000006, 2024-09-13T09:14:32.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0059535409999999997, 2024-09-13T09:14:32.607 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.607 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.607 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:32.607 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:32.607 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:32.607 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 30365 2024-09-13T09:14:32.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.607 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797253+0000", 2024-09-13T09:14:32.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797253+0000", 2024-09-13T09:14:32.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797251+0000", 2024-09-13T09:14:32.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-09-13T09:14:32.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797262+0000", 2024-09-13T09:14:32.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0993e-05 2024-09-13T09:14:32.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797281+0000", 2024-09-13T09:14:32.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8216999999999999e-05 2024-09-13T09:14:32.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797290+0000", 2024-09-13T09:14:32.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5230000000000001e-06 2024-09-13T09:14:32.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797339+0000", 2024-09-13T09:14:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8854999999999999e-05 2024-09-13T09:14:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797437+0000", 2024-09-13T09:14:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8105999999999999e-05 2024-09-13T09:14:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:14:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.797842+0000", 2024-09-13T09:14:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00040497999999999999 2024-09-13T09:14:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.800594+0000", 2024-09-13T09:14:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002752193 2024-09-13T09:14:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.803126+0000", 2024-09-13T09:14:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025320070000000002 2024-09-13T09:14:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.803177+0000", 2024-09-13T09:14:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0513000000000002e-05 2024-09-13T09:14:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:22.803206+0000", 2024-09-13T09:14:32.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9747000000000002e-05 2024-09-13T09:14:32.612 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.612 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.612 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.612 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4387.0:30611 3.7 3:e0345511:::rbd_data.1123b259dc65.000000000000000b:head [delete] snapc e=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-13T09:14:32.612 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:14:23.893039+0000", 2024-09-13T09:14:32.612 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 8.6252385959999991, 2024-09-13T09:14:32.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003465072, 2024-09-13T09:14:32.612 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:32.612 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:14:32.613 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:32.613 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4387", 2024-09-13T09:14:32.613 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/3269166003", 2024-09-13T09:14:32.613 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 30611 2024-09-13T09:14:32.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.613 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:32.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:32.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893039+0000", 2024-09-13T09:14:32.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:32.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893039+0000", 2024-09-13T09:14:32.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:32.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:32.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893038+0000", 2024-09-13T09:14:32.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:32.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:32.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893043+0000", 2024-09-13T09:14:32.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3739999999999997e-06 2024-09-13T09:14:32.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:32.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893055+0000", 2024-09-13T09:14:32.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2138e-05 2024-09-13T09:14:32.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:32.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893083+0000", 2024-09-13T09:14:32.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7589999999999998e-05 2024-09-13T09:14:32.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:32.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893158+0000", 2024-09-13T09:14:32.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5318e-05 2024-09-13T09:14:32.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:32.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893343+0000", 2024-09-13T09:14:32.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000184747 2024-09-13T09:14:32.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for subops from 1", 2024-09-13T09:14:32.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.893440+0000", 2024-09-13T09:14:32.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6751999999999998e-05 2024-09-13T09:14:32.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op_commit", 2024-09-13T09:14:32.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.896088+0000", 2024-09-13T09:14:32.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026486370000000001 2024-09-13T09:14:32.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:14:32.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.896422+0000", 2024-09-13T09:14:32.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000333526 2024-09-13T09:14:32.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-09-13T09:14:32.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.896467+0000", 2024-09-13T09:14:32.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5439000000000003e-05 2024-09-13T09:14:32.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:32.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:32.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:32.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:14:23.896504+0000", 2024-09-13T09:14:32.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6547000000000001e-05 2024-09-13T09:14:32.618 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.618 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.618 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.618 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:32.618 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:32.618 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:32.619 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:32.618+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:32.620 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:14:32.704 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /var/log/ceph/exp.3.2.5 2024-09-13T09:14:32.720 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:32.718+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:32.768 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:32.768 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:32.768 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:32.768 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:32.777 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:14:32.822 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:32.820+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:32.922 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:32.921+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:32.923 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:32.924 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:32.924 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:32.924 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:32.924 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:32.932 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:14:33.023 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:33.021+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:33.042 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:33.042 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-09-13T09:14:33.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-09-13T09:14:33.042 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-09-13T09:14:33.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.043 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20080 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6053888~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.043 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:33.043 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.51125374599999, 2024-09-13T09:14:33.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048542742, 2024-09-13T09:14:33.043 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.043 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.043 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20080 2024-09-13T09:14:33.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.043 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:33.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528924+0000", 2024-09-13T09:14:33.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528923+0000", 2024-09-13T09:14:33.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528927+0000", 2024-09-13T09:14:33.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2090000000000001e-06 2024-09-13T09:14:33.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528933+0000", 2024-09-13T09:14:33.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7530000000000004e-06 2024-09-13T09:14:33.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528938+0000", 2024-09-13T09:14:33.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:14:33.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536226+0000", 2024-09-13T09:14:33.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072882700000000003 2024-09-13T09:14:33.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536233+0000", 2024-09-13T09:14:33.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3810000000000001e-06 2024-09-13T09:14:33.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534919+0000", 2024-09-13T09:14:33.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99868568499999999 2024-09-13T09:14:33.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534929+0000", 2024-09-13T09:14:33.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7100000000000002e-06 2024-09-13T09:14:33.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577320+0000", 2024-09-13T09:14:33.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423915290000001 2024-09-13T09:14:33.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577378+0000", 2024-09-13T09:14:33.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7528000000000002e-05 2024-09-13T09:14:33.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577467+0000", 2024-09-13T09:14:33.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8831000000000004e-05 2024-09-13T09:14:33.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.048 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.049 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20081 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6076416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.049 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:33.049 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.511235711, 2024-09-13T09:14:33.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048699257, 2024-09-13T09:14:33.049 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.049 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.049 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.050 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.050 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.050 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20081 2024-09-13T09:14:33.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.050 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:33.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:33.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528942+0000", 2024-09-13T09:14:33.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528944+0000", 2024-09-13T09:14:33.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:14:33.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528950+0000", 2024-09-13T09:14:33.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8930000000000002e-06 2024-09-13T09:14:33.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528954+0000", 2024-09-13T09:14:33.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:14:33.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536259+0000", 2024-09-13T09:14:33.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073046270000000002 2024-09-13T09:14:33.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536266+0000", 2024-09-13T09:14:33.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:14:33.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534969+0000", 2024-09-13T09:14:33.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99870272500000001 2024-09-13T09:14:33.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.534977+0000", 2024-09-13T09:14:33.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2630000000000003e-06 2024-09-13T09:14:33.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577491+0000", 2024-09-13T09:14:33.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0425141710000001 2024-09-13T09:14:33.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577546+0000", 2024-09-13T09:14:33.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4412e-05 2024-09-13T09:14:33.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577641+0000", 2024-09-13T09:14:33.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5569999999999995e-05 2024-09-13T09:14:33.055 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.055 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.055 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.055 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20082 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6098944~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.055 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:33.055 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.511219041, 2024-09-13T09:14:33.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048902193, 2024-09-13T09:14:33.055 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.055 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.055 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.055 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.056 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.056 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20082 2024-09-13T09:14:33.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.056 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:33.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.056 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.056 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528959+0000", 2024-09-13T09:14:33.056 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528958+0000", 2024-09-13T09:14:33.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528960+0000", 2024-09-13T09:14:33.057 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:14:33.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.057 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.057 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528966+0000", 2024-09-13T09:14:33.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:14:33.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528972+0000", 2024-09-13T09:14:33.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3890000000000004e-06 2024-09-13T09:14:33.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.058 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.058 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536292+0000", 2024-09-13T09:14:33.058 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073199550000000004 2024-09-13T09:14:33.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536300+0000", 2024-09-13T09:14:33.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:14:33.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.059 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.059 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535014+0000", 2024-09-13T09:14:33.059 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99871430500000002 2024-09-13T09:14:33.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535022+0000", 2024-09-13T09:14:33.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2880000000000004e-06 2024-09-13T09:14:33.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577680+0000", 2024-09-13T09:14:33.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.042658174 2024-09-13T09:14:33.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.060 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.060 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577744+0000", 2024-09-13T09:14:33.060 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4115000000000005e-05 2024-09-13T09:14:33.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.061 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.061 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577861+0000", 2024-09-13T09:14:33.061 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116929 2024-09-13T09:14:33.061 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.061 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.061 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.062 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20083 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6121472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.062 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:33.062 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.51120140899999, 2024-09-13T09:14:33.062 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049070334, 2024-09-13T09:14:33.062 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.062 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.062 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.062 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.062 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.062 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20083 2024-09-13T09:14:33.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.062 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:33.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.063 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.063 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:33.063 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.528976+0000", 2024-09-13T09:14:33.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529004+0000", 2024-09-13T09:14:33.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8123000000000002e-05 2024-09-13T09:14:33.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.064 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.064 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529012+0000", 2024-09-13T09:14:33.064 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3599999999999996e-06 2024-09-13T09:14:33.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529018+0000", 2024-09-13T09:14:33.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:14:33.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.065 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536326+0000", 2024-09-13T09:14:33.065 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073081350000000003 2024-09-13T09:14:33.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.065 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536333+0000", 2024-09-13T09:14:33.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.267e-06 2024-09-13T09:14:33.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535056+0000", 2024-09-13T09:14:33.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.998723468 2024-09-13T09:14:33.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.066 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.066 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535065+0000", 2024-09-13T09:14:33.066 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-09-13T09:14:33.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577899+0000", 2024-09-13T09:14:33.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0428334699999999 2024-09-13T09:14:33.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.577954+0000", 2024-09-13T09:14:33.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4802000000000001e-05 2024-09-13T09:14:33.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.067 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.067 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578047+0000", 2024-09-13T09:14:33.067 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3016999999999994e-05 2024-09-13T09:14:33.068 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.068 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.068 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.068 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20084 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6144000~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.068 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:33.068 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.511155104, 2024-09-13T09:14:33.068 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049385628, 2024-09-13T09:14:33.068 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.068 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.068 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.068 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.068 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.069 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20084 2024-09-13T09:14:33.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.069 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:33.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.069 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.069 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529023+0000", 2024-09-13T09:14:33.069 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.070 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.070 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529022+0000", 2024-09-13T09:14:33.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.070 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.070 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529025+0000", 2024-09-13T09:14:33.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.469e-06 2024-09-13T09:14:33.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.070 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.070 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529032+0000", 2024-09-13T09:14:33.070 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:14:33.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529043+0000", 2024-09-13T09:14:33.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0526999999999999e-05 2024-09-13T09:14:33.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536358+0000", 2024-09-13T09:14:33.071 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073156879999999999 2024-09-13T09:14:33.071 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.071 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.071 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.071 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536365+0000", 2024-09-13T09:14:33.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9369999999999998e-06 2024-09-13T09:14:33.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535097+0000", 2024-09-13T09:14:33.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99873154799999997 2024-09-13T09:14:33.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.072 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.072 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.072 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535107+0000", 2024-09-13T09:14:33.072 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0046e-05 2024-09-13T09:14:33.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.073 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578081+0000", 2024-09-13T09:14:33.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0429737240000001 2024-09-13T09:14:33.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.073 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578219+0000", 2024-09-13T09:14:33.073 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137952 2024-09-13T09:14:33.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.073 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.073 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.073 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578408+0000", 2024-09-13T09:14:33.074 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018956299999999999 2024-09-13T09:14:33.074 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.074 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.074 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.074 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20085 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6166528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.074 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:33.074 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.511129803, 2024-09-13T09:14:33.074 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0495276389999999, 2024-09-13T09:14:33.074 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.074 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.074 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.075 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.075 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.075 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20085 2024-09-13T09:14:33.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.075 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.075 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.075 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:33.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.075 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.075 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529048+0000", 2024-09-13T09:14:33.075 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.076 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.076 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529047+0000", 2024-09-13T09:14:33.076 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.076 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.076 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529050+0000", 2024-09-13T09:14:33.076 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6350000000000002e-06 2024-09-13T09:14:33.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.076 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529057+0000", 2024-09-13T09:14:33.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9249999999999998e-06 2024-09-13T09:14:33.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529062+0000", 2024-09-13T09:14:33.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.809e-06 2024-09-13T09:14:33.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.077 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.077 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536391+0000", 2024-09-13T09:14:33.077 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073290389999999999 2024-09-13T09:14:33.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536398+0000", 2024-09-13T09:14:33.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4100000000000002e-06 2024-09-13T09:14:33.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535151+0000", 2024-09-13T09:14:33.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99875283999999998 2024-09-13T09:14:33.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.078 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.078 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535162+0000", 2024-09-13T09:14:33.078 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0515e-05 2024-09-13T09:14:33.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578435+0000", 2024-09-13T09:14:33.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0432731989999999 2024-09-13T09:14:33.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.079 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578490+0000", 2024-09-13T09:14:33.079 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5470999999999998e-05 2024-09-13T09:14:33.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.079 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.080 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578576+0000", 2024-09-13T09:14:33.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5222999999999994e-05 2024-09-13T09:14:33.080 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.080 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.080 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.080 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20086 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6189056~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.080 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:33.080 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.511111574, 2024-09-13T09:14:33.080 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498679380000002, 2024-09-13T09:14:33.080 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.080 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.081 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.081 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.081 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.081 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20086 2024-09-13T09:14:33.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.081 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.081 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.081 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:33.081 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.081 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.082 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:33.082 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.082 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.082 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529066+0000", 2024-09-13T09:14:33.082 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.082 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.082 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529068+0000", 2024-09-13T09:14:33.082 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:14:33.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.083 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.083 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529074+0000", 2024-09-13T09:14:33.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.993e-06 2024-09-13T09:14:33.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.083 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.083 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529079+0000", 2024-09-13T09:14:33.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:14:33.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.083 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.083 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536423+0000", 2024-09-13T09:14:33.083 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073448070000000001 2024-09-13T09:14:33.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536431+0000", 2024-09-13T09:14:33.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:14:33.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535196+0000", 2024-09-13T09:14:33.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876571300000005 2024-09-13T09:14:33.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.084 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.084 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535201+0000", 2024-09-13T09:14:33.084 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:14:33.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578608+0000", 2024-09-13T09:14:33.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0434069669999999 2024-09-13T09:14:33.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578760+0000", 2024-09-13T09:14:33.085 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015158700000000001 2024-09-13T09:14:33.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.085 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.085 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578934+0000", 2024-09-13T09:14:33.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017428100000000001 2024-09-13T09:14:33.086 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.086 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.086 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.086 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20087 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6211584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.086 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:33.086 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.51106425499999, 2024-09-13T09:14:33.086 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049986825, 2024-09-13T09:14:33.086 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.086 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.086 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.087 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.087 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.087 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20087 2024-09-13T09:14:33.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.087 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.087 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.087 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:33.087 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.087 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.087 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:33.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529113+0000", 2024-09-13T09:14:33.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.088 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.088 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529117+0000", 2024-09-13T09:14:33.088 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.715e-06 2024-09-13T09:14:33.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529126+0000", 2024-09-13T09:14:33.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2690000000000005e-06 2024-09-13T09:14:33.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529132+0000", 2024-09-13T09:14:33.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4910000000000002e-06 2024-09-13T09:14:33.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.089 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.089 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536457+0000", 2024-09-13T09:14:33.089 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007325013 2024-09-13T09:14:33.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536465+0000", 2024-09-13T09:14:33.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1930000000000004e-06 2024-09-13T09:14:33.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535233+0000", 2024-09-13T09:14:33.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876843400000004 2024-09-13T09:14:33.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.090 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.090 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535240+0000", 2024-09-13T09:14:33.090 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:14:33.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.578959+0000", 2024-09-13T09:14:33.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0437193869999999 2024-09-13T09:14:33.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.091 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579013+0000", 2024-09-13T09:14:33.091 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4441000000000003e-05 2024-09-13T09:14:33.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.091 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.092 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579100+0000", 2024-09-13T09:14:33.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6818999999999997e-05 2024-09-13T09:14:33.092 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.092 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.092 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.092 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20088 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6234112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.092 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:33.092 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.510550017, 2024-09-13T09:14:33.092 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0496566679999999, 2024-09-13T09:14:33.092 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.092 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.093 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.093 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.093 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.093 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20088 2024-09-13T09:14:33.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.093 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.093 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:33.093 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.093 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529628+0000", 2024-09-13T09:14:33.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529627+0000", 2024-09-13T09:14:33.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-09-13T09:14:33.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.094 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.094 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529637+0000", 2024-09-13T09:14:33.094 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0423e-05 2024-09-13T09:14:33.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529648+0000", 2024-09-13T09:14:33.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0825999999999999e-05 2024-09-13T09:14:33.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529655+0000", 2024-09-13T09:14:33.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:14:33.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.095 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.095 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536490+0000", 2024-09-13T09:14:33.095 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068357690000000002 2024-09-13T09:14:33.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536498+0000", 2024-09-13T09:14:33.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3869999999999997e-06 2024-09-13T09:14:33.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535270+0000", 2024-09-13T09:14:33.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99877184500000005 2024-09-13T09:14:33.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.096 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.096 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535275+0000", 2024-09-13T09:14:33.096 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1970000000000004e-06 2024-09-13T09:14:33.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579125+0000", 2024-09-13T09:14:33.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.043849808 2024-09-13T09:14:33.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.097 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579192+0000", 2024-09-13T09:14:33.097 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7123000000000001e-05 2024-09-13T09:14:33.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.097 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.098 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579284+0000", 2024-09-13T09:14:33.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2681000000000003e-05 2024-09-13T09:14:33.098 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.098 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.098 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.098 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20089 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6256640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.098 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:33.098 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.51051548700001, 2024-09-13T09:14:33.098 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0498093559999999, 2024-09-13T09:14:33.098 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.098 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.099 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.099 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.099 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.099 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20089 2024-09-13T09:14:33.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.099 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.099 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.099 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:33.099 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.099 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.099 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:33.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.100 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529662+0000", 2024-09-13T09:14:33.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.100 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.100 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529665+0000", 2024-09-13T09:14:33.100 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:14:33.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529671+0000", 2024-09-13T09:14:33.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.426e-06 2024-09-13T09:14:33.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.101 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.101 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529676+0000", 2024-09-13T09:14:33.101 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6240000000000001e-06 2024-09-13T09:14:33.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536523+0000", 2024-09-13T09:14:33.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068469810000000002 2024-09-13T09:14:33.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536530+0000", 2024-09-13T09:14:33.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.148e-06 2024-09-13T09:14:33.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.102 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.102 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535298+0000", 2024-09-13T09:14:33.102 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99876838400000001 2024-09-13T09:14:33.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.103 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535302+0000", 2024-09-13T09:14:33.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0249999999999996e-06 2024-09-13T09:14:33.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.103 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579326+0000", 2024-09-13T09:14:33.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0440243039999999 2024-09-13T09:14:33.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.103 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.103 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579391+0000", 2024-09-13T09:14:33.103 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4201000000000004e-05 2024-09-13T09:14:33.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.104 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.104 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579472+0000", 2024-09-13T09:14:33.104 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0939000000000006e-05 2024-09-13T09:14:33.104 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.104 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.104 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.104 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20090 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6279168~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.104 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:33.104 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.51049758000001, 2024-09-13T09:14:33.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049954821, 2024-09-13T09:14:33.105 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.105 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.105 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.105 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.105 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.105 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20090 2024-09-13T09:14:33.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.105 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.105 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.105 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:33.105 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:33.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529680+0000", 2024-09-13T09:14:33.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.106 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.106 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529682+0000", 2024-09-13T09:14:33.106 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1670000000000002e-06 2024-09-13T09:14:33.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529689+0000", 2024-09-13T09:14:33.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:14:33.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.529694+0000", 2024-09-13T09:14:33.107 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:14:33.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.107 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.107 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536850+0000", 2024-09-13T09:14:33.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0071560269999999997 2024-09-13T09:14:33.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.536858+0000", 2024-09-13T09:14:33.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.96e-06 2024-09-13T09:14:33.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.108 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.108 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535325+0000", 2024-09-13T09:14:33.108 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99846701699999996 2024-09-13T09:14:33.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.109 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.535329+0000", 2024-09-13T09:14:33.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:14:33.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.109 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579494+0000", 2024-09-13T09:14:33.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0441651679999999 2024-09-13T09:14:33.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.109 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.109 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579550+0000", 2024-09-13T09:14:33.109 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6280000000000003e-05 2024-09-13T09:14:33.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.110 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.110 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.579635+0000", 2024-09-13T09:14:33.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4604000000000005e-05 2024-09-13T09:14:33.110 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.110 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.110 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.110 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20213 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6168576~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.110 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:33.110 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.501370732, 2024-09-13T09:14:33.110 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0483973080000002, 2024-09-13T09:14:33.110 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.111 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.111 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.111 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.111 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.111 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20213 2024-09-13T09:14:33.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.111 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.111 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:33.111 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.111 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:33.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.112 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538807+0000", 2024-09-13T09:14:33.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.112 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.112 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538839+0000", 2024-09-13T09:14:33.112 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2255000000000002e-05 2024-09-13T09:14:33.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538847+0000", 2024-09-13T09:14:33.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4770000000000003e-06 2024-09-13T09:14:33.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538853+0000", 2024-09-13T09:14:33.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7660000000000004e-06 2024-09-13T09:14:33.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.113 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.113 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539350+0000", 2024-09-13T09:14:33.113 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049653499999999999 2024-09-13T09:14:33.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539357+0000", 2024-09-13T09:14:33.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:14:33.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536512+0000", 2024-09-13T09:14:33.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99715542300000004 2024-09-13T09:14:33.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.114 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.114 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536516+0000", 2024-09-13T09:14:33.114 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:14:33.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587048+0000", 2024-09-13T09:14:33.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0505321670000001 2024-09-13T09:14:33.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.115 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587106+0000", 2024-09-13T09:14:33.115 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7503999999999997e-05 2024-09-13T09:14:33.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.115 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.116 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587204+0000", 2024-09-13T09:14:33.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8454000000000003e-05 2024-09-13T09:14:33.116 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.116 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.116 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.116 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20214 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6191104~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.116 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:33.116 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.50131915199999, 2024-09-13T09:14:33.116 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0485107500000002, 2024-09-13T09:14:33.116 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.116 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.116 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.117 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.117 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20214 2024-09-13T09:14:33.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.117 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.117 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.117 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:33.117 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.117 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.117 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538859+0000", 2024-09-13T09:14:33.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538858+0000", 2024-09-13T09:14:33.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.118 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538860+0000", 2024-09-13T09:14:33.118 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4609999999999998e-06 2024-09-13T09:14:33.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.118 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.118 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538866+0000", 2024-09-13T09:14:33.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.806e-06 2024-09-13T09:14:33.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538871+0000", 2024-09-13T09:14:33.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:14:33.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.119 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.119 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.119 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539382+0000", 2024-09-13T09:14:33.119 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051102500000000002 2024-09-13T09:14:33.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539389+0000", 2024-09-13T09:14:33.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2799999999999998e-06 2024-09-13T09:14:33.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536552+0000", 2024-09-13T09:14:33.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99716330900000005 2024-09-13T09:14:33.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.120 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.120 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536562+0000", 2024-09-13T09:14:33.120 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:14:33.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587232+0000", 2024-09-13T09:14:33.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0506703980000001 2024-09-13T09:14:33.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587285+0000", 2024-09-13T09:14:33.121 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2921999999999999e-05 2024-09-13T09:14:33.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.121 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.121 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587369+0000", 2024-09-13T09:14:33.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4110000000000006e-05 2024-09-13T09:14:33.122 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.122 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.122 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.122 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20215 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6213632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.122 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:33.122 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.501302876, 2024-09-13T09:14:33.122 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.04868089, 2024-09-13T09:14:33.122 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.122 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.122 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.122 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.123 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.123 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20215 2024-09-13T09:14:33.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.123 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:33.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.123 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.123 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538875+0000", 2024-09-13T09:14:33.123 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.124 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.124 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538874+0000", 2024-09-13T09:14:33.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.124 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.124 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538876+0000", 2024-09-13T09:14:33.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:14:33.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.124 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.124 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538882+0000", 2024-09-13T09:14:33.124 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6470000000000003e-06 2024-09-13T09:14:33.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538886+0000", 2024-09-13T09:14:33.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0210000000000001e-06 2024-09-13T09:14:33.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539410+0000", 2024-09-13T09:14:33.125 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052388499999999998 2024-09-13T09:14:33.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.125 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.125 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539417+0000", 2024-09-13T09:14:33.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6730000000000003e-06 2024-09-13T09:14:33.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536588+0000", 2024-09-13T09:14:33.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717082400000001 2024-09-13T09:14:33.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.126 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.126 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536591+0000", 2024-09-13T09:14:33.126 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7440000000000001e-06 2024-09-13T09:14:33.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587392+0000", 2024-09-13T09:14:33.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.050800886 2024-09-13T09:14:33.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587447+0000", 2024-09-13T09:14:33.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4531999999999999e-05 2024-09-13T09:14:33.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.127 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.127 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587556+0000", 2024-09-13T09:14:33.127 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010902499999999999 2024-09-13T09:14:33.127 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.128 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.128 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.128 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20216 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6236160~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.128 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:33.128 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.50128710200001, 2024-09-13T09:14:33.128 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489699670000001, 2024-09-13T09:14:33.128 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.128 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.128 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.128 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.128 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.129 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20216 2024-09-13T09:14:33.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.129 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:33.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.129 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.129 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538891+0000", 2024-09-13T09:14:33.129 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.130 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538890+0000", 2024-09-13T09:14:33.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.130 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538892+0000", 2024-09-13T09:14:33.130 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0260000000000002e-06 2024-09-13T09:14:33.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.130 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538897+0000", 2024-09-13T09:14:33.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:14:33.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538902+0000", 2024-09-13T09:14:33.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.087e-06 2024-09-13T09:14:33.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.131 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.131 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539437+0000", 2024-09-13T09:14:33.131 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053584399999999995 2024-09-13T09:14:33.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539444+0000", 2024-09-13T09:14:33.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0829999999999998e-06 2024-09-13T09:14:33.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536622+0000", 2024-09-13T09:14:33.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99717751700000001 2024-09-13T09:14:33.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.132 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.132 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536632+0000", 2024-09-13T09:14:33.132 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5540000000000003e-06 2024-09-13T09:14:33.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587589+0000", 2024-09-13T09:14:33.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0509578079999999 2024-09-13T09:14:33.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.133 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587695+0000", 2024-09-13T09:14:33.133 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105196 2024-09-13T09:14:33.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.133 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.134 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587861+0000", 2024-09-13T09:14:33.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016606100000000001 2024-09-13T09:14:33.134 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.134 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.134 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.134 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20217 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6258688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.134 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:33.134 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.501272227, 2024-09-13T09:14:33.134 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0491692480000001, 2024-09-13T09:14:33.134 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.134 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.135 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.135 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.135 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.135 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20217 2024-09-13T09:14:33.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.135 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.135 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.135 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:33.135 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:33.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538905+0000", 2024-09-13T09:14:33.136 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.136 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.136 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538907+0000", 2024-09-13T09:14:33.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:14:33.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538913+0000", 2024-09-13T09:14:33.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7139999999999998e-06 2024-09-13T09:14:33.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.137 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.137 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538919+0000", 2024-09-13T09:14:33.137 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7060000000000001e-06 2024-09-13T09:14:33.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.138 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.138 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539467+0000", 2024-09-13T09:14:33.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00054805299999999995 2024-09-13T09:14:33.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.138 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.138 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539473+0000", 2024-09-13T09:14:33.138 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8979999999999999e-06 2024-09-13T09:14:33.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536657+0000", 2024-09-13T09:14:33.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718314399999997 2024-09-13T09:14:33.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536660+0000", 2024-09-13T09:14:33.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.58e-06 2024-09-13T09:14:33.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.139 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.139 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587894+0000", 2024-09-13T09:14:33.139 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.051234177 2024-09-13T09:14:33.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.587980+0000", 2024-09-13T09:14:33.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5555000000000003e-05 2024-09-13T09:14:33.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.140 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.140 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588075+0000", 2024-09-13T09:14:33.140 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4807000000000002e-05 2024-09-13T09:14:33.140 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.140 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.140 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.141 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20218 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 6281216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.141 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:33.141 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.501254765, 2024-09-13T09:14:33.141 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.049459122, 2024-09-13T09:14:33.141 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.141 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.141 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.141 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.141 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.141 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20218 2024-09-13T09:14:33.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.142 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:33.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538923+0000", 2024-09-13T09:14:33.142 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.142 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.142 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538922+0000", 2024-09-13T09:14:33.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538925+0000", 2024-09-13T09:14:33.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.339e-06 2024-09-13T09:14:33.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.143 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.143 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538931+0000", 2024-09-13T09:14:33.143 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:14:33.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.538935+0000", 2024-09-13T09:14:33.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2089999999999999e-06 2024-09-13T09:14:33.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539495+0000", 2024-09-13T09:14:33.144 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00056018500000000004 2024-09-13T09:14:33.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.144 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.144 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.539502+0000", 2024-09-13T09:14:33.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:14:33.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536688+0000", 2024-09-13T09:14:33.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99718654100000004 2024-09-13T09:14:33.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.145 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.145 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536698+0000", 2024-09-13T09:14:33.145 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6020000000000006e-06 2024-09-13T09:14:33.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588106+0000", 2024-09-13T09:14:33.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.05140794 2024-09-13T09:14:33.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588209+0000", 2024-09-13T09:14:33.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102731 2024-09-13T09:14:33.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.146 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.146 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588382+0000", 2024-09-13T09:14:33.146 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017337799999999999 2024-09-13T09:14:33.146 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.147 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.147 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.147 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20313 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5539840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.147 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:33.147 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.49965413300001, 2024-09-13T09:14:33.147 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0484340209999998, 2024-09-13T09:14:33.147 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.147 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.147 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.147 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.147 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.147 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20313 2024-09-13T09:14:33.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.148 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:33.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.148 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.148 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540524+0000", 2024-09-13T09:14:33.148 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540523+0000", 2024-09-13T09:14:33.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540526+0000", 2024-09-13T09:14:33.149 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:14:33.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.149 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.149 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540533+0000", 2024-09-13T09:14:33.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.092e-06 2024-09-13T09:14:33.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540537+0000", 2024-09-13T09:14:33.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.69e-06 2024-09-13T09:14:33.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.150 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.150 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540608+0000", 2024-09-13T09:14:33.150 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0630000000000006e-05 2024-09-13T09:14:33.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540615+0000", 2024-09-13T09:14:33.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.418e-06 2024-09-13T09:14:33.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.151 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536758+0000", 2024-09-13T09:14:33.151 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99614220099999995 2024-09-13T09:14:33.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.151 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536766+0000", 2024-09-13T09:14:33.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7979999999999995e-06 2024-09-13T09:14:33.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588654+0000", 2024-09-13T09:14:33.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0518873339999999 2024-09-13T09:14:33.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.152 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.152 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588783+0000", 2024-09-13T09:14:33.152 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012956799999999999 2024-09-13T09:14:33.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.153 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.153 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588958+0000", 2024-09-13T09:14:33.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017427999999999999 2024-09-13T09:14:33.153 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.153 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.153 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.153 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20314 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5562368~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.153 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:33.153 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.49961408199999, 2024-09-13T09:14:33.153 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0486442760000001, 2024-09-13T09:14:33.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.154 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.154 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.154 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.154 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.154 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20314 2024-09-13T09:14:33.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.154 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.154 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:33.154 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.154 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540564+0000", 2024-09-13T09:14:33.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540563+0000", 2024-09-13T09:14:33.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.155 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.155 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540566+0000", 2024-09-13T09:14:33.155 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.616e-06 2024-09-13T09:14:33.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540580+0000", 2024-09-13T09:14:33.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3849000000000001e-05 2024-09-13T09:14:33.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540591+0000", 2024-09-13T09:14:33.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1019000000000001e-05 2024-09-13T09:14:33.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.156 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.156 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540644+0000", 2024-09-13T09:14:33.156 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3616999999999998e-05 2024-09-13T09:14:33.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540652+0000", 2024-09-13T09:14:33.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6639999999999998e-06 2024-09-13T09:14:33.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536791+0000", 2024-09-13T09:14:33.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99613924799999998 2024-09-13T09:14:33.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.157 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.157 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536795+0000", 2024-09-13T09:14:33.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.765e-06 2024-09-13T09:14:33.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.588996+0000", 2024-09-13T09:14:33.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052200679 2024-09-13T09:14:33.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589112+0000", 2024-09-13T09:14:33.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011654099999999999 2024-09-13T09:14:33.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589208+0000", 2024-09-13T09:14:33.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5729000000000005e-05 2024-09-13T09:14:33.159 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.159 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.159 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.159 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4903.0:20315 3.12 3:4dfa34b1:::rbd_data.1123d4249472.0000000000000000:10 [read 5584896~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-13T09:14:33.159 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:33.159 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 110.499570319, 2024-09-13T09:14:33.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0489031240000002, 2024-09-13T09:14:33.159 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-09-13T09:14:33.159 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-09-13T09:14:33.159 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-09-13T09:14:33.160 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4903", 2024-09-13T09:14:33.160 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.157:0/4077202819", 2024-09-13T09:14:33.160 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 20315 2024-09-13T09:14:33.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.160 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-09-13T09:14:33.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-09-13T09:14:33.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:33.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-09-13T09:14:33.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:33.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-09-13T09:14:33.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-09-13T09:14:33.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540607+0000", 2024-09-13T09:14:33.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-09-13T09:14:33.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-09-13T09:14:33.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540619+0000", 2024-09-13T09:14:33.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.167e-05 2024-09-13T09:14:33.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-09-13T09:14:33.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540630+0000", 2024-09-13T09:14:33.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1187e-05 2024-09-13T09:14:33.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-09-13T09:14:33.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540635+0000", 2024-09-13T09:14:33.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:14:33.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540984+0000", 2024-09-13T09:14:33.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034897599999999997 2024-09-13T09:14:33.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:42.540994+0000", 2024-09-13T09:14:33.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0299000000000001e-05 2024-09-13T09:14:33.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536829+0000", 2024-09-13T09:14:33.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.99583495 2024-09-13T09:14:33.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-09-13T09:14:33.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:43.536838+0000", 2024-09-13T09:14:33.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1700000000000003e-06 2024-09-13T09:14:33.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-09-13T09:14:33.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589238+0000", 2024-09-13T09:14:33.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.052399648 2024-09-13T09:14:33.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-09-13T09:14:33.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589351+0000", 2024-09-13T09:14:33.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011247999999999999 2024-09-13T09:14:33.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-09-13T09:14:33.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-09-13T09:14:33.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-09-13T09:14:33.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-09-13T09:12:44.589511+0000", 2024-09-13T09:14:33.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015997200000000001 2024-09-13T09:14:33.165 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.165 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.165 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.165 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-09-13T09:14:33.165 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-09-13T09:14:33.165 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:33.167 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:14:33.167 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:33.165+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:33.267 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:33.265+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:33.323 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:33.324 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:33.324 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-09-13T09:14:33.324 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:33.333 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:14:33.368 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:33.366+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:33.451 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:14:33.481 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:14:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-09-13T09:14:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-09-13T09:14:33.481 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-09-13T09:14:33.481 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:14:33.481 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:33.479+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:33.490 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:14:33.574 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:33.572+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:33.574 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-09-13T09:14:33.575 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:14:33.591 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-09-13T09:14:33.592 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:14:33.608 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-09-13T09:14:33.608 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:14:33.625 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-09-13T09:14:33.625 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:14:33.639 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:33.639 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:33.641 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-09-13T09:14:33.642 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:14:33.642 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:14:33.646 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:14:33.646 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:14:33.648 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:14:33.689 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:33.687+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:33.754 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:33.755 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:33.763 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:14:33.772 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:33.771+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:33.865 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:33.865 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:33.872 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:33.871+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:33.874 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:14:33.973 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:33.972+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:33.976 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:33.976 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:33.985 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:14:34.074 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:34.073+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:34.128 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:34.129 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:34.137 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:14:34.179 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:34.177+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:34.277 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:34.275+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:34.280 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:14:34.283 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:34.283 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:34.293 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:14:34.378 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:34.376+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:34.405 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:34.405 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:34.414 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:14:34.479 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:34.477+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:34.522 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:34.523 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:34.531 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:14:34.615 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:34.613+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:34.678 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:34.679 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:34.681 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:34.679+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:34.686 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:14:34.782 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-13T09:14:34.781+0000 7fe79fde2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31627) UID: 0 2024-09-13T09:14:34.834 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:34.834 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:34.843 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:14:34.887 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-09-13T09:14:34.885+0000 7f3799f7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31832) UID: 0 2024-09-13T09:14:34.950 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:34.950 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:34.958 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:14:35.000 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:34.998+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:35.085 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-09-13T09:14:35.083+0000 7fa385d28640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 53433) UID: 0 2024-09-13T09:14:35.106 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:35.106 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:35.115 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:14:35.186 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-09-13T09:14:35.184+0000 7f2a594c2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31623) UID: 0 2024-09-13T09:14:35.254 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-09-13T09:14:35.266 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:35.266 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:35.276 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:14:35.317 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-09-13T09:14:35.315+0000 7eff35b13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31830) UID: 0 2024-09-13T09:14:35.348 INFO:teuthology.orchestra.run.smithi157.stdout:All operations completed A-OK! 2024-09-13T09:14:35.360 DEBUG:teuthology.parallel:result is None 2024-09-13T09:14:35.361 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-13T09:14:35.370 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-13T09:14:35.378 INFO:tasks.thrashosds:joining thrashosds 2024-09-13T09:14:35.424 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:35.424 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:35.433 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:14:35.540 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:35.540 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:35.549 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:14:35.656 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:35.656 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:35.664 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:14:35.775 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:35.776 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:35.785 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:14:35.886 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:35.887 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:35.895 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:14:35.997 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:35.998 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:36.007 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:14:36.110 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:36.110 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:36.118 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:14:36.221 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:36.221 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:36.230 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:14:36.333 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:36.333 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:36.342 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:14:36.461 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:36.461 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:36.470 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:14:36.577 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:36.577 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:36.585 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:14:36.691 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-09-13T09:14:36.691 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:14:37.270 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:14:37.770 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-13T09:14:37.770 INFO:tasks.thrashosds.thrasher:Reviving osd 5 2024-09-13T09:14:37.770 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T09:14:37.770 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T09:14:37.773 INFO:tasks.ceph.osd.5:Started 2024-09-13T09:14:37.773 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:14:37.882 INFO:teuthology.orchestra.run.smithi129.stderr:no valid command found; 10 closest matches: 2024-09-13T09:14:37.883 INFO:teuthology.orchestra.run.smithi129.stderr:0 2024-09-13T09:14:37.883 INFO:teuthology.orchestra.run.smithi129.stderr:1 2024-09-13T09:14:37.883 INFO:teuthology.orchestra.run.smithi129.stderr:2 2024-09-13T09:14:37.883 INFO:teuthology.orchestra.run.smithi129.stderr:abort 2024-09-13T09:14:37.883 INFO:teuthology.orchestra.run.smithi129.stderr:assert 2024-09-13T09:14:37.883 INFO:teuthology.orchestra.run.smithi129.stderr:config diff 2024-09-13T09:14:37.883 INFO:teuthology.orchestra.run.smithi129.stderr:config diff get 2024-09-13T09:14:37.883 INFO:teuthology.orchestra.run.smithi129.stderr:config get 2024-09-13T09:14:37.883 INFO:teuthology.orchestra.run.smithi129.stderr:config help [] 2024-09-13T09:14:37.883 INFO:teuthology.orchestra.run.smithi129.stderr:config set ... 2024-09-13T09:14:37.883 INFO:teuthology.orchestra.run.smithi129.stderr:admin_socket: invalid command 2024-09-13T09:14:37.885 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T09:14:37.885 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-13T09:14:38.289 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-09-13T09:14:38.305 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:14:38.370 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:38.368+0000 7f858f4d6640 -1 Falling back to public interface 2024-09-13T09:14:39.289 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-09-13T09:14:41.884 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:41.882+0000 7f858f4d6640 -1 osd.5 131 log_to_monitors true 2024-09-13T09:14:42.320 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-13T09:14:42.318+0000 7f8583273640 -1 osd.5 145 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:14:42.886 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:14:43.001 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-13T09:14:43.001 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-13T09:14:43.001 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-13T09:14:43.001 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-13T09:14:43.010 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-13T09:14:43.138 INFO:teuthology.orchestra.run.smithi044.stdout:mon.a: {} 2024-09-13T09:14:43.139 INFO:teuthology.orchestra.run.smithi044.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:14:43.149 INFO:teuthology.orchestra.run.smithi044.stdout:mon.b: {} 2024-09-13T09:14:43.149 INFO:teuthology.orchestra.run.smithi044.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:14:43.157 INFO:teuthology.orchestra.run.smithi044.stdout:mon.c: {} 2024-09-13T09:14:43.158 INFO:teuthology.orchestra.run.smithi044.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:14:43.169 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T09:14:44.306 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T09:14:44.325 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-13T09:14:45.323 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-13T09:14:45.339 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T09:14:46.331 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T09:14:46.347 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-13T09:14:47.337 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-13T09:14:47.355 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T09:14:48.347 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T09:14:48.365 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-13T09:14:49.347 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-13T09:14:49.364 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T09:14:50.255 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T09:14:50.272 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-13T09:14:51.357 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-13T09:14:51.374 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T09:14:52.364 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T09:14:52.380 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-13T09:14:53.370 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-13T09:14:53.387 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T09:14:54.377 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T09:14:54.394 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-13T09:14:55.382 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-13T09:14:55.399 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-13T09:14:55.399 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-13T09:14:55.399 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-13T09:14:55.399 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-13T09:14:55.399 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T09:14:55.399 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:14:55.692 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:14:55.692 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":158,"fsid":"06cda68b-dd73-4266-b5e6-93ae2b3f7bd5","created":"2024-09-13T09:11:45.458147+0000","modified":"2024-09-13T09:14:55.377898+0000","last_up_change":"2024-09-13T09:14:43.297335+0000","last_in_change":"2024-09-13T09:12:42.513669+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:11:57.562930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T09:12:03.214789+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T09:12:12.120436+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":79,"snap_epoch":142,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"156c6730-f804-4806-bc2d-3a7d0856c2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6810","nonce":2268459392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6815","nonce":2268459392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6823","nonce":2268459392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6817","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6819","nonce":2268459392}]},"public_addr":"172.21.15.44:6810/2268459392","cluster_addr":"172.21.15.44:6815/2268459392","heartbeat_back_addr":"172.21.15.44:6823/2268459392","heartbeat_front_addr":"172.21.15.44:6819/2268459392","state":["exists","up"]},{"osd":1,"uuid":"00735025-3bf9-4dd0-8b42-f08f1bd1c421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6801","nonce":2209612586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6803","nonce":2209612586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6807","nonce":2209612586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6805","nonce":2209612586}]},"public_addr":"172.21.15.44:6801/2209612586","cluster_addr":"172.21.15.44:6803/2209612586","heartbeat_back_addr":"172.21.15.44:6807/2209612586","heartbeat_front_addr":"172.21.15.44:6805/2209612586","state":["exists","up"]},{"osd":2,"uuid":"395516ac-33b6-4f43-8333-dcb1f4cded32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":84,"up_from":122,"up_thru":133,"down_at":85,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":2109106906},{"type":"v1","addr":"172.21.15.44:6811","nonce":2109106906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2109106906},{"type":"v1","addr":"172.21.15.44:6814","nonce":2109106906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2109106906},{"type":"v1","addr":"172.21.15.44:6822","nonce":2109106906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2109106906},{"type":"v1","addr":"172.21.15.44:6818","nonce":2109106906}]},"public_addr":"172.21.15.44:6811/2109106906","cluster_addr":"172.21.15.44:6814/2109106906","heartbeat_back_addr":"172.21.15.44:6822/2109106906","heartbeat_front_addr":"172.21.15.44:6818/2109106906","state":["exists","up"]},{"osd":3,"uuid":"429e877f-5462-402a-80e5-92a7b17a2eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6809","nonce":2132151186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6811","nonce":2132151186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6815","nonce":2132151186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6813","nonce":2132151186}]},"public_addr":"172.21.15.129:6809/2132151186","cluster_addr":"172.21.15.129:6811/2132151186","heartbeat_back_addr":"172.21.15.129:6815/2132151186","heartbeat_front_addr":"172.21.15.129:6813/2132151186","state":["exists","up"]},{"osd":4,"uuid":"9b3be757-15ee-4157-80a8-0537cd1bcf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393542416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393542416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393542416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393542416}]},"public_addr":"172.21.15.129:6801/3393542416","cluster_addr":"172.21.15.129:6803/3393542416","heartbeat_back_addr":"172.21.15.129:6807/3393542416","heartbeat_front_addr":"172.21.15.129:6805/3393542416","state":["exists","up"]},{"osd":5,"uuid":"c7601d5e-2164-45e1-ab21-368d9e3af27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":131,"up_from":146,"up_thru":146,"down_at":133,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1674281758},{"type":"v1","addr":"172.21.15.129:6817","nonce":1674281758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1674281758},{"type":"v1","addr":"172.21.15.129:6819","nonce":1674281758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1674281758},{"type":"v1","addr":"172.21.15.129:6823","nonce":1674281758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1674281758},{"type":"v1","addr":"172.21.15.129:6821","nonce":1674281758}]},"public_addr":"172.21.15.129:6817/1674281758","cluster_addr":"172.21.15.129:6819/1674281758","heartbeat_back_addr":"172.21.15.129:6823/1674281758","heartbeat_front_addr":"172.21.15.129:6821/1674281758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.230474+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.195476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.248729+0000","dead_epoch":85},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.195482+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.198746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.279957+0000","dead_epoch":133}],"pg_upmap":[{"pgid":"2.7","osds":[5,4]},{"pgid":"3.0","osds":[5,1]},{"pgid":"3.1b","osds":[2,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:14:55.704 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T09:14:55.705 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:14:55.988 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:14:55.989 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":158,"fsid":"06cda68b-dd73-4266-b5e6-93ae2b3f7bd5","created":"2024-09-13T09:11:45.458147+0000","modified":"2024-09-13T09:14:55.377898+0000","last_up_change":"2024-09-13T09:14:43.297335+0000","last_in_change":"2024-09-13T09:12:42.513669+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:11:57.562930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T09:12:03.214789+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T09:12:12.120436+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":79,"snap_epoch":142,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"156c6730-f804-4806-bc2d-3a7d0856c2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6810","nonce":2268459392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6815","nonce":2268459392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6823","nonce":2268459392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6817","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6819","nonce":2268459392}]},"public_addr":"172.21.15.44:6810/2268459392","cluster_addr":"172.21.15.44:6815/2268459392","heartbeat_back_addr":"172.21.15.44:6823/2268459392","heartbeat_front_addr":"172.21.15.44:6819/2268459392","state":["exists","up"]},{"osd":1,"uuid":"00735025-3bf9-4dd0-8b42-f08f1bd1c421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6801","nonce":2209612586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6803","nonce":2209612586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6807","nonce":2209612586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6805","nonce":2209612586}]},"public_addr":"172.21.15.44:6801/2209612586","cluster_addr":"172.21.15.44:6803/2209612586","heartbeat_back_addr":"172.21.15.44:6807/2209612586","heartbeat_front_addr":"172.21.15.44:6805/2209612586","state":["exists","up"]},{"osd":2,"uuid":"395516ac-33b6-4f43-8333-dcb1f4cded32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":84,"up_from":122,"up_thru":133,"down_at":85,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":2109106906},{"type":"v1","addr":"172.21.15.44:6811","nonce":2109106906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2109106906},{"type":"v1","addr":"172.21.15.44:6814","nonce":2109106906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2109106906},{"type":"v1","addr":"172.21.15.44:6822","nonce":2109106906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2109106906},{"type":"v1","addr":"172.21.15.44:6818","nonce":2109106906}]},"public_addr":"172.21.15.44:6811/2109106906","cluster_addr":"172.21.15.44:6814/2109106906","heartbeat_back_addr":"172.21.15.44:6822/2109106906","heartbeat_front_addr":"172.21.15.44:6818/2109106906","state":["exists","up"]},{"osd":3,"uuid":"429e877f-5462-402a-80e5-92a7b17a2eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6809","nonce":2132151186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6811","nonce":2132151186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6815","nonce":2132151186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6813","nonce":2132151186}]},"public_addr":"172.21.15.129:6809/2132151186","cluster_addr":"172.21.15.129:6811/2132151186","heartbeat_back_addr":"172.21.15.129:6815/2132151186","heartbeat_front_addr":"172.21.15.129:6813/2132151186","state":["exists","up"]},{"osd":4,"uuid":"9b3be757-15ee-4157-80a8-0537cd1bcf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393542416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393542416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393542416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393542416}]},"public_addr":"172.21.15.129:6801/3393542416","cluster_addr":"172.21.15.129:6803/3393542416","heartbeat_back_addr":"172.21.15.129:6807/3393542416","heartbeat_front_addr":"172.21.15.129:6805/3393542416","state":["exists","up"]},{"osd":5,"uuid":"c7601d5e-2164-45e1-ab21-368d9e3af27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":131,"up_from":146,"up_thru":146,"down_at":133,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1674281758},{"type":"v1","addr":"172.21.15.129:6817","nonce":1674281758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1674281758},{"type":"v1","addr":"172.21.15.129:6819","nonce":1674281758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1674281758},{"type":"v1","addr":"172.21.15.129:6823","nonce":1674281758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1674281758},{"type":"v1","addr":"172.21.15.129:6821","nonce":1674281758}]},"public_addr":"172.21.15.129:6817/1674281758","cluster_addr":"172.21.15.129:6819/1674281758","heartbeat_back_addr":"172.21.15.129:6823/1674281758","heartbeat_front_addr":"172.21.15.129:6821/1674281758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.230474+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.195476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.248729+0000","dead_epoch":85},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.195482+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.198746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.279957+0000","dead_epoch":133}],"pg_upmap":[{"pgid":"2.7","osds":[5,4]},{"pgid":"3.0","osds":[5,1]},{"pgid":"3.1b","osds":[2,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:14:56.002 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T09:14:56.002 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T09:14:56.003 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T09:14:56.003 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T09:14:56.003 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T09:14:56.004 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T09:14:56.184 INFO:teuthology.orchestra.run.smithi044.stdout:523986010122 2024-09-13T09:14:56.185 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:14:56.186 INFO:teuthology.orchestra.run.smithi044.stdout:47244640296 2024-09-13T09:14:56.186 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:14:56.208 INFO:teuthology.orchestra.run.smithi044.stdout:47244640296 2024-09-13T09:14:56.208 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:14:56.210 INFO:teuthology.orchestra.run.smithi044.stdout:627065225220 2024-09-13T09:14:56.210 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:14:56.228 INFO:teuthology.orchestra.run.smithi044.stdout:47244640295 2024-09-13T09:14:56.229 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:14:56.229 INFO:teuthology.orchestra.run.smithi044.stdout:47244640297 2024-09-13T09:14:56.230 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:14:56.537 INFO:teuthology.orchestra.run.smithi044.stdout:47244640295 2024-09-13T09:14:56.541 INFO:teuthology.orchestra.run.smithi044.stdout:523986010121 2024-09-13T09:14:56.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640296 got 47244640295 for osd.4 2024-09-13T09:14:56.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 523986010122 got 523986010121 for osd.2 2024-09-13T09:14:56.641 INFO:teuthology.orchestra.run.smithi044.stdout:627065225219 2024-09-13T09:14:56.647 INFO:teuthology.orchestra.run.smithi044.stdout:47244640295 2024-09-13T09:14:56.655 INFO:tasks.ceph.ceph_manager.ceph:need seq 627065225220 got 627065225219 for osd.5 2024-09-13T09:14:56.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640296 got 47244640295 for osd.1 2024-09-13T09:14:56.692 INFO:teuthology.orchestra.run.smithi044.stdout:47244640296 2024-09-13T09:14:56.701 INFO:teuthology.orchestra.run.smithi044.stdout:47244640294 2024-09-13T09:14:56.703 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640297 got 47244640296 for osd.3 2024-09-13T09:14:56.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640295 got 47244640294 for osd.0 2024-09-13T09:14:57.553 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:14:57.556 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:14:57.656 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:14:57.666 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:14:57.705 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:14:57.716 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:14:57.936 INFO:teuthology.orchestra.run.smithi044.stdout:47244640295 2024-09-13T09:14:57.956 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640296 got 47244640295 for osd.4 2024-09-13T09:14:57.969 INFO:teuthology.orchestra.run.smithi044.stdout:523986010121 2024-09-13T09:14:57.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 523986010122 got 523986010121 for osd.2 2024-09-13T09:14:58.060 INFO:teuthology.orchestra.run.smithi044.stdout:627065225219 2024-09-13T09:14:58.068 INFO:teuthology.orchestra.run.smithi044.stdout:47244640295 2024-09-13T09:14:58.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 627065225220 got 627065225219 for osd.5 2024-09-13T09:14:58.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640296 got 47244640295 for osd.1 2024-09-13T09:14:58.099 INFO:teuthology.orchestra.run.smithi044.stdout:47244640296 2024-09-13T09:14:58.104 INFO:teuthology.orchestra.run.smithi044.stdout:47244640294 2024-09-13T09:14:58.112 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640297 got 47244640296 for osd.3 2024-09-13T09:14:58.117 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640295 got 47244640294 for osd.0 2024-09-13T09:14:58.956 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:14:58.989 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:14:59.075 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:14:59.082 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:14:59.113 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:14:59.118 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:14:59.340 INFO:teuthology.orchestra.run.smithi044.stdout:47244640297 2024-09-13T09:14:59.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640296 got 47244640297 for osd.4 2024-09-13T09:14:59.354 DEBUG:teuthology.parallel:result is None 2024-09-13T09:14:59.374 INFO:teuthology.orchestra.run.smithi044.stdout:523986010122 2024-09-13T09:14:59.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 523986010122 got 523986010122 for osd.2 2024-09-13T09:14:59.393 DEBUG:teuthology.parallel:result is None 2024-09-13T09:14:59.472 INFO:teuthology.orchestra.run.smithi044.stdout:47244640298 2024-09-13T09:14:59.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640297 got 47244640298 for osd.3 2024-09-13T09:14:59.489 DEBUG:teuthology.parallel:result is None 2024-09-13T09:14:59.510 INFO:teuthology.orchestra.run.smithi044.stdout:47244640297 2024-09-13T09:14:59.519 INFO:teuthology.orchestra.run.smithi044.stdout:627065225221 2024-09-13T09:14:59.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640296 got 47244640297 for osd.1 2024-09-13T09:14:59.524 DEBUG:teuthology.parallel:result is None 2024-09-13T09:14:59.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 627065225220 got 627065225221 for osd.5 2024-09-13T09:14:59.533 DEBUG:teuthology.parallel:result is None 2024-09-13T09:14:59.541 INFO:teuthology.orchestra.run.smithi044.stdout:47244640296 2024-09-13T09:14:59.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640295 got 47244640296 for osd.0 2024-09-13T09:14:59.554 DEBUG:teuthology.parallel:result is None 2024-09-13T09:14:59.554 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T09:14:59.555 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:14:59.879 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:14:59.880 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:14:59.890 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-13T09:14:59.549222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":68,"num_read":356867,"num_read_kb":1364416,"num_write":26735,"num_write_kb":362137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":666,"num_bytes_recovered":19347930,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23113,"ondisk_log_size":23113,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":309988,"kb_used_data":12220,"kb_used_omap":74,"kb_used_meta":297589,"kb_avail":565921052,"statfs":{"total":579820584960,"available":579503157248,"internally_reserved":0,"allocated":12513280,"data_stored":5913802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76719,"internal_metadata":304731217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.181465"},"pg_stats":[{"pgid":"3.1f","version":"142'857","reported_seq":9653,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375199+0000","last_change":"2024-09-13T09:14:35.254388+0000","last_active":"2024-09-13T09:14:54.375199+0000","last_peered":"2024-09-13T09:14:54.375199+0000","last_clean":"2024-09-13T09:14:54.375199+0000","last_became_active":"2024-09-13T09:12:43.534990+0000","last_became_peered":"2024-09-13T09:12:43.534990+0000","last_unstale":"2024-09-13T09:14:54.375199+0000","last_undegraded":"2024-09-13T09:14:54.375199+0000","last_fullsized":"2024-09-13T09:14:54.375199+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:45.810610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093900099999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":11993,"num_read_kb":21637,"num_write":921,"num_write_kb":7095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":848912,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"140'605","reported_seq":7893,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375888+0000","last_change":"2024-09-13T09:14:35.256646+0000","last_active":"2024-09-13T09:14:54.375888+0000","last_peered":"2024-09-13T09:14:54.375888+0000","last_clean":"2024-09-13T09:14:54.375888+0000","last_became_active":"2024-09-13T09:14:16.749234+0000","last_became_peered":"2024-09-13T09:14:16.749234+0000","last_unstale":"2024-09-13T09:14:54.375888+0000","last_undegraded":"2024-09-13T09:14:54.375888+0000","last_fullsized":"2024-09-13T09:14:54.375888+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:18:42.349049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020237760000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10031,"num_read_kb":30858,"num_write":737,"num_write_kb":10943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":313856,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"147'1435","reported_seq":16444,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688945+0000","last_change":"2024-09-13T09:14:44.400159+0000","last_active":"2024-09-13T09:14:54.688945+0000","last_peered":"2024-09-13T09:14:54.688945+0000","last_clean":"2024-09-13T09:14:54.688945+0000","last_became_active":"2024-09-13T09:14:44.312734+0000","last_became_peered":"2024-09-13T09:14:44.312734+0000","last_unstale":"2024-09-13T09:14:54.688945+0000","last_undegraded":"2024-09-13T09:14:54.688945+0000","last_fullsized":"2024-09-13T09:14:54.688945+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1435,"log_dups_size":0,"ondisk_log_size":1435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.934438+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011136765999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":20060,"num_read_kb":34855,"num_write":1554,"num_write_kb":7371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"147'1480","reported_seq":8609,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689014+0000","last_change":"2024-09-13T09:14:44.511014+0000","last_active":"2024-09-13T09:14:54.689014+0000","last_peered":"2024-09-13T09:14:54.689014+0000","last_clean":"2024-09-13T09:14:54.689014+0000","last_became_active":"2024-09-13T09:14:44.311550+0000","last_became_peered":"2024-09-13T09:14:44.311550+0000","last_unstale":"2024-09-13T09:14:54.689014+0000","last_undegraded":"2024-09-13T09:14:54.689014+0000","last_fullsized":"2024-09-13T09:14:54.689014+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1480,"log_dups_size":0,"ondisk_log_size":1480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:16:38.500976+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.022244201000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13526,"num_read_kb":55870,"num_write":2225,"num_write_kb":20106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"142'635","reported_seq":9508,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375955+0000","last_change":"2024-09-13T09:14:35.256035+0000","last_active":"2024-09-13T09:14:54.375955+0000","last_peered":"2024-09-13T09:14:54.375955+0000","last_clean":"2024-09-13T09:14:54.375955+0000","last_became_active":"2024-09-13T09:14:19.772731+0000","last_became_peered":"2024-09-13T09:14:19.772731+0000","last_unstale":"2024-09-13T09:14:54.375955+0000","last_undegraded":"2024-09-13T09:14:54.375955+0000","last_fullsized":"2024-09-13T09:14:54.375955+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:24:40.563709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001509144,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10475,"num_read_kb":52366,"num_write":680,"num_write_kb":12577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1411144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"141'628","reported_seq":14444,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189254+0000","last_change":"2024-09-13T09:14:35.259843+0000","last_active":"2024-09-13T09:14:55.189254+0000","last_peered":"2024-09-13T09:14:55.189254+0000","last_clean":"2024-09-13T09:14:55.189254+0000","last_became_active":"2024-09-13T09:12:13.362533+0000","last_became_peered":"2024-09-13T09:12:13.362533+0000","last_unstale":"2024-09-13T09:14:55.189254+0000","last_undegraded":"2024-09-13T09:14:55.189254+0000","last_fullsized":"2024-09-13T09:14:55.189254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:50.399315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017407639999999999,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":16059,"num_read_kb":55558,"num_write":617,"num_write_kb":11980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"138'567","reported_seq":8658,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376230+0000","last_change":"2024-09-13T09:14:48.352186+0000","last_active":"2024-09-13T09:14:54.376230+0000","last_peered":"2024-09-13T09:14:54.376230+0000","last_clean":"2024-09-13T09:14:54.376230+0000","last_became_active":"2024-09-13T09:14:48.350477+0000","last_became_peered":"2024-09-13T09:14:48.350477+0000","last_unstale":"2024-09-13T09:14:54.376230+0000","last_undegraded":"2024-09-13T09:14:54.376230+0000","last_fullsized":"2024-09-13T09:14:54.376230+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:57.759426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074163899999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8163,"num_read_kb":27760,"num_write":669,"num_write_kb":11976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"139'576","reported_seq":11071,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375944+0000","last_change":"2024-09-13T09:14:35.255587+0000","last_active":"2024-09-13T09:14:54.375944+0000","last_peered":"2024-09-13T09:14:54.375944+0000","last_clean":"2024-09-13T09:14:54.375944+0000","last_became_active":"2024-09-13T09:14:19.772799+0000","last_became_peered":"2024-09-13T09:14:19.772799+0000","last_unstale":"2024-09-13T09:14:54.375944+0000","last_undegraded":"2024-09-13T09:14:54.375944+0000","last_fullsized":"2024-09-13T09:14:54.375944+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:41:21.622976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001235604,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13056,"num_read_kb":24986,"num_write":439,"num_write_kb":7320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":104960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"147'434","reported_seq":5711,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375290+0000","last_change":"2024-09-13T09:14:44.358128+0000","last_active":"2024-09-13T09:14:54.375290+0000","last_peered":"2024-09-13T09:14:54.375290+0000","last_clean":"2024-09-13T09:14:54.375290+0000","last_became_active":"2024-09-13T09:14:44.317071+0000","last_became_peered":"2024-09-13T09:14:44.317071+0000","last_unstale":"2024-09-13T09:14:54.375290+0000","last_undegraded":"2024-09-13T09:14:54.375290+0000","last_fullsized":"2024-09-13T09:14:54.375290+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:42.567052+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015690497000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6024,"num_read_kb":31779,"num_write":503,"num_write_kb":7984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"147'972","reported_seq":6743,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689023+0000","last_change":"2024-09-13T09:14:44.415816+0000","last_active":"2024-09-13T09:14:54.689023+0000","last_peered":"2024-09-13T09:14:54.689023+0000","last_clean":"2024-09-13T09:14:54.689023+0000","last_became_active":"2024-09-13T09:14:44.309746+0000","last_became_peered":"2024-09-13T09:14:44.309746+0000","last_unstale":"2024-09-13T09:14:54.689023+0000","last_undegraded":"2024-09-13T09:14:54.689023+0000","last_fullsized":"2024-09-13T09:14:54.689023+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":972,"log_dups_size":0,"ondisk_log_size":972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:16:58.217708+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0084025269999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8520,"num_read_kb":43161,"num_write":1440,"num_write_kb":11751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":706121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"139'872","reported_seq":20338,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189647+0000","last_change":"2024-09-13T09:14:35.256574+0000","last_active":"2024-09-13T09:14:55.189647+0000","last_peered":"2024-09-13T09:14:55.189647+0000","last_clean":"2024-09-13T09:14:55.189647+0000","last_became_active":"2024-09-13T09:12:46.250696+0000","last_became_peered":"2024-09-13T09:12:46.250696+0000","last_unstale":"2024-09-13T09:14:55.189647+0000","last_undegraded":"2024-09-13T09:14:55.189647+0000","last_fullsized":"2024-09-13T09:14:55.189647+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:43.821997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019682329999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":21840,"num_read_kb":55389,"num_write":1298,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1136450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"140'665","reported_seq":8589,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189530+0000","last_change":"2024-09-13T09:14:35.255745+0000","last_active":"2024-09-13T09:14:55.189530+0000","last_peered":"2024-09-13T09:14:55.189530+0000","last_clean":"2024-09-13T09:14:55.189530+0000","last_became_active":"2024-09-13T09:12:46.254016+0000","last_became_peered":"2024-09-13T09:12:46.254016+0000","last_unstale":"2024-09-13T09:14:55.189530+0000","last_undegraded":"2024-09-13T09:14:55.189530+0000","last_fullsized":"2024-09-13T09:14:55.189530+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:50.193567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013411219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10880,"num_read_kb":53346,"num_write":603,"num_write_kb":9445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1110608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688996+0000","last_change":"2024-09-13T09:14:44.316371+0000","last_active":"2024-09-13T09:14:54.688996+0000","last_peered":"2024-09-13T09:14:54.688996+0000","last_clean":"2024-09-13T09:14:54.688996+0000","last_became_active":"2024-09-13T09:14:44.316062+0000","last_became_peered":"2024-09-13T09:14:44.316062+0000","last_unstale":"2024-09-13T09:14:54.688996+0000","last_undegraded":"2024-09-13T09:14:54.688996+0000","last_fullsized":"2024-09-13T09:14:54.688996+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:06:27.687674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"142'753","reported_seq":10733,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375022+0000","last_change":"2024-09-13T09:14:35.253996+0000","last_active":"2024-09-13T09:14:54.375022+0000","last_peered":"2024-09-13T09:14:54.375022+0000","last_clean":"2024-09-13T09:14:54.375022+0000","last_became_active":"2024-09-13T09:12:13.365496+0000","last_became_peered":"2024-09-13T09:12:13.365496+0000","last_unstale":"2024-09-13T09:14:54.375022+0000","last_undegraded":"2024-09-13T09:14:54.375022+0000","last_fullsized":"2024-09-13T09:14:54.375022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:29:49.115977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063501900000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11213,"num_read_kb":65815,"num_write":999,"num_write_kb":12170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":156,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376437+0000","last_change":"2024-09-13T09:14:48.352849+0000","last_active":"2024-09-13T09:14:54.376437+0000","last_peered":"2024-09-13T09:14:54.376437+0000","last_clean":"2024-09-13T09:14:54.376437+0000","last_became_active":"2024-09-13T09:14:48.352502+0000","last_became_peered":"2024-09-13T09:14:48.352502+0000","last_unstale":"2024-09-13T09:14:54.376437+0000","last_undegraded":"2024-09-13T09:14:54.376437+0000","last_fullsized":"2024-09-13T09:14:54.376437+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:04.532498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"141'1047","reported_seq":10558,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375958+0000","last_change":"2024-09-13T09:14:35.256434+0000","last_active":"2024-09-13T09:14:54.375958+0000","last_peered":"2024-09-13T09:14:54.375958+0000","last_clean":"2024-09-13T09:14:54.375958+0000","last_became_active":"2024-09-13T09:14:16.752774+0000","last_became_peered":"2024-09-13T09:14:16.752774+0000","last_unstale":"2024-09-13T09:14:54.375958+0000","last_undegraded":"2024-09-13T09:14:54.375958+0000","last_fullsized":"2024-09-13T09:14:54.375958+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1047,"log_dups_size":0,"ondisk_log_size":1047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:37:58.086486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001863822,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":16931,"num_read_kb":50979,"num_write":1279,"num_write_kb":12713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":413184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":121,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189554+0000","last_change":"2024-09-13T09:12:43.533829+0000","last_active":"2024-09-13T09:14:55.189554+0000","last_peered":"2024-09-13T09:14:55.189554+0000","last_clean":"2024-09-13T09:14:55.189554+0000","last_became_active":"2024-09-13T09:12:43.533557+0000","last_became_peered":"2024-09-13T09:12:43.533557+0000","last_unstale":"2024-09-13T09:14:55.189554+0000","last_undegraded":"2024-09-13T09:14:55.189554+0000","last_fullsized":"2024-09-13T09:14:55.189554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:51.939164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"141'707","reported_seq":13355,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376484+0000","last_change":"2024-09-13T09:14:48.351345+0000","last_active":"2024-09-13T09:14:54.376484+0000","last_peered":"2024-09-13T09:14:54.376484+0000","last_clean":"2024-09-13T09:14:54.376484+0000","last_became_active":"2024-09-13T09:14:48.348909+0000","last_became_peered":"2024-09-13T09:14:48.348909+0000","last_unstale":"2024-09-13T09:14:54.376484+0000","last_undegraded":"2024-09-13T09:14:54.376484+0000","last_fullsized":"2024-09-13T09:14:54.376484+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:55:37.482616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074057,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":15357,"num_read_kb":39900,"num_write":798,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":29184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376531+0000","last_change":"2024-09-13T09:14:48.351236+0000","last_active":"2024-09-13T09:14:54.376531+0000","last_peered":"2024-09-13T09:14:54.376531+0000","last_clean":"2024-09-13T09:14:54.376531+0000","last_became_active":"2024-09-13T09:14:48.348560+0000","last_became_peered":"2024-09-13T09:14:48.348560+0000","last_unstale":"2024-09-13T09:14:54.376531+0000","last_undegraded":"2024-09-13T09:14:54.376531+0000","last_fullsized":"2024-09-13T09:14:54.376531+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:21.050148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"142'748","reported_seq":7661,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688947+0000","last_change":"2024-09-13T09:14:44.367184+0000","last_active":"2024-09-13T09:14:54.688947+0000","last_peered":"2024-09-13T09:14:54.688947+0000","last_clean":"2024-09-13T09:14:54.688947+0000","last_became_active":"2024-09-13T09:14:44.309435+0000","last_became_peered":"2024-09-13T09:14:44.309435+0000","last_unstale":"2024-09-13T09:14:54.688947+0000","last_undegraded":"2024-09-13T09:14:54.688947+0000","last_fullsized":"2024-09-13T09:14:54.688947+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:57.739078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039806149999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10405,"num_read_kb":27276,"num_write":715,"num_write_kb":2474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":182784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"142'634","reported_seq":11576,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189327+0000","last_change":"2024-09-13T09:14:35.258893+0000","last_active":"2024-09-13T09:14:55.189327+0000","last_peered":"2024-09-13T09:14:55.189327+0000","last_clean":"2024-09-13T09:14:55.189327+0000","last_became_active":"2024-09-13T09:12:13.367770+0000","last_became_peered":"2024-09-13T09:12:13.367770+0000","last_unstale":"2024-09-13T09:14:55.189327+0000","last_undegraded":"2024-09-13T09:14:55.189327+0000","last_fullsized":"2024-09-13T09:14:55.189327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:21:50.690425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001191176,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10993,"num_read_kb":67376,"num_write":601,"num_write_kb":14012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":122,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688871+0000","last_change":"2024-09-13T09:14:44.312125+0000","last_active":"2024-09-13T09:14:54.688871+0000","last_peered":"2024-09-13T09:14:54.688871+0000","last_clean":"2024-09-13T09:14:54.688871+0000","last_became_active":"2024-09-13T09:14:44.311178+0000","last_became_peered":"2024-09-13T09:14:44.311178+0000","last_unstale":"2024-09-13T09:14:54.688871+0000","last_undegraded":"2024-09-13T09:14:54.688871+0000","last_fullsized":"2024-09-13T09:14:54.688871+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:46.245017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"147'620","reported_seq":9651,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688842+0000","last_change":"2024-09-13T09:14:44.440284+0000","last_active":"2024-09-13T09:14:54.688842+0000","last_peered":"2024-09-13T09:14:54.688842+0000","last_clean":"2024-09-13T09:14:54.688842+0000","last_became_active":"2024-09-13T09:14:44.310016+0000","last_became_peered":"2024-09-13T09:14:44.310016+0000","last_unstale":"2024-09-13T09:14:54.688842+0000","last_undegraded":"2024-09-13T09:14:54.688842+0000","last_fullsized":"2024-09-13T09:14:54.688842+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:55.802300+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.007574824,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":9521,"num_read_kb":48768,"num_write":764,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1695776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376024+0000","last_change":"2024-09-13T09:14:16.749189+0000","last_active":"2024-09-13T09:14:54.376024+0000","last_peered":"2024-09-13T09:14:54.376024+0000","last_clean":"2024-09-13T09:14:54.376024+0000","last_became_active":"2024-09-13T09:14:16.748865+0000","last_became_peered":"2024-09-13T09:14:16.748865+0000","last_unstale":"2024-09-13T09:14:54.376024+0000","last_undegraded":"2024-09-13T09:14:54.376024+0000","last_fullsized":"2024-09-13T09:14:54.376024+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:25:39.217684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"142'332","reported_seq":3860,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.374904+0000","last_change":"2024-09-13T09:14:35.253968+0000","last_active":"2024-09-13T09:14:54.374904+0000","last_peered":"2024-09-13T09:14:54.374904+0000","last_clean":"2024-09-13T09:14:54.374904+0000","last_became_active":"2024-09-13T09:12:13.367633+0000","last_became_peered":"2024-09-13T09:12:13.367633+0000","last_unstale":"2024-09-13T09:14:54.374904+0000","last_undegraded":"2024-09-13T09:14:54.374904+0000","last_fullsized":"2024-09-13T09:14:54.374904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:34.734632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094036499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3280,"num_read_kb":26753,"num_write":312,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189164+0000","last_change":"2024-09-13T09:12:43.532201+0000","last_active":"2024-09-13T09:14:55.189164+0000","last_peered":"2024-09-13T09:14:55.189164+0000","last_clean":"2024-09-13T09:14:55.189164+0000","last_became_active":"2024-09-13T09:12:43.531942+0000","last_became_peered":"2024-09-13T09:12:43.531942+0000","last_unstale":"2024-09-13T09:14:55.189164+0000","last_undegraded":"2024-09-13T09:14:55.189164+0000","last_fullsized":"2024-09-13T09:14:55.189164+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:26.408883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"141'653","reported_seq":3816,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376444+0000","last_change":"2024-09-13T09:14:48.351027+0000","last_active":"2024-09-13T09:14:54.376444+0000","last_peered":"2024-09-13T09:14:54.376444+0000","last_clean":"2024-09-13T09:14:54.376444+0000","last_became_active":"2024-09-13T09:14:48.349138+0000","last_became_peered":"2024-09-13T09:14:48.349138+0000","last_unstale":"2024-09-13T09:14:54.376444+0000","last_undegraded":"2024-09-13T09:14:54.376444+0000","last_fullsized":"2024-09-13T09:14:54.376444+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:46.295942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039583799999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4158,"num_read_kb":20449,"num_write":848,"num_write_kb":5187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":155,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688806+0000","last_change":"2024-09-13T09:14:44.309742+0000","last_active":"2024-09-13T09:14:54.688806+0000","last_peered":"2024-09-13T09:14:54.688806+0000","last_clean":"2024-09-13T09:14:54.688806+0000","last_became_active":"2024-09-13T09:14:44.309293+0000","last_became_peered":"2024-09-13T09:14:44.309293+0000","last_unstale":"2024-09-13T09:14:54.688806+0000","last_undegraded":"2024-09-13T09:14:54.688806+0000","last_fullsized":"2024-09-13T09:14:54.688806+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:28:57.831071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":324,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189255+0000","last_change":"2024-09-13T09:12:46.248714+0000","last_active":"2024-09-13T09:14:55.189255+0000","last_peered":"2024-09-13T09:14:55.189255+0000","last_clean":"2024-09-13T09:14:55.189255+0000","last_became_active":"2024-09-13T09:12:46.248451+0000","last_became_peered":"2024-09-13T09:12:46.248451+0000","last_unstale":"2024-09-13T09:14:55.189255+0000","last_undegraded":"2024-09-13T09:14:55.189255+0000","last_fullsized":"2024-09-13T09:14:55.189255+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:27.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"141'498","reported_seq":4698,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376040+0000","last_change":"2024-09-13T09:14:35.257020+0000","last_active":"2024-09-13T09:14:54.376040+0000","last_peered":"2024-09-13T09:14:54.376040+0000","last_clean":"2024-09-13T09:14:54.376040+0000","last_became_active":"2024-09-13T09:14:16.754685+0000","last_became_peered":"2024-09-13T09:14:16.754685+0000","last_unstale":"2024-09-13T09:14:54.376040+0000","last_undegraded":"2024-09-13T09:14:54.376040+0000","last_fullsized":"2024-09-13T09:14:54.376040+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:04.966596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022974409999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4733,"num_read_kb":22943,"num_write":604,"num_write_kb":15013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":45056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"142'1216","reported_seq":13783,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189219+0000","last_change":"2024-09-13T09:14:35.259629+0000","last_active":"2024-09-13T09:14:55.189219+0000","last_peered":"2024-09-13T09:14:55.189219+0000","last_clean":"2024-09-13T09:14:55.189219+0000","last_became_active":"2024-09-13T09:14:19.774300+0000","last_became_peered":"2024-09-13T09:14:19.774300+0000","last_unstale":"2024-09-13T09:14:55.189219+0000","last_undegraded":"2024-09-13T09:14:55.189219+0000","last_fullsized":"2024-09-13T09:14:55.189219+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1216,"log_dups_size":0,"ondisk_log_size":1216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:36:43.693732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001648159,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18678,"num_read_kb":75566,"num_write":1537,"num_write_kb":13034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1249280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"141'596","reported_seq":11902,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.381858+0000","last_change":"2024-09-13T09:14:35.256213+0000","last_active":"2024-09-13T09:14:55.381858+0000","last_peered":"2024-09-13T09:14:55.381858+0000","last_clean":"2024-09-13T09:14:55.381858+0000","last_became_active":"2024-09-13T09:12:46.252578+0000","last_became_peered":"2024-09-13T09:12:46.252578+0000","last_unstale":"2024-09-13T09:14:55.381858+0000","last_undegraded":"2024-09-13T09:14:55.381858+0000","last_fullsized":"2024-09-13T09:14:55.381858+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:03.931492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001634775,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":12679,"num_read_kb":50965,"num_write":515,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2590736,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"141'525","reported_seq":8198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376250+0000","last_change":"2024-09-13T09:14:48.351125+0000","last_active":"2024-09-13T09:14:54.376250+0000","last_peered":"2024-09-13T09:14:54.376250+0000","last_clean":"2024-09-13T09:14:54.376250+0000","last_became_active":"2024-09-13T09:14:48.349512+0000","last_became_peered":"2024-09-13T09:14:48.349512+0000","last_unstale":"2024-09-13T09:14:54.376250+0000","last_undegraded":"2024-09-13T09:14:54.376250+0000","last_fullsized":"2024-09-13T09:14:54.376250+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:17.628884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016583220000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8119,"num_read_kb":73640,"num_write":604,"num_write_kb":21508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"147'866","reported_seq":13468,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688927+0000","last_change":"2024-09-13T09:14:44.477496+0000","last_active":"2024-09-13T09:14:54.688927+0000","last_peered":"2024-09-13T09:14:54.688927+0000","last_clean":"2024-09-13T09:14:54.688927+0000","last_became_active":"2024-09-13T09:14:44.313818+0000","last_became_peered":"2024-09-13T09:14:44.313818+0000","last_unstale":"2024-09-13T09:14:54.688927+0000","last_undegraded":"2024-09-13T09:14:54.688927+0000","last_fullsized":"2024-09-13T09:14:54.688927+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:00.720161+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.022344567999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":15285,"num_read_kb":90031,"num_write":1043,"num_write_kb":32713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"135'363","reported_seq":3876,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189530+0000","last_change":"2024-09-13T09:14:35.255447+0000","last_active":"2024-09-13T09:14:55.189530+0000","last_peered":"2024-09-13T09:14:55.189530+0000","last_clean":"2024-09-13T09:14:55.189530+0000","last_became_active":"2024-09-13T09:14:16.751511+0000","last_became_peered":"2024-09-13T09:14:16.751511+0000","last_unstale":"2024-09-13T09:14:55.189530+0000","last_undegraded":"2024-09-13T09:14:55.189530+0000","last_fullsized":"2024-09-13T09:14:55.189530+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:57.453044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011111789999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3406,"num_read_kb":19855,"num_write":419,"num_write_kb":12151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":4294187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"147'381","reported_seq":5925,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688949+0000","last_change":"2024-09-13T09:14:44.516389+0000","last_active":"2024-09-13T09:14:54.688949+0000","last_peered":"2024-09-13T09:14:54.688949+0000","last_clean":"2024-09-13T09:14:54.688949+0000","last_became_active":"2024-09-13T09:14:44.315188+0000","last_became_peered":"2024-09-13T09:14:44.315188+0000","last_unstale":"2024-09-13T09:14:54.688949+0000","last_undegraded":"2024-09-13T09:14:54.688949+0000","last_fullsized":"2024-09-13T09:14:54.688949+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:10.043789+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0058820570000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6135,"num_read_kb":25216,"num_write":405,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"141'532","reported_seq":7039,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689064+0000","last_change":"2024-09-13T09:14:44.426902+0000","last_active":"2024-09-13T09:14:54.689064+0000","last_peered":"2024-09-13T09:14:54.689064+0000","last_clean":"2024-09-13T09:14:54.689064+0000","last_became_active":"2024-09-13T09:14:44.309465+0000","last_became_peered":"2024-09-13T09:14:44.309465+0000","last_unstale":"2024-09-13T09:14:54.689064+0000","last_undegraded":"2024-09-13T09:14:54.689064+0000","last_fullsized":"2024-09-13T09:14:54.689064+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:19:43.291662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039480829999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6633,"num_read_kb":20805,"num_write":596,"num_write_kb":7043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":635004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"147'533","reported_seq":7949,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688828+0000","last_change":"2024-09-13T09:14:44.375157+0000","last_active":"2024-09-13T09:14:54.688828+0000","last_peered":"2024-09-13T09:14:54.688828+0000","last_clean":"2024-09-13T09:14:54.688828+0000","last_became_active":"2024-09-13T09:14:44.308779+0000","last_became_peered":"2024-09-13T09:14:44.308779+0000","last_unstale":"2024-09-13T09:14:54.688828+0000","last_undegraded":"2024-09-13T09:14:54.688828+0000","last_fullsized":"2024-09-13T09:14:54.688828+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:00.643049+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0066958310000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7617,"num_read_kb":37352,"num_write":622,"num_write_kb":17983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"139'545","reported_seq":4837,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189301+0000","last_change":"2024-09-13T09:14:35.256957+0000","last_active":"2024-09-13T09:14:55.189301+0000","last_peered":"2024-09-13T09:14:55.189301+0000","last_clean":"2024-09-13T09:14:55.189301+0000","last_became_active":"2024-09-13T09:12:43.532504+0000","last_became_peered":"2024-09-13T09:12:43.532504+0000","last_unstale":"2024-09-13T09:14:55.189301+0000","last_undegraded":"2024-09-13T09:14:55.189301+0000","last_fullsized":"2024-09-13T09:14:55.189301+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:25:04.225563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021463989999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5509,"num_read_kb":32081,"num_write":673,"num_write_kb":7448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":694311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"141'1115","reported_seq":20789,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.374845+0000","last_change":"2024-09-13T09:14:35.253524+0000","last_active":"2024-09-13T09:14:54.374845+0000","last_peered":"2024-09-13T09:14:54.374845+0000","last_clean":"2024-09-13T09:14:54.374845+0000","last_became_active":"2024-09-13T09:12:46.248261+0000","last_became_peered":"2024-09-13T09:12:46.248261+0000","last_unstale":"2024-09-13T09:14:54.374845+0000","last_undegraded":"2024-09-13T09:14:54.374845+0000","last_fullsized":"2024-09-13T09:14:54.374845+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:44.056762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092792899999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":26946,"num_read_kb":67879,"num_write":1035,"num_write_kb":10041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":676864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"141'603","reported_seq":6565,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189413+0000","last_change":"2024-09-13T09:14:44.326177+0000","last_active":"2024-09-13T09:14:55.189413+0000","last_peered":"2024-09-13T09:14:55.189413+0000","last_clean":"2024-09-13T09:14:55.189413+0000","last_became_active":"2024-09-13T09:14:44.314564+0000","last_became_peered":"2024-09-13T09:14:44.314564+0000","last_unstale":"2024-09-13T09:14:55.189413+0000","last_undegraded":"2024-09-13T09:14:55.189413+0000","last_fullsized":"2024-09-13T09:14:55.189413+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:58:58.520651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050931029999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8526,"num_read_kb":13102,"num_write":519,"num_write_kb":3941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":28777,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":68,"num_read":356751,"num_read_kb":1364316,"num_write":26574,"num_write_kb":359719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":18167194,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28441,"internal_metadata":0},"log_size":22993,"ondisk_log_size":22993,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":146,"seq":627065225221,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31180,"kb_used_data":1848,"kb_used_omap":31,"kb_used_meta":29280,"kb_avail":94340660,"statfs":{"total":96636764160,"available":96604835840,"internally_reserved":0,"allocated":1892352,"data_stored":790048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32283,"internal_metadata":29983205},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640297,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64952,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306888,"statfs":{"total":96636764160,"available":96570253312,"internally_reserved":0,"allocated":1880064,"data_stored":785020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39700000000000002},"min":{"1min":0.28199999999999997,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.52700000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.40200000000000002,"15min":0.39100000000000001},"min":{"1min":0.26800000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.31,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41799999999999998,"15min":0.41999999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56799999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.48399999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.438},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53400000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.38200000000000001,"15min":0.38900000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.52400000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.23799999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54400000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.375},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40000000000000002,"15min":0.40799999999999997},"min":{"1min":0.25,"5min":0.249,"15min":0.249},"max":{"1min":0.55400000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46999999999999997}]}]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59784,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312056,"statfs":{"total":96636764160,"available":96575545344,"internally_reserved":0,"allocated":2473984,"data_stored":1378241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.40600000000000003},"min":{"1min":0.21099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.47499999999999998,"15min":0.44500000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.54500000000000004,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.5990000000000002,"5min":4.5990000000000002,"15min":4.5990000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.40100000000000002,"15min":0.377},"min":{"1min":0.23799999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":5.165,"5min":5.165,"15min":5.165},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38300000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.45000000000000001,"15min":0.40300000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":8.7959999999999994,"5min":8.7959999999999994,"15min":8.7959999999999994},"last":0.50600000000000001}]}]},{"osd":2,"up_from":122,"seq":523986010122,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35080,"kb_used_data":1844,"kb_used_omap":37,"kb_used_meta":33178,"kb_avail":94336760,"statfs":{"total":96636764160,"available":96600842240,"internally_reserved":0,"allocated":1888256,"data_stored":790029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38061,"internal_metadata":33975123},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":1,"up_from":11,"seq":47244640297,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55112,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316728,"statfs":{"total":96636764160,"available":96580329472,"internally_reserved":0,"allocated":1896448,"data_stored":790067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.32700000000000001,"15min":0.33700000000000002},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.54700000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.32100000000000001,"15min":0.32500000000000001},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.56000000000000005,"5min":0.753,"15min":0.753},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.379},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.433}]},{"osd":3,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.496,"15min":0.50700000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.60099999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.58199999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.46600000000000003,"15min":0.47399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.622,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.68300000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.496,"15min":0.504},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.747},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.47099999999999997,"15min":0.48399999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.66900000000000004}]}]},{"osd":0,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63880,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307960,"statfs":{"total":96636764160,"available":96571351040,"internally_reserved":0,"allocated":2482176,"data_stored":1380397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.26600000000000001,"5min":0.27900000000000003,"15min":0.28100000000000003},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.28000000000000003,"15min":0.28199999999999997},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":0.54800000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.84399999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.44800000000000001,"15min":0.45900000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.69899999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.442,"15min":0.44500000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.745}]},{"osd":4,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45000000000000001,"15min":0.45600000000000002},"min":{"1min":0.26300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.874,"5min":0.879,"15min":0.879},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.45000000000000001,"15min":0.45700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68500000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.88}]},{"osd":5,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46700000000000003,"15min":0.47599999999999998},"min":{"1min":0.29199999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88700000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.45500000000000002,"15min":0.45800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69799999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.77600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16703,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11738,"internal_metadata":0}]}} 2024-09-13T09:14:59.895 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:15:00.177 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:15:00.178 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:15:00.188 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-13T09:14:59.549222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":68,"num_read":356867,"num_read_kb":1364416,"num_write":26735,"num_write_kb":362137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":666,"num_bytes_recovered":19347930,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23113,"ondisk_log_size":23113,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":309988,"kb_used_data":12220,"kb_used_omap":74,"kb_used_meta":297589,"kb_avail":565921052,"statfs":{"total":579820584960,"available":579503157248,"internally_reserved":0,"allocated":12513280,"data_stored":5913802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76719,"internal_metadata":304731217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.181465"},"pg_stats":[{"pgid":"3.1f","version":"142'857","reported_seq":9653,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375199+0000","last_change":"2024-09-13T09:14:35.254388+0000","last_active":"2024-09-13T09:14:54.375199+0000","last_peered":"2024-09-13T09:14:54.375199+0000","last_clean":"2024-09-13T09:14:54.375199+0000","last_became_active":"2024-09-13T09:12:43.534990+0000","last_became_peered":"2024-09-13T09:12:43.534990+0000","last_unstale":"2024-09-13T09:14:54.375199+0000","last_undegraded":"2024-09-13T09:14:54.375199+0000","last_fullsized":"2024-09-13T09:14:54.375199+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:45.810610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093900099999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":11993,"num_read_kb":21637,"num_write":921,"num_write_kb":7095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":848912,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"140'605","reported_seq":7893,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375888+0000","last_change":"2024-09-13T09:14:35.256646+0000","last_active":"2024-09-13T09:14:54.375888+0000","last_peered":"2024-09-13T09:14:54.375888+0000","last_clean":"2024-09-13T09:14:54.375888+0000","last_became_active":"2024-09-13T09:14:16.749234+0000","last_became_peered":"2024-09-13T09:14:16.749234+0000","last_unstale":"2024-09-13T09:14:54.375888+0000","last_undegraded":"2024-09-13T09:14:54.375888+0000","last_fullsized":"2024-09-13T09:14:54.375888+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:18:42.349049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020237760000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10031,"num_read_kb":30858,"num_write":737,"num_write_kb":10943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":313856,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"147'1435","reported_seq":16444,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688945+0000","last_change":"2024-09-13T09:14:44.400159+0000","last_active":"2024-09-13T09:14:54.688945+0000","last_peered":"2024-09-13T09:14:54.688945+0000","last_clean":"2024-09-13T09:14:54.688945+0000","last_became_active":"2024-09-13T09:14:44.312734+0000","last_became_peered":"2024-09-13T09:14:44.312734+0000","last_unstale":"2024-09-13T09:14:54.688945+0000","last_undegraded":"2024-09-13T09:14:54.688945+0000","last_fullsized":"2024-09-13T09:14:54.688945+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1435,"log_dups_size":0,"ondisk_log_size":1435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.934438+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011136765999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":20060,"num_read_kb":34855,"num_write":1554,"num_write_kb":7371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"147'1480","reported_seq":8609,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689014+0000","last_change":"2024-09-13T09:14:44.511014+0000","last_active":"2024-09-13T09:14:54.689014+0000","last_peered":"2024-09-13T09:14:54.689014+0000","last_clean":"2024-09-13T09:14:54.689014+0000","last_became_active":"2024-09-13T09:14:44.311550+0000","last_became_peered":"2024-09-13T09:14:44.311550+0000","last_unstale":"2024-09-13T09:14:54.689014+0000","last_undegraded":"2024-09-13T09:14:54.689014+0000","last_fullsized":"2024-09-13T09:14:54.689014+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1480,"log_dups_size":0,"ondisk_log_size":1480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:16:38.500976+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.022244201000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13526,"num_read_kb":55870,"num_write":2225,"num_write_kb":20106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"142'635","reported_seq":9508,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375955+0000","last_change":"2024-09-13T09:14:35.256035+0000","last_active":"2024-09-13T09:14:54.375955+0000","last_peered":"2024-09-13T09:14:54.375955+0000","last_clean":"2024-09-13T09:14:54.375955+0000","last_became_active":"2024-09-13T09:14:19.772731+0000","last_became_peered":"2024-09-13T09:14:19.772731+0000","last_unstale":"2024-09-13T09:14:54.375955+0000","last_undegraded":"2024-09-13T09:14:54.375955+0000","last_fullsized":"2024-09-13T09:14:54.375955+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:24:40.563709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001509144,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10475,"num_read_kb":52366,"num_write":680,"num_write_kb":12577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1411144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"141'628","reported_seq":14444,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189254+0000","last_change":"2024-09-13T09:14:35.259843+0000","last_active":"2024-09-13T09:14:55.189254+0000","last_peered":"2024-09-13T09:14:55.189254+0000","last_clean":"2024-09-13T09:14:55.189254+0000","last_became_active":"2024-09-13T09:12:13.362533+0000","last_became_peered":"2024-09-13T09:12:13.362533+0000","last_unstale":"2024-09-13T09:14:55.189254+0000","last_undegraded":"2024-09-13T09:14:55.189254+0000","last_fullsized":"2024-09-13T09:14:55.189254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:50.399315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017407639999999999,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":16059,"num_read_kb":55558,"num_write":617,"num_write_kb":11980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"138'567","reported_seq":8658,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376230+0000","last_change":"2024-09-13T09:14:48.352186+0000","last_active":"2024-09-13T09:14:54.376230+0000","last_peered":"2024-09-13T09:14:54.376230+0000","last_clean":"2024-09-13T09:14:54.376230+0000","last_became_active":"2024-09-13T09:14:48.350477+0000","last_became_peered":"2024-09-13T09:14:48.350477+0000","last_unstale":"2024-09-13T09:14:54.376230+0000","last_undegraded":"2024-09-13T09:14:54.376230+0000","last_fullsized":"2024-09-13T09:14:54.376230+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:57.759426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074163899999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8163,"num_read_kb":27760,"num_write":669,"num_write_kb":11976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"139'576","reported_seq":11071,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375944+0000","last_change":"2024-09-13T09:14:35.255587+0000","last_active":"2024-09-13T09:14:54.375944+0000","last_peered":"2024-09-13T09:14:54.375944+0000","last_clean":"2024-09-13T09:14:54.375944+0000","last_became_active":"2024-09-13T09:14:19.772799+0000","last_became_peered":"2024-09-13T09:14:19.772799+0000","last_unstale":"2024-09-13T09:14:54.375944+0000","last_undegraded":"2024-09-13T09:14:54.375944+0000","last_fullsized":"2024-09-13T09:14:54.375944+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:41:21.622976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001235604,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13056,"num_read_kb":24986,"num_write":439,"num_write_kb":7320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":104960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"147'434","reported_seq":5711,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375290+0000","last_change":"2024-09-13T09:14:44.358128+0000","last_active":"2024-09-13T09:14:54.375290+0000","last_peered":"2024-09-13T09:14:54.375290+0000","last_clean":"2024-09-13T09:14:54.375290+0000","last_became_active":"2024-09-13T09:14:44.317071+0000","last_became_peered":"2024-09-13T09:14:44.317071+0000","last_unstale":"2024-09-13T09:14:54.375290+0000","last_undegraded":"2024-09-13T09:14:54.375290+0000","last_fullsized":"2024-09-13T09:14:54.375290+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:42.567052+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015690497000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6024,"num_read_kb":31779,"num_write":503,"num_write_kb":7984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"147'972","reported_seq":6743,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689023+0000","last_change":"2024-09-13T09:14:44.415816+0000","last_active":"2024-09-13T09:14:54.689023+0000","last_peered":"2024-09-13T09:14:54.689023+0000","last_clean":"2024-09-13T09:14:54.689023+0000","last_became_active":"2024-09-13T09:14:44.309746+0000","last_became_peered":"2024-09-13T09:14:44.309746+0000","last_unstale":"2024-09-13T09:14:54.689023+0000","last_undegraded":"2024-09-13T09:14:54.689023+0000","last_fullsized":"2024-09-13T09:14:54.689023+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":972,"log_dups_size":0,"ondisk_log_size":972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:16:58.217708+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0084025269999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8520,"num_read_kb":43161,"num_write":1440,"num_write_kb":11751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":706121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"139'872","reported_seq":20338,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189647+0000","last_change":"2024-09-13T09:14:35.256574+0000","last_active":"2024-09-13T09:14:55.189647+0000","last_peered":"2024-09-13T09:14:55.189647+0000","last_clean":"2024-09-13T09:14:55.189647+0000","last_became_active":"2024-09-13T09:12:46.250696+0000","last_became_peered":"2024-09-13T09:12:46.250696+0000","last_unstale":"2024-09-13T09:14:55.189647+0000","last_undegraded":"2024-09-13T09:14:55.189647+0000","last_fullsized":"2024-09-13T09:14:55.189647+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:43.821997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019682329999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":21840,"num_read_kb":55389,"num_write":1298,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1136450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"140'665","reported_seq":8589,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189530+0000","last_change":"2024-09-13T09:14:35.255745+0000","last_active":"2024-09-13T09:14:55.189530+0000","last_peered":"2024-09-13T09:14:55.189530+0000","last_clean":"2024-09-13T09:14:55.189530+0000","last_became_active":"2024-09-13T09:12:46.254016+0000","last_became_peered":"2024-09-13T09:12:46.254016+0000","last_unstale":"2024-09-13T09:14:55.189530+0000","last_undegraded":"2024-09-13T09:14:55.189530+0000","last_fullsized":"2024-09-13T09:14:55.189530+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:50.193567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013411219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10880,"num_read_kb":53346,"num_write":603,"num_write_kb":9445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1110608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688996+0000","last_change":"2024-09-13T09:14:44.316371+0000","last_active":"2024-09-13T09:14:54.688996+0000","last_peered":"2024-09-13T09:14:54.688996+0000","last_clean":"2024-09-13T09:14:54.688996+0000","last_became_active":"2024-09-13T09:14:44.316062+0000","last_became_peered":"2024-09-13T09:14:44.316062+0000","last_unstale":"2024-09-13T09:14:54.688996+0000","last_undegraded":"2024-09-13T09:14:54.688996+0000","last_fullsized":"2024-09-13T09:14:54.688996+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:06:27.687674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"142'753","reported_seq":10733,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375022+0000","last_change":"2024-09-13T09:14:35.253996+0000","last_active":"2024-09-13T09:14:54.375022+0000","last_peered":"2024-09-13T09:14:54.375022+0000","last_clean":"2024-09-13T09:14:54.375022+0000","last_became_active":"2024-09-13T09:12:13.365496+0000","last_became_peered":"2024-09-13T09:12:13.365496+0000","last_unstale":"2024-09-13T09:14:54.375022+0000","last_undegraded":"2024-09-13T09:14:54.375022+0000","last_fullsized":"2024-09-13T09:14:54.375022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:29:49.115977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063501900000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11213,"num_read_kb":65815,"num_write":999,"num_write_kb":12170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":156,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376437+0000","last_change":"2024-09-13T09:14:48.352849+0000","last_active":"2024-09-13T09:14:54.376437+0000","last_peered":"2024-09-13T09:14:54.376437+0000","last_clean":"2024-09-13T09:14:54.376437+0000","last_became_active":"2024-09-13T09:14:48.352502+0000","last_became_peered":"2024-09-13T09:14:48.352502+0000","last_unstale":"2024-09-13T09:14:54.376437+0000","last_undegraded":"2024-09-13T09:14:54.376437+0000","last_fullsized":"2024-09-13T09:14:54.376437+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:04.532498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"141'1047","reported_seq":10558,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375958+0000","last_change":"2024-09-13T09:14:35.256434+0000","last_active":"2024-09-13T09:14:54.375958+0000","last_peered":"2024-09-13T09:14:54.375958+0000","last_clean":"2024-09-13T09:14:54.375958+0000","last_became_active":"2024-09-13T09:14:16.752774+0000","last_became_peered":"2024-09-13T09:14:16.752774+0000","last_unstale":"2024-09-13T09:14:54.375958+0000","last_undegraded":"2024-09-13T09:14:54.375958+0000","last_fullsized":"2024-09-13T09:14:54.375958+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1047,"log_dups_size":0,"ondisk_log_size":1047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:37:58.086486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001863822,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":16931,"num_read_kb":50979,"num_write":1279,"num_write_kb":12713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":413184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":121,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189554+0000","last_change":"2024-09-13T09:12:43.533829+0000","last_active":"2024-09-13T09:14:55.189554+0000","last_peered":"2024-09-13T09:14:55.189554+0000","last_clean":"2024-09-13T09:14:55.189554+0000","last_became_active":"2024-09-13T09:12:43.533557+0000","last_became_peered":"2024-09-13T09:12:43.533557+0000","last_unstale":"2024-09-13T09:14:55.189554+0000","last_undegraded":"2024-09-13T09:14:55.189554+0000","last_fullsized":"2024-09-13T09:14:55.189554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:51.939164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"141'707","reported_seq":13355,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376484+0000","last_change":"2024-09-13T09:14:48.351345+0000","last_active":"2024-09-13T09:14:54.376484+0000","last_peered":"2024-09-13T09:14:54.376484+0000","last_clean":"2024-09-13T09:14:54.376484+0000","last_became_active":"2024-09-13T09:14:48.348909+0000","last_became_peered":"2024-09-13T09:14:48.348909+0000","last_unstale":"2024-09-13T09:14:54.376484+0000","last_undegraded":"2024-09-13T09:14:54.376484+0000","last_fullsized":"2024-09-13T09:14:54.376484+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:55:37.482616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074057,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":15357,"num_read_kb":39900,"num_write":798,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":29184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376531+0000","last_change":"2024-09-13T09:14:48.351236+0000","last_active":"2024-09-13T09:14:54.376531+0000","last_peered":"2024-09-13T09:14:54.376531+0000","last_clean":"2024-09-13T09:14:54.376531+0000","last_became_active":"2024-09-13T09:14:48.348560+0000","last_became_peered":"2024-09-13T09:14:48.348560+0000","last_unstale":"2024-09-13T09:14:54.376531+0000","last_undegraded":"2024-09-13T09:14:54.376531+0000","last_fullsized":"2024-09-13T09:14:54.376531+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:21.050148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"142'748","reported_seq":7661,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688947+0000","last_change":"2024-09-13T09:14:44.367184+0000","last_active":"2024-09-13T09:14:54.688947+0000","last_peered":"2024-09-13T09:14:54.688947+0000","last_clean":"2024-09-13T09:14:54.688947+0000","last_became_active":"2024-09-13T09:14:44.309435+0000","last_became_peered":"2024-09-13T09:14:44.309435+0000","last_unstale":"2024-09-13T09:14:54.688947+0000","last_undegraded":"2024-09-13T09:14:54.688947+0000","last_fullsized":"2024-09-13T09:14:54.688947+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:57.739078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039806149999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10405,"num_read_kb":27276,"num_write":715,"num_write_kb":2474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":182784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"142'634","reported_seq":11576,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189327+0000","last_change":"2024-09-13T09:14:35.258893+0000","last_active":"2024-09-13T09:14:55.189327+0000","last_peered":"2024-09-13T09:14:55.189327+0000","last_clean":"2024-09-13T09:14:55.189327+0000","last_became_active":"2024-09-13T09:12:13.367770+0000","last_became_peered":"2024-09-13T09:12:13.367770+0000","last_unstale":"2024-09-13T09:14:55.189327+0000","last_undegraded":"2024-09-13T09:14:55.189327+0000","last_fullsized":"2024-09-13T09:14:55.189327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:21:50.690425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001191176,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10993,"num_read_kb":67376,"num_write":601,"num_write_kb":14012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":122,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688871+0000","last_change":"2024-09-13T09:14:44.312125+0000","last_active":"2024-09-13T09:14:54.688871+0000","last_peered":"2024-09-13T09:14:54.688871+0000","last_clean":"2024-09-13T09:14:54.688871+0000","last_became_active":"2024-09-13T09:14:44.311178+0000","last_became_peered":"2024-09-13T09:14:44.311178+0000","last_unstale":"2024-09-13T09:14:54.688871+0000","last_undegraded":"2024-09-13T09:14:54.688871+0000","last_fullsized":"2024-09-13T09:14:54.688871+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:46.245017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"147'620","reported_seq":9651,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688842+0000","last_change":"2024-09-13T09:14:44.440284+0000","last_active":"2024-09-13T09:14:54.688842+0000","last_peered":"2024-09-13T09:14:54.688842+0000","last_clean":"2024-09-13T09:14:54.688842+0000","last_became_active":"2024-09-13T09:14:44.310016+0000","last_became_peered":"2024-09-13T09:14:44.310016+0000","last_unstale":"2024-09-13T09:14:54.688842+0000","last_undegraded":"2024-09-13T09:14:54.688842+0000","last_fullsized":"2024-09-13T09:14:54.688842+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:55.802300+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.007574824,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":9521,"num_read_kb":48768,"num_write":764,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1695776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376024+0000","last_change":"2024-09-13T09:14:16.749189+0000","last_active":"2024-09-13T09:14:54.376024+0000","last_peered":"2024-09-13T09:14:54.376024+0000","last_clean":"2024-09-13T09:14:54.376024+0000","last_became_active":"2024-09-13T09:14:16.748865+0000","last_became_peered":"2024-09-13T09:14:16.748865+0000","last_unstale":"2024-09-13T09:14:54.376024+0000","last_undegraded":"2024-09-13T09:14:54.376024+0000","last_fullsized":"2024-09-13T09:14:54.376024+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:25:39.217684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"142'332","reported_seq":3860,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.374904+0000","last_change":"2024-09-13T09:14:35.253968+0000","last_active":"2024-09-13T09:14:54.374904+0000","last_peered":"2024-09-13T09:14:54.374904+0000","last_clean":"2024-09-13T09:14:54.374904+0000","last_became_active":"2024-09-13T09:12:13.367633+0000","last_became_peered":"2024-09-13T09:12:13.367633+0000","last_unstale":"2024-09-13T09:14:54.374904+0000","last_undegraded":"2024-09-13T09:14:54.374904+0000","last_fullsized":"2024-09-13T09:14:54.374904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:34.734632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094036499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3280,"num_read_kb":26753,"num_write":312,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189164+0000","last_change":"2024-09-13T09:12:43.532201+0000","last_active":"2024-09-13T09:14:55.189164+0000","last_peered":"2024-09-13T09:14:55.189164+0000","last_clean":"2024-09-13T09:14:55.189164+0000","last_became_active":"2024-09-13T09:12:43.531942+0000","last_became_peered":"2024-09-13T09:12:43.531942+0000","last_unstale":"2024-09-13T09:14:55.189164+0000","last_undegraded":"2024-09-13T09:14:55.189164+0000","last_fullsized":"2024-09-13T09:14:55.189164+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:26.408883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"141'653","reported_seq":3816,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376444+0000","last_change":"2024-09-13T09:14:48.351027+0000","last_active":"2024-09-13T09:14:54.376444+0000","last_peered":"2024-09-13T09:14:54.376444+0000","last_clean":"2024-09-13T09:14:54.376444+0000","last_became_active":"2024-09-13T09:14:48.349138+0000","last_became_peered":"2024-09-13T09:14:48.349138+0000","last_unstale":"2024-09-13T09:14:54.376444+0000","last_undegraded":"2024-09-13T09:14:54.376444+0000","last_fullsized":"2024-09-13T09:14:54.376444+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:46.295942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039583799999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4158,"num_read_kb":20449,"num_write":848,"num_write_kb":5187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":155,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688806+0000","last_change":"2024-09-13T09:14:44.309742+0000","last_active":"2024-09-13T09:14:54.688806+0000","last_peered":"2024-09-13T09:14:54.688806+0000","last_clean":"2024-09-13T09:14:54.688806+0000","last_became_active":"2024-09-13T09:14:44.309293+0000","last_became_peered":"2024-09-13T09:14:44.309293+0000","last_unstale":"2024-09-13T09:14:54.688806+0000","last_undegraded":"2024-09-13T09:14:54.688806+0000","last_fullsized":"2024-09-13T09:14:54.688806+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:28:57.831071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":324,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189255+0000","last_change":"2024-09-13T09:12:46.248714+0000","last_active":"2024-09-13T09:14:55.189255+0000","last_peered":"2024-09-13T09:14:55.189255+0000","last_clean":"2024-09-13T09:14:55.189255+0000","last_became_active":"2024-09-13T09:12:46.248451+0000","last_became_peered":"2024-09-13T09:12:46.248451+0000","last_unstale":"2024-09-13T09:14:55.189255+0000","last_undegraded":"2024-09-13T09:14:55.189255+0000","last_fullsized":"2024-09-13T09:14:55.189255+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:27.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"141'498","reported_seq":4698,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376040+0000","last_change":"2024-09-13T09:14:35.257020+0000","last_active":"2024-09-13T09:14:54.376040+0000","last_peered":"2024-09-13T09:14:54.376040+0000","last_clean":"2024-09-13T09:14:54.376040+0000","last_became_active":"2024-09-13T09:14:16.754685+0000","last_became_peered":"2024-09-13T09:14:16.754685+0000","last_unstale":"2024-09-13T09:14:54.376040+0000","last_undegraded":"2024-09-13T09:14:54.376040+0000","last_fullsized":"2024-09-13T09:14:54.376040+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:04.966596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022974409999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4733,"num_read_kb":22943,"num_write":604,"num_write_kb":15013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":45056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"142'1216","reported_seq":13783,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189219+0000","last_change":"2024-09-13T09:14:35.259629+0000","last_active":"2024-09-13T09:14:55.189219+0000","last_peered":"2024-09-13T09:14:55.189219+0000","last_clean":"2024-09-13T09:14:55.189219+0000","last_became_active":"2024-09-13T09:14:19.774300+0000","last_became_peered":"2024-09-13T09:14:19.774300+0000","last_unstale":"2024-09-13T09:14:55.189219+0000","last_undegraded":"2024-09-13T09:14:55.189219+0000","last_fullsized":"2024-09-13T09:14:55.189219+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1216,"log_dups_size":0,"ondisk_log_size":1216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:36:43.693732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001648159,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18678,"num_read_kb":75566,"num_write":1537,"num_write_kb":13034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1249280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"141'596","reported_seq":11902,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.381858+0000","last_change":"2024-09-13T09:14:35.256213+0000","last_active":"2024-09-13T09:14:55.381858+0000","last_peered":"2024-09-13T09:14:55.381858+0000","last_clean":"2024-09-13T09:14:55.381858+0000","last_became_active":"2024-09-13T09:12:46.252578+0000","last_became_peered":"2024-09-13T09:12:46.252578+0000","last_unstale":"2024-09-13T09:14:55.381858+0000","last_undegraded":"2024-09-13T09:14:55.381858+0000","last_fullsized":"2024-09-13T09:14:55.381858+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:03.931492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001634775,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":12679,"num_read_kb":50965,"num_write":515,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2590736,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"141'525","reported_seq":8198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376250+0000","last_change":"2024-09-13T09:14:48.351125+0000","last_active":"2024-09-13T09:14:54.376250+0000","last_peered":"2024-09-13T09:14:54.376250+0000","last_clean":"2024-09-13T09:14:54.376250+0000","last_became_active":"2024-09-13T09:14:48.349512+0000","last_became_peered":"2024-09-13T09:14:48.349512+0000","last_unstale":"2024-09-13T09:14:54.376250+0000","last_undegraded":"2024-09-13T09:14:54.376250+0000","last_fullsized":"2024-09-13T09:14:54.376250+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:17.628884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016583220000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8119,"num_read_kb":73640,"num_write":604,"num_write_kb":21508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"147'866","reported_seq":13468,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688927+0000","last_change":"2024-09-13T09:14:44.477496+0000","last_active":"2024-09-13T09:14:54.688927+0000","last_peered":"2024-09-13T09:14:54.688927+0000","last_clean":"2024-09-13T09:14:54.688927+0000","last_became_active":"2024-09-13T09:14:44.313818+0000","last_became_peered":"2024-09-13T09:14:44.313818+0000","last_unstale":"2024-09-13T09:14:54.688927+0000","last_undegraded":"2024-09-13T09:14:54.688927+0000","last_fullsized":"2024-09-13T09:14:54.688927+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:00.720161+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.022344567999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":15285,"num_read_kb":90031,"num_write":1043,"num_write_kb":32713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"135'363","reported_seq":3876,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189530+0000","last_change":"2024-09-13T09:14:35.255447+0000","last_active":"2024-09-13T09:14:55.189530+0000","last_peered":"2024-09-13T09:14:55.189530+0000","last_clean":"2024-09-13T09:14:55.189530+0000","last_became_active":"2024-09-13T09:14:16.751511+0000","last_became_peered":"2024-09-13T09:14:16.751511+0000","last_unstale":"2024-09-13T09:14:55.189530+0000","last_undegraded":"2024-09-13T09:14:55.189530+0000","last_fullsized":"2024-09-13T09:14:55.189530+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:57.453044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011111789999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3406,"num_read_kb":19855,"num_write":419,"num_write_kb":12151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":4294187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"147'381","reported_seq":5925,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688949+0000","last_change":"2024-09-13T09:14:44.516389+0000","last_active":"2024-09-13T09:14:54.688949+0000","last_peered":"2024-09-13T09:14:54.688949+0000","last_clean":"2024-09-13T09:14:54.688949+0000","last_became_active":"2024-09-13T09:14:44.315188+0000","last_became_peered":"2024-09-13T09:14:44.315188+0000","last_unstale":"2024-09-13T09:14:54.688949+0000","last_undegraded":"2024-09-13T09:14:54.688949+0000","last_fullsized":"2024-09-13T09:14:54.688949+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:10.043789+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0058820570000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6135,"num_read_kb":25216,"num_write":405,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"141'532","reported_seq":7039,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689064+0000","last_change":"2024-09-13T09:14:44.426902+0000","last_active":"2024-09-13T09:14:54.689064+0000","last_peered":"2024-09-13T09:14:54.689064+0000","last_clean":"2024-09-13T09:14:54.689064+0000","last_became_active":"2024-09-13T09:14:44.309465+0000","last_became_peered":"2024-09-13T09:14:44.309465+0000","last_unstale":"2024-09-13T09:14:54.689064+0000","last_undegraded":"2024-09-13T09:14:54.689064+0000","last_fullsized":"2024-09-13T09:14:54.689064+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:19:43.291662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039480829999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6633,"num_read_kb":20805,"num_write":596,"num_write_kb":7043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":635004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"147'533","reported_seq":7949,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688828+0000","last_change":"2024-09-13T09:14:44.375157+0000","last_active":"2024-09-13T09:14:54.688828+0000","last_peered":"2024-09-13T09:14:54.688828+0000","last_clean":"2024-09-13T09:14:54.688828+0000","last_became_active":"2024-09-13T09:14:44.308779+0000","last_became_peered":"2024-09-13T09:14:44.308779+0000","last_unstale":"2024-09-13T09:14:54.688828+0000","last_undegraded":"2024-09-13T09:14:54.688828+0000","last_fullsized":"2024-09-13T09:14:54.688828+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:00.643049+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0066958310000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7617,"num_read_kb":37352,"num_write":622,"num_write_kb":17983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"139'545","reported_seq":4837,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189301+0000","last_change":"2024-09-13T09:14:35.256957+0000","last_active":"2024-09-13T09:14:55.189301+0000","last_peered":"2024-09-13T09:14:55.189301+0000","last_clean":"2024-09-13T09:14:55.189301+0000","last_became_active":"2024-09-13T09:12:43.532504+0000","last_became_peered":"2024-09-13T09:12:43.532504+0000","last_unstale":"2024-09-13T09:14:55.189301+0000","last_undegraded":"2024-09-13T09:14:55.189301+0000","last_fullsized":"2024-09-13T09:14:55.189301+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:25:04.225563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021463989999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5509,"num_read_kb":32081,"num_write":673,"num_write_kb":7448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":694311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"141'1115","reported_seq":20789,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.374845+0000","last_change":"2024-09-13T09:14:35.253524+0000","last_active":"2024-09-13T09:14:54.374845+0000","last_peered":"2024-09-13T09:14:54.374845+0000","last_clean":"2024-09-13T09:14:54.374845+0000","last_became_active":"2024-09-13T09:12:46.248261+0000","last_became_peered":"2024-09-13T09:12:46.248261+0000","last_unstale":"2024-09-13T09:14:54.374845+0000","last_undegraded":"2024-09-13T09:14:54.374845+0000","last_fullsized":"2024-09-13T09:14:54.374845+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:44.056762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092792899999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":26946,"num_read_kb":67879,"num_write":1035,"num_write_kb":10041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":676864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"141'603","reported_seq":6565,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189413+0000","last_change":"2024-09-13T09:14:44.326177+0000","last_active":"2024-09-13T09:14:55.189413+0000","last_peered":"2024-09-13T09:14:55.189413+0000","last_clean":"2024-09-13T09:14:55.189413+0000","last_became_active":"2024-09-13T09:14:44.314564+0000","last_became_peered":"2024-09-13T09:14:44.314564+0000","last_unstale":"2024-09-13T09:14:55.189413+0000","last_undegraded":"2024-09-13T09:14:55.189413+0000","last_fullsized":"2024-09-13T09:14:55.189413+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:58:58.520651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050931029999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8526,"num_read_kb":13102,"num_write":519,"num_write_kb":3941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":28777,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":68,"num_read":356751,"num_read_kb":1364316,"num_write":26574,"num_write_kb":359719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":18167194,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28441,"internal_metadata":0},"log_size":22993,"ondisk_log_size":22993,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":146,"seq":627065225221,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31180,"kb_used_data":1848,"kb_used_omap":31,"kb_used_meta":29280,"kb_avail":94340660,"statfs":{"total":96636764160,"available":96604835840,"internally_reserved":0,"allocated":1892352,"data_stored":790048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32283,"internal_metadata":29983205},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640297,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64952,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306888,"statfs":{"total":96636764160,"available":96570253312,"internally_reserved":0,"allocated":1880064,"data_stored":785020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39700000000000002},"min":{"1min":0.28199999999999997,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.52700000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.40200000000000002,"15min":0.39100000000000001},"min":{"1min":0.26800000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.31,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41799999999999998,"15min":0.41999999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56799999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.48399999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.438},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53400000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.38200000000000001,"15min":0.38900000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.52400000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.23799999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54400000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.375},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40000000000000002,"15min":0.40799999999999997},"min":{"1min":0.25,"5min":0.249,"15min":0.249},"max":{"1min":0.55400000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46999999999999997}]}]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59784,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312056,"statfs":{"total":96636764160,"available":96575545344,"internally_reserved":0,"allocated":2473984,"data_stored":1378241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.40600000000000003},"min":{"1min":0.21099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.47499999999999998,"15min":0.44500000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.54500000000000004,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.5990000000000002,"5min":4.5990000000000002,"15min":4.5990000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.40100000000000002,"15min":0.377},"min":{"1min":0.23799999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":5.165,"5min":5.165,"15min":5.165},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38300000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.45000000000000001,"15min":0.40300000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":8.7959999999999994,"5min":8.7959999999999994,"15min":8.7959999999999994},"last":0.50600000000000001}]}]},{"osd":2,"up_from":122,"seq":523986010122,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35080,"kb_used_data":1844,"kb_used_omap":37,"kb_used_meta":33178,"kb_avail":94336760,"statfs":{"total":96636764160,"available":96600842240,"internally_reserved":0,"allocated":1888256,"data_stored":790029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38061,"internal_metadata":33975123},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":1,"up_from":11,"seq":47244640297,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55112,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316728,"statfs":{"total":96636764160,"available":96580329472,"internally_reserved":0,"allocated":1896448,"data_stored":790067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.32700000000000001,"15min":0.33700000000000002},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.54700000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.32100000000000001,"15min":0.32500000000000001},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.56000000000000005,"5min":0.753,"15min":0.753},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.379},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.433}]},{"osd":3,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.496,"15min":0.50700000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.60099999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.58199999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.46600000000000003,"15min":0.47399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.622,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.68300000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.496,"15min":0.504},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.747},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.47099999999999997,"15min":0.48399999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.66900000000000004}]}]},{"osd":0,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63880,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307960,"statfs":{"total":96636764160,"available":96571351040,"internally_reserved":0,"allocated":2482176,"data_stored":1380397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.26600000000000001,"5min":0.27900000000000003,"15min":0.28100000000000003},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.28000000000000003,"15min":0.28199999999999997},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":0.54800000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.84399999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.44800000000000001,"15min":0.45900000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.69899999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.442,"15min":0.44500000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.745}]},{"osd":4,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45000000000000001,"15min":0.45600000000000002},"min":{"1min":0.26300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.874,"5min":0.879,"15min":0.879},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.45000000000000001,"15min":0.45700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68500000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.88}]},{"osd":5,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46700000000000003,"15min":0.47599999999999998},"min":{"1min":0.29199999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88700000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.45500000000000002,"15min":0.45800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69799999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.77600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16703,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11738,"internal_metadata":0}]}} 2024-09-13T09:15:00.192 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T09:15:00.192 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-13T09:15:00.494 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T09:15:00.508 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T09:15:00.508 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:15:00.793 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:15:00.794 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:15:00.805 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-13T09:14:59.549222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":68,"num_read":356867,"num_read_kb":1364416,"num_write":26735,"num_write_kb":362137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":666,"num_bytes_recovered":19347930,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23113,"ondisk_log_size":23113,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":309988,"kb_used_data":12220,"kb_used_omap":74,"kb_used_meta":297589,"kb_avail":565921052,"statfs":{"total":579820584960,"available":579503157248,"internally_reserved":0,"allocated":12513280,"data_stored":5913802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76719,"internal_metadata":304731217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.181465"},"pg_stats":[{"pgid":"3.1f","version":"142'857","reported_seq":9653,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375199+0000","last_change":"2024-09-13T09:14:35.254388+0000","last_active":"2024-09-13T09:14:54.375199+0000","last_peered":"2024-09-13T09:14:54.375199+0000","last_clean":"2024-09-13T09:14:54.375199+0000","last_became_active":"2024-09-13T09:12:43.534990+0000","last_became_peered":"2024-09-13T09:12:43.534990+0000","last_unstale":"2024-09-13T09:14:54.375199+0000","last_undegraded":"2024-09-13T09:14:54.375199+0000","last_fullsized":"2024-09-13T09:14:54.375199+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:45.810610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093900099999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":11993,"num_read_kb":21637,"num_write":921,"num_write_kb":7095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":848912,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"140'605","reported_seq":7893,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375888+0000","last_change":"2024-09-13T09:14:35.256646+0000","last_active":"2024-09-13T09:14:54.375888+0000","last_peered":"2024-09-13T09:14:54.375888+0000","last_clean":"2024-09-13T09:14:54.375888+0000","last_became_active":"2024-09-13T09:14:16.749234+0000","last_became_peered":"2024-09-13T09:14:16.749234+0000","last_unstale":"2024-09-13T09:14:54.375888+0000","last_undegraded":"2024-09-13T09:14:54.375888+0000","last_fullsized":"2024-09-13T09:14:54.375888+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:18:42.349049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020237760000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10031,"num_read_kb":30858,"num_write":737,"num_write_kb":10943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":313856,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"147'1435","reported_seq":16444,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688945+0000","last_change":"2024-09-13T09:14:44.400159+0000","last_active":"2024-09-13T09:14:54.688945+0000","last_peered":"2024-09-13T09:14:54.688945+0000","last_clean":"2024-09-13T09:14:54.688945+0000","last_became_active":"2024-09-13T09:14:44.312734+0000","last_became_peered":"2024-09-13T09:14:44.312734+0000","last_unstale":"2024-09-13T09:14:54.688945+0000","last_undegraded":"2024-09-13T09:14:54.688945+0000","last_fullsized":"2024-09-13T09:14:54.688945+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1435,"log_dups_size":0,"ondisk_log_size":1435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.934438+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011136765999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":20060,"num_read_kb":34855,"num_write":1554,"num_write_kb":7371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"147'1480","reported_seq":8609,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689014+0000","last_change":"2024-09-13T09:14:44.511014+0000","last_active":"2024-09-13T09:14:54.689014+0000","last_peered":"2024-09-13T09:14:54.689014+0000","last_clean":"2024-09-13T09:14:54.689014+0000","last_became_active":"2024-09-13T09:14:44.311550+0000","last_became_peered":"2024-09-13T09:14:44.311550+0000","last_unstale":"2024-09-13T09:14:54.689014+0000","last_undegraded":"2024-09-13T09:14:54.689014+0000","last_fullsized":"2024-09-13T09:14:54.689014+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1480,"log_dups_size":0,"ondisk_log_size":1480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:16:38.500976+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.022244201000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13526,"num_read_kb":55870,"num_write":2225,"num_write_kb":20106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"142'635","reported_seq":9508,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375955+0000","last_change":"2024-09-13T09:14:35.256035+0000","last_active":"2024-09-13T09:14:54.375955+0000","last_peered":"2024-09-13T09:14:54.375955+0000","last_clean":"2024-09-13T09:14:54.375955+0000","last_became_active":"2024-09-13T09:14:19.772731+0000","last_became_peered":"2024-09-13T09:14:19.772731+0000","last_unstale":"2024-09-13T09:14:54.375955+0000","last_undegraded":"2024-09-13T09:14:54.375955+0000","last_fullsized":"2024-09-13T09:14:54.375955+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:24:40.563709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001509144,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10475,"num_read_kb":52366,"num_write":680,"num_write_kb":12577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1411144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"141'628","reported_seq":14444,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189254+0000","last_change":"2024-09-13T09:14:35.259843+0000","last_active":"2024-09-13T09:14:55.189254+0000","last_peered":"2024-09-13T09:14:55.189254+0000","last_clean":"2024-09-13T09:14:55.189254+0000","last_became_active":"2024-09-13T09:12:13.362533+0000","last_became_peered":"2024-09-13T09:12:13.362533+0000","last_unstale":"2024-09-13T09:14:55.189254+0000","last_undegraded":"2024-09-13T09:14:55.189254+0000","last_fullsized":"2024-09-13T09:14:55.189254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:50.399315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017407639999999999,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":16059,"num_read_kb":55558,"num_write":617,"num_write_kb":11980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"138'567","reported_seq":8658,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376230+0000","last_change":"2024-09-13T09:14:48.352186+0000","last_active":"2024-09-13T09:14:54.376230+0000","last_peered":"2024-09-13T09:14:54.376230+0000","last_clean":"2024-09-13T09:14:54.376230+0000","last_became_active":"2024-09-13T09:14:48.350477+0000","last_became_peered":"2024-09-13T09:14:48.350477+0000","last_unstale":"2024-09-13T09:14:54.376230+0000","last_undegraded":"2024-09-13T09:14:54.376230+0000","last_fullsized":"2024-09-13T09:14:54.376230+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:57.759426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074163899999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8163,"num_read_kb":27760,"num_write":669,"num_write_kb":11976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"139'576","reported_seq":11071,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375944+0000","last_change":"2024-09-13T09:14:35.255587+0000","last_active":"2024-09-13T09:14:54.375944+0000","last_peered":"2024-09-13T09:14:54.375944+0000","last_clean":"2024-09-13T09:14:54.375944+0000","last_became_active":"2024-09-13T09:14:19.772799+0000","last_became_peered":"2024-09-13T09:14:19.772799+0000","last_unstale":"2024-09-13T09:14:54.375944+0000","last_undegraded":"2024-09-13T09:14:54.375944+0000","last_fullsized":"2024-09-13T09:14:54.375944+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:41:21.622976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001235604,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13056,"num_read_kb":24986,"num_write":439,"num_write_kb":7320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":104960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"147'434","reported_seq":5711,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375290+0000","last_change":"2024-09-13T09:14:44.358128+0000","last_active":"2024-09-13T09:14:54.375290+0000","last_peered":"2024-09-13T09:14:54.375290+0000","last_clean":"2024-09-13T09:14:54.375290+0000","last_became_active":"2024-09-13T09:14:44.317071+0000","last_became_peered":"2024-09-13T09:14:44.317071+0000","last_unstale":"2024-09-13T09:14:54.375290+0000","last_undegraded":"2024-09-13T09:14:54.375290+0000","last_fullsized":"2024-09-13T09:14:54.375290+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:42.567052+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015690497000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6024,"num_read_kb":31779,"num_write":503,"num_write_kb":7984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"147'972","reported_seq":6743,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689023+0000","last_change":"2024-09-13T09:14:44.415816+0000","last_active":"2024-09-13T09:14:54.689023+0000","last_peered":"2024-09-13T09:14:54.689023+0000","last_clean":"2024-09-13T09:14:54.689023+0000","last_became_active":"2024-09-13T09:14:44.309746+0000","last_became_peered":"2024-09-13T09:14:44.309746+0000","last_unstale":"2024-09-13T09:14:54.689023+0000","last_undegraded":"2024-09-13T09:14:54.689023+0000","last_fullsized":"2024-09-13T09:14:54.689023+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":972,"log_dups_size":0,"ondisk_log_size":972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:16:58.217708+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0084025269999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8520,"num_read_kb":43161,"num_write":1440,"num_write_kb":11751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":706121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"139'872","reported_seq":20338,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189647+0000","last_change":"2024-09-13T09:14:35.256574+0000","last_active":"2024-09-13T09:14:55.189647+0000","last_peered":"2024-09-13T09:14:55.189647+0000","last_clean":"2024-09-13T09:14:55.189647+0000","last_became_active":"2024-09-13T09:12:46.250696+0000","last_became_peered":"2024-09-13T09:12:46.250696+0000","last_unstale":"2024-09-13T09:14:55.189647+0000","last_undegraded":"2024-09-13T09:14:55.189647+0000","last_fullsized":"2024-09-13T09:14:55.189647+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:43.821997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019682329999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":21840,"num_read_kb":55389,"num_write":1298,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1136450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"140'665","reported_seq":8589,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189530+0000","last_change":"2024-09-13T09:14:35.255745+0000","last_active":"2024-09-13T09:14:55.189530+0000","last_peered":"2024-09-13T09:14:55.189530+0000","last_clean":"2024-09-13T09:14:55.189530+0000","last_became_active":"2024-09-13T09:12:46.254016+0000","last_became_peered":"2024-09-13T09:12:46.254016+0000","last_unstale":"2024-09-13T09:14:55.189530+0000","last_undegraded":"2024-09-13T09:14:55.189530+0000","last_fullsized":"2024-09-13T09:14:55.189530+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:50.193567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013411219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10880,"num_read_kb":53346,"num_write":603,"num_write_kb":9445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1110608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688996+0000","last_change":"2024-09-13T09:14:44.316371+0000","last_active":"2024-09-13T09:14:54.688996+0000","last_peered":"2024-09-13T09:14:54.688996+0000","last_clean":"2024-09-13T09:14:54.688996+0000","last_became_active":"2024-09-13T09:14:44.316062+0000","last_became_peered":"2024-09-13T09:14:44.316062+0000","last_unstale":"2024-09-13T09:14:54.688996+0000","last_undegraded":"2024-09-13T09:14:54.688996+0000","last_fullsized":"2024-09-13T09:14:54.688996+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:06:27.687674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"142'753","reported_seq":10733,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375022+0000","last_change":"2024-09-13T09:14:35.253996+0000","last_active":"2024-09-13T09:14:54.375022+0000","last_peered":"2024-09-13T09:14:54.375022+0000","last_clean":"2024-09-13T09:14:54.375022+0000","last_became_active":"2024-09-13T09:12:13.365496+0000","last_became_peered":"2024-09-13T09:12:13.365496+0000","last_unstale":"2024-09-13T09:14:54.375022+0000","last_undegraded":"2024-09-13T09:14:54.375022+0000","last_fullsized":"2024-09-13T09:14:54.375022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:29:49.115977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063501900000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11213,"num_read_kb":65815,"num_write":999,"num_write_kb":12170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":156,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376437+0000","last_change":"2024-09-13T09:14:48.352849+0000","last_active":"2024-09-13T09:14:54.376437+0000","last_peered":"2024-09-13T09:14:54.376437+0000","last_clean":"2024-09-13T09:14:54.376437+0000","last_became_active":"2024-09-13T09:14:48.352502+0000","last_became_peered":"2024-09-13T09:14:48.352502+0000","last_unstale":"2024-09-13T09:14:54.376437+0000","last_undegraded":"2024-09-13T09:14:54.376437+0000","last_fullsized":"2024-09-13T09:14:54.376437+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:04.532498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"141'1047","reported_seq":10558,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375958+0000","last_change":"2024-09-13T09:14:35.256434+0000","last_active":"2024-09-13T09:14:54.375958+0000","last_peered":"2024-09-13T09:14:54.375958+0000","last_clean":"2024-09-13T09:14:54.375958+0000","last_became_active":"2024-09-13T09:14:16.752774+0000","last_became_peered":"2024-09-13T09:14:16.752774+0000","last_unstale":"2024-09-13T09:14:54.375958+0000","last_undegraded":"2024-09-13T09:14:54.375958+0000","last_fullsized":"2024-09-13T09:14:54.375958+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1047,"log_dups_size":0,"ondisk_log_size":1047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:37:58.086486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001863822,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":16931,"num_read_kb":50979,"num_write":1279,"num_write_kb":12713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":413184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":121,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189554+0000","last_change":"2024-09-13T09:12:43.533829+0000","last_active":"2024-09-13T09:14:55.189554+0000","last_peered":"2024-09-13T09:14:55.189554+0000","last_clean":"2024-09-13T09:14:55.189554+0000","last_became_active":"2024-09-13T09:12:43.533557+0000","last_became_peered":"2024-09-13T09:12:43.533557+0000","last_unstale":"2024-09-13T09:14:55.189554+0000","last_undegraded":"2024-09-13T09:14:55.189554+0000","last_fullsized":"2024-09-13T09:14:55.189554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:51.939164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"141'707","reported_seq":13355,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376484+0000","last_change":"2024-09-13T09:14:48.351345+0000","last_active":"2024-09-13T09:14:54.376484+0000","last_peered":"2024-09-13T09:14:54.376484+0000","last_clean":"2024-09-13T09:14:54.376484+0000","last_became_active":"2024-09-13T09:14:48.348909+0000","last_became_peered":"2024-09-13T09:14:48.348909+0000","last_unstale":"2024-09-13T09:14:54.376484+0000","last_undegraded":"2024-09-13T09:14:54.376484+0000","last_fullsized":"2024-09-13T09:14:54.376484+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:55:37.482616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074057,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":15357,"num_read_kb":39900,"num_write":798,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":29184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376531+0000","last_change":"2024-09-13T09:14:48.351236+0000","last_active":"2024-09-13T09:14:54.376531+0000","last_peered":"2024-09-13T09:14:54.376531+0000","last_clean":"2024-09-13T09:14:54.376531+0000","last_became_active":"2024-09-13T09:14:48.348560+0000","last_became_peered":"2024-09-13T09:14:48.348560+0000","last_unstale":"2024-09-13T09:14:54.376531+0000","last_undegraded":"2024-09-13T09:14:54.376531+0000","last_fullsized":"2024-09-13T09:14:54.376531+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:21.050148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"142'748","reported_seq":7661,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688947+0000","last_change":"2024-09-13T09:14:44.367184+0000","last_active":"2024-09-13T09:14:54.688947+0000","last_peered":"2024-09-13T09:14:54.688947+0000","last_clean":"2024-09-13T09:14:54.688947+0000","last_became_active":"2024-09-13T09:14:44.309435+0000","last_became_peered":"2024-09-13T09:14:44.309435+0000","last_unstale":"2024-09-13T09:14:54.688947+0000","last_undegraded":"2024-09-13T09:14:54.688947+0000","last_fullsized":"2024-09-13T09:14:54.688947+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:57.739078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039806149999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10405,"num_read_kb":27276,"num_write":715,"num_write_kb":2474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":182784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"142'634","reported_seq":11576,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189327+0000","last_change":"2024-09-13T09:14:35.258893+0000","last_active":"2024-09-13T09:14:55.189327+0000","last_peered":"2024-09-13T09:14:55.189327+0000","last_clean":"2024-09-13T09:14:55.189327+0000","last_became_active":"2024-09-13T09:12:13.367770+0000","last_became_peered":"2024-09-13T09:12:13.367770+0000","last_unstale":"2024-09-13T09:14:55.189327+0000","last_undegraded":"2024-09-13T09:14:55.189327+0000","last_fullsized":"2024-09-13T09:14:55.189327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:21:50.690425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001191176,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10993,"num_read_kb":67376,"num_write":601,"num_write_kb":14012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":122,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688871+0000","last_change":"2024-09-13T09:14:44.312125+0000","last_active":"2024-09-13T09:14:54.688871+0000","last_peered":"2024-09-13T09:14:54.688871+0000","last_clean":"2024-09-13T09:14:54.688871+0000","last_became_active":"2024-09-13T09:14:44.311178+0000","last_became_peered":"2024-09-13T09:14:44.311178+0000","last_unstale":"2024-09-13T09:14:54.688871+0000","last_undegraded":"2024-09-13T09:14:54.688871+0000","last_fullsized":"2024-09-13T09:14:54.688871+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:46.245017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"147'620","reported_seq":9651,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688842+0000","last_change":"2024-09-13T09:14:44.440284+0000","last_active":"2024-09-13T09:14:54.688842+0000","last_peered":"2024-09-13T09:14:54.688842+0000","last_clean":"2024-09-13T09:14:54.688842+0000","last_became_active":"2024-09-13T09:14:44.310016+0000","last_became_peered":"2024-09-13T09:14:44.310016+0000","last_unstale":"2024-09-13T09:14:54.688842+0000","last_undegraded":"2024-09-13T09:14:54.688842+0000","last_fullsized":"2024-09-13T09:14:54.688842+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:55.802300+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.007574824,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":9521,"num_read_kb":48768,"num_write":764,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1695776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376024+0000","last_change":"2024-09-13T09:14:16.749189+0000","last_active":"2024-09-13T09:14:54.376024+0000","last_peered":"2024-09-13T09:14:54.376024+0000","last_clean":"2024-09-13T09:14:54.376024+0000","last_became_active":"2024-09-13T09:14:16.748865+0000","last_became_peered":"2024-09-13T09:14:16.748865+0000","last_unstale":"2024-09-13T09:14:54.376024+0000","last_undegraded":"2024-09-13T09:14:54.376024+0000","last_fullsized":"2024-09-13T09:14:54.376024+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:25:39.217684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"142'332","reported_seq":3860,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.374904+0000","last_change":"2024-09-13T09:14:35.253968+0000","last_active":"2024-09-13T09:14:54.374904+0000","last_peered":"2024-09-13T09:14:54.374904+0000","last_clean":"2024-09-13T09:14:54.374904+0000","last_became_active":"2024-09-13T09:12:13.367633+0000","last_became_peered":"2024-09-13T09:12:13.367633+0000","last_unstale":"2024-09-13T09:14:54.374904+0000","last_undegraded":"2024-09-13T09:14:54.374904+0000","last_fullsized":"2024-09-13T09:14:54.374904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:34.734632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094036499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3280,"num_read_kb":26753,"num_write":312,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189164+0000","last_change":"2024-09-13T09:12:43.532201+0000","last_active":"2024-09-13T09:14:55.189164+0000","last_peered":"2024-09-13T09:14:55.189164+0000","last_clean":"2024-09-13T09:14:55.189164+0000","last_became_active":"2024-09-13T09:12:43.531942+0000","last_became_peered":"2024-09-13T09:12:43.531942+0000","last_unstale":"2024-09-13T09:14:55.189164+0000","last_undegraded":"2024-09-13T09:14:55.189164+0000","last_fullsized":"2024-09-13T09:14:55.189164+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:26.408883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"141'653","reported_seq":3816,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376444+0000","last_change":"2024-09-13T09:14:48.351027+0000","last_active":"2024-09-13T09:14:54.376444+0000","last_peered":"2024-09-13T09:14:54.376444+0000","last_clean":"2024-09-13T09:14:54.376444+0000","last_became_active":"2024-09-13T09:14:48.349138+0000","last_became_peered":"2024-09-13T09:14:48.349138+0000","last_unstale":"2024-09-13T09:14:54.376444+0000","last_undegraded":"2024-09-13T09:14:54.376444+0000","last_fullsized":"2024-09-13T09:14:54.376444+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:46.295942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039583799999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4158,"num_read_kb":20449,"num_write":848,"num_write_kb":5187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":155,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688806+0000","last_change":"2024-09-13T09:14:44.309742+0000","last_active":"2024-09-13T09:14:54.688806+0000","last_peered":"2024-09-13T09:14:54.688806+0000","last_clean":"2024-09-13T09:14:54.688806+0000","last_became_active":"2024-09-13T09:14:44.309293+0000","last_became_peered":"2024-09-13T09:14:44.309293+0000","last_unstale":"2024-09-13T09:14:54.688806+0000","last_undegraded":"2024-09-13T09:14:54.688806+0000","last_fullsized":"2024-09-13T09:14:54.688806+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:28:57.831071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":324,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189255+0000","last_change":"2024-09-13T09:12:46.248714+0000","last_active":"2024-09-13T09:14:55.189255+0000","last_peered":"2024-09-13T09:14:55.189255+0000","last_clean":"2024-09-13T09:14:55.189255+0000","last_became_active":"2024-09-13T09:12:46.248451+0000","last_became_peered":"2024-09-13T09:12:46.248451+0000","last_unstale":"2024-09-13T09:14:55.189255+0000","last_undegraded":"2024-09-13T09:14:55.189255+0000","last_fullsized":"2024-09-13T09:14:55.189255+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:27.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"141'498","reported_seq":4698,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376040+0000","last_change":"2024-09-13T09:14:35.257020+0000","last_active":"2024-09-13T09:14:54.376040+0000","last_peered":"2024-09-13T09:14:54.376040+0000","last_clean":"2024-09-13T09:14:54.376040+0000","last_became_active":"2024-09-13T09:14:16.754685+0000","last_became_peered":"2024-09-13T09:14:16.754685+0000","last_unstale":"2024-09-13T09:14:54.376040+0000","last_undegraded":"2024-09-13T09:14:54.376040+0000","last_fullsized":"2024-09-13T09:14:54.376040+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:04.966596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022974409999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4733,"num_read_kb":22943,"num_write":604,"num_write_kb":15013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":45056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"142'1216","reported_seq":13783,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189219+0000","last_change":"2024-09-13T09:14:35.259629+0000","last_active":"2024-09-13T09:14:55.189219+0000","last_peered":"2024-09-13T09:14:55.189219+0000","last_clean":"2024-09-13T09:14:55.189219+0000","last_became_active":"2024-09-13T09:14:19.774300+0000","last_became_peered":"2024-09-13T09:14:19.774300+0000","last_unstale":"2024-09-13T09:14:55.189219+0000","last_undegraded":"2024-09-13T09:14:55.189219+0000","last_fullsized":"2024-09-13T09:14:55.189219+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1216,"log_dups_size":0,"ondisk_log_size":1216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:36:43.693732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001648159,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18678,"num_read_kb":75566,"num_write":1537,"num_write_kb":13034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1249280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"141'596","reported_seq":11902,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.381858+0000","last_change":"2024-09-13T09:14:35.256213+0000","last_active":"2024-09-13T09:14:55.381858+0000","last_peered":"2024-09-13T09:14:55.381858+0000","last_clean":"2024-09-13T09:14:55.381858+0000","last_became_active":"2024-09-13T09:12:46.252578+0000","last_became_peered":"2024-09-13T09:12:46.252578+0000","last_unstale":"2024-09-13T09:14:55.381858+0000","last_undegraded":"2024-09-13T09:14:55.381858+0000","last_fullsized":"2024-09-13T09:14:55.381858+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:03.931492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001634775,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":12679,"num_read_kb":50965,"num_write":515,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2590736,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"141'525","reported_seq":8198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376250+0000","last_change":"2024-09-13T09:14:48.351125+0000","last_active":"2024-09-13T09:14:54.376250+0000","last_peered":"2024-09-13T09:14:54.376250+0000","last_clean":"2024-09-13T09:14:54.376250+0000","last_became_active":"2024-09-13T09:14:48.349512+0000","last_became_peered":"2024-09-13T09:14:48.349512+0000","last_unstale":"2024-09-13T09:14:54.376250+0000","last_undegraded":"2024-09-13T09:14:54.376250+0000","last_fullsized":"2024-09-13T09:14:54.376250+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:17.628884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016583220000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8119,"num_read_kb":73640,"num_write":604,"num_write_kb":21508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"147'866","reported_seq":13468,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688927+0000","last_change":"2024-09-13T09:14:44.477496+0000","last_active":"2024-09-13T09:14:54.688927+0000","last_peered":"2024-09-13T09:14:54.688927+0000","last_clean":"2024-09-13T09:14:54.688927+0000","last_became_active":"2024-09-13T09:14:44.313818+0000","last_became_peered":"2024-09-13T09:14:44.313818+0000","last_unstale":"2024-09-13T09:14:54.688927+0000","last_undegraded":"2024-09-13T09:14:54.688927+0000","last_fullsized":"2024-09-13T09:14:54.688927+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:00.720161+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.022344567999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":15285,"num_read_kb":90031,"num_write":1043,"num_write_kb":32713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"135'363","reported_seq":3876,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189530+0000","last_change":"2024-09-13T09:14:35.255447+0000","last_active":"2024-09-13T09:14:55.189530+0000","last_peered":"2024-09-13T09:14:55.189530+0000","last_clean":"2024-09-13T09:14:55.189530+0000","last_became_active":"2024-09-13T09:14:16.751511+0000","last_became_peered":"2024-09-13T09:14:16.751511+0000","last_unstale":"2024-09-13T09:14:55.189530+0000","last_undegraded":"2024-09-13T09:14:55.189530+0000","last_fullsized":"2024-09-13T09:14:55.189530+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:57.453044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011111789999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3406,"num_read_kb":19855,"num_write":419,"num_write_kb":12151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":4294187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"147'381","reported_seq":5925,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688949+0000","last_change":"2024-09-13T09:14:44.516389+0000","last_active":"2024-09-13T09:14:54.688949+0000","last_peered":"2024-09-13T09:14:54.688949+0000","last_clean":"2024-09-13T09:14:54.688949+0000","last_became_active":"2024-09-13T09:14:44.315188+0000","last_became_peered":"2024-09-13T09:14:44.315188+0000","last_unstale":"2024-09-13T09:14:54.688949+0000","last_undegraded":"2024-09-13T09:14:54.688949+0000","last_fullsized":"2024-09-13T09:14:54.688949+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:10.043789+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0058820570000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6135,"num_read_kb":25216,"num_write":405,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"141'532","reported_seq":7039,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689064+0000","last_change":"2024-09-13T09:14:44.426902+0000","last_active":"2024-09-13T09:14:54.689064+0000","last_peered":"2024-09-13T09:14:54.689064+0000","last_clean":"2024-09-13T09:14:54.689064+0000","last_became_active":"2024-09-13T09:14:44.309465+0000","last_became_peered":"2024-09-13T09:14:44.309465+0000","last_unstale":"2024-09-13T09:14:54.689064+0000","last_undegraded":"2024-09-13T09:14:54.689064+0000","last_fullsized":"2024-09-13T09:14:54.689064+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:19:43.291662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039480829999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6633,"num_read_kb":20805,"num_write":596,"num_write_kb":7043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":635004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"147'533","reported_seq":7949,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688828+0000","last_change":"2024-09-13T09:14:44.375157+0000","last_active":"2024-09-13T09:14:54.688828+0000","last_peered":"2024-09-13T09:14:54.688828+0000","last_clean":"2024-09-13T09:14:54.688828+0000","last_became_active":"2024-09-13T09:14:44.308779+0000","last_became_peered":"2024-09-13T09:14:44.308779+0000","last_unstale":"2024-09-13T09:14:54.688828+0000","last_undegraded":"2024-09-13T09:14:54.688828+0000","last_fullsized":"2024-09-13T09:14:54.688828+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:00.643049+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0066958310000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7617,"num_read_kb":37352,"num_write":622,"num_write_kb":17983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"139'545","reported_seq":4837,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189301+0000","last_change":"2024-09-13T09:14:35.256957+0000","last_active":"2024-09-13T09:14:55.189301+0000","last_peered":"2024-09-13T09:14:55.189301+0000","last_clean":"2024-09-13T09:14:55.189301+0000","last_became_active":"2024-09-13T09:12:43.532504+0000","last_became_peered":"2024-09-13T09:12:43.532504+0000","last_unstale":"2024-09-13T09:14:55.189301+0000","last_undegraded":"2024-09-13T09:14:55.189301+0000","last_fullsized":"2024-09-13T09:14:55.189301+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:25:04.225563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021463989999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5509,"num_read_kb":32081,"num_write":673,"num_write_kb":7448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":694311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"141'1115","reported_seq":20789,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.374845+0000","last_change":"2024-09-13T09:14:35.253524+0000","last_active":"2024-09-13T09:14:54.374845+0000","last_peered":"2024-09-13T09:14:54.374845+0000","last_clean":"2024-09-13T09:14:54.374845+0000","last_became_active":"2024-09-13T09:12:46.248261+0000","last_became_peered":"2024-09-13T09:12:46.248261+0000","last_unstale":"2024-09-13T09:14:54.374845+0000","last_undegraded":"2024-09-13T09:14:54.374845+0000","last_fullsized":"2024-09-13T09:14:54.374845+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:44.056762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092792899999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":26946,"num_read_kb":67879,"num_write":1035,"num_write_kb":10041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":676864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"141'603","reported_seq":6565,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189413+0000","last_change":"2024-09-13T09:14:44.326177+0000","last_active":"2024-09-13T09:14:55.189413+0000","last_peered":"2024-09-13T09:14:55.189413+0000","last_clean":"2024-09-13T09:14:55.189413+0000","last_became_active":"2024-09-13T09:14:44.314564+0000","last_became_peered":"2024-09-13T09:14:44.314564+0000","last_unstale":"2024-09-13T09:14:55.189413+0000","last_undegraded":"2024-09-13T09:14:55.189413+0000","last_fullsized":"2024-09-13T09:14:55.189413+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:58:58.520651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050931029999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8526,"num_read_kb":13102,"num_write":519,"num_write_kb":3941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":28777,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":68,"num_read":356751,"num_read_kb":1364316,"num_write":26574,"num_write_kb":359719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":18167194,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28441,"internal_metadata":0},"log_size":22993,"ondisk_log_size":22993,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":146,"seq":627065225221,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31180,"kb_used_data":1848,"kb_used_omap":31,"kb_used_meta":29280,"kb_avail":94340660,"statfs":{"total":96636764160,"available":96604835840,"internally_reserved":0,"allocated":1892352,"data_stored":790048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32283,"internal_metadata":29983205},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640297,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64952,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306888,"statfs":{"total":96636764160,"available":96570253312,"internally_reserved":0,"allocated":1880064,"data_stored":785020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39700000000000002},"min":{"1min":0.28199999999999997,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.52700000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.40200000000000002,"15min":0.39100000000000001},"min":{"1min":0.26800000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.31,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41799999999999998,"15min":0.41999999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56799999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.48399999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.438},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53400000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.38200000000000001,"15min":0.38900000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.52400000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.23799999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54400000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.375},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40000000000000002,"15min":0.40799999999999997},"min":{"1min":0.25,"5min":0.249,"15min":0.249},"max":{"1min":0.55400000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46999999999999997}]}]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59784,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312056,"statfs":{"total":96636764160,"available":96575545344,"internally_reserved":0,"allocated":2473984,"data_stored":1378241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.40600000000000003},"min":{"1min":0.21099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.47499999999999998,"15min":0.44500000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.54500000000000004,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.5990000000000002,"5min":4.5990000000000002,"15min":4.5990000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.40100000000000002,"15min":0.377},"min":{"1min":0.23799999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":5.165,"5min":5.165,"15min":5.165},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38300000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.45000000000000001,"15min":0.40300000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":8.7959999999999994,"5min":8.7959999999999994,"15min":8.7959999999999994},"last":0.50600000000000001}]}]},{"osd":2,"up_from":122,"seq":523986010122,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35080,"kb_used_data":1844,"kb_used_omap":37,"kb_used_meta":33178,"kb_avail":94336760,"statfs":{"total":96636764160,"available":96600842240,"internally_reserved":0,"allocated":1888256,"data_stored":790029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38061,"internal_metadata":33975123},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":1,"up_from":11,"seq":47244640297,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55112,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316728,"statfs":{"total":96636764160,"available":96580329472,"internally_reserved":0,"allocated":1896448,"data_stored":790067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.32700000000000001,"15min":0.33700000000000002},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.54700000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.32100000000000001,"15min":0.32500000000000001},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.56000000000000005,"5min":0.753,"15min":0.753},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.379},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.433}]},{"osd":3,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.496,"15min":0.50700000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.60099999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.58199999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.46600000000000003,"15min":0.47399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.622,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.68300000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.496,"15min":0.504},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.747},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.47099999999999997,"15min":0.48399999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.66900000000000004}]}]},{"osd":0,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63880,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307960,"statfs":{"total":96636764160,"available":96571351040,"internally_reserved":0,"allocated":2482176,"data_stored":1380397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.26600000000000001,"5min":0.27900000000000003,"15min":0.28100000000000003},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.28000000000000003,"15min":0.28199999999999997},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":0.54800000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.84399999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.44800000000000001,"15min":0.45900000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.69899999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.442,"15min":0.44500000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.745}]},{"osd":4,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45000000000000001,"15min":0.45600000000000002},"min":{"1min":0.26300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.874,"5min":0.879,"15min":0.879},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.45000000000000001,"15min":0.45700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68500000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.88}]},{"osd":5,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46700000000000003,"15min":0.47599999999999998},"min":{"1min":0.29199999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88700000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.45500000000000002,"15min":0.45800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69799999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.77600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16703,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11738,"internal_metadata":0}]}} 2024-09-13T09:15:00.808 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:15:01.093 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:15:01.094 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:15:01.105 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-13T09:14:59.549222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":68,"num_read":356867,"num_read_kb":1364416,"num_write":26735,"num_write_kb":362137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":666,"num_bytes_recovered":19347930,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23113,"ondisk_log_size":23113,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":309988,"kb_used_data":12220,"kb_used_omap":74,"kb_used_meta":297589,"kb_avail":565921052,"statfs":{"total":579820584960,"available":579503157248,"internally_reserved":0,"allocated":12513280,"data_stored":5913802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76719,"internal_metadata":304731217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.181465"},"pg_stats":[{"pgid":"3.1f","version":"142'857","reported_seq":9653,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375199+0000","last_change":"2024-09-13T09:14:35.254388+0000","last_active":"2024-09-13T09:14:54.375199+0000","last_peered":"2024-09-13T09:14:54.375199+0000","last_clean":"2024-09-13T09:14:54.375199+0000","last_became_active":"2024-09-13T09:12:43.534990+0000","last_became_peered":"2024-09-13T09:12:43.534990+0000","last_unstale":"2024-09-13T09:14:54.375199+0000","last_undegraded":"2024-09-13T09:14:54.375199+0000","last_fullsized":"2024-09-13T09:14:54.375199+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:45.810610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093900099999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":11993,"num_read_kb":21637,"num_write":921,"num_write_kb":7095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":848912,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"140'605","reported_seq":7893,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375888+0000","last_change":"2024-09-13T09:14:35.256646+0000","last_active":"2024-09-13T09:14:54.375888+0000","last_peered":"2024-09-13T09:14:54.375888+0000","last_clean":"2024-09-13T09:14:54.375888+0000","last_became_active":"2024-09-13T09:14:16.749234+0000","last_became_peered":"2024-09-13T09:14:16.749234+0000","last_unstale":"2024-09-13T09:14:54.375888+0000","last_undegraded":"2024-09-13T09:14:54.375888+0000","last_fullsized":"2024-09-13T09:14:54.375888+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:18:42.349049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020237760000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10031,"num_read_kb":30858,"num_write":737,"num_write_kb":10943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":313856,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"147'1435","reported_seq":16444,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688945+0000","last_change":"2024-09-13T09:14:44.400159+0000","last_active":"2024-09-13T09:14:54.688945+0000","last_peered":"2024-09-13T09:14:54.688945+0000","last_clean":"2024-09-13T09:14:54.688945+0000","last_became_active":"2024-09-13T09:14:44.312734+0000","last_became_peered":"2024-09-13T09:14:44.312734+0000","last_unstale":"2024-09-13T09:14:54.688945+0000","last_undegraded":"2024-09-13T09:14:54.688945+0000","last_fullsized":"2024-09-13T09:14:54.688945+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1435,"log_dups_size":0,"ondisk_log_size":1435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.934438+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011136765999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":20060,"num_read_kb":34855,"num_write":1554,"num_write_kb":7371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"147'1480","reported_seq":8609,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689014+0000","last_change":"2024-09-13T09:14:44.511014+0000","last_active":"2024-09-13T09:14:54.689014+0000","last_peered":"2024-09-13T09:14:54.689014+0000","last_clean":"2024-09-13T09:14:54.689014+0000","last_became_active":"2024-09-13T09:14:44.311550+0000","last_became_peered":"2024-09-13T09:14:44.311550+0000","last_unstale":"2024-09-13T09:14:54.689014+0000","last_undegraded":"2024-09-13T09:14:54.689014+0000","last_fullsized":"2024-09-13T09:14:54.689014+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1480,"log_dups_size":0,"ondisk_log_size":1480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:16:38.500976+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.022244201000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13526,"num_read_kb":55870,"num_write":2225,"num_write_kb":20106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"142'635","reported_seq":9508,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375955+0000","last_change":"2024-09-13T09:14:35.256035+0000","last_active":"2024-09-13T09:14:54.375955+0000","last_peered":"2024-09-13T09:14:54.375955+0000","last_clean":"2024-09-13T09:14:54.375955+0000","last_became_active":"2024-09-13T09:14:19.772731+0000","last_became_peered":"2024-09-13T09:14:19.772731+0000","last_unstale":"2024-09-13T09:14:54.375955+0000","last_undegraded":"2024-09-13T09:14:54.375955+0000","last_fullsized":"2024-09-13T09:14:54.375955+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:24:40.563709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001509144,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10475,"num_read_kb":52366,"num_write":680,"num_write_kb":12577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1411144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"141'628","reported_seq":14444,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189254+0000","last_change":"2024-09-13T09:14:35.259843+0000","last_active":"2024-09-13T09:14:55.189254+0000","last_peered":"2024-09-13T09:14:55.189254+0000","last_clean":"2024-09-13T09:14:55.189254+0000","last_became_active":"2024-09-13T09:12:13.362533+0000","last_became_peered":"2024-09-13T09:12:13.362533+0000","last_unstale":"2024-09-13T09:14:55.189254+0000","last_undegraded":"2024-09-13T09:14:55.189254+0000","last_fullsized":"2024-09-13T09:14:55.189254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:50.399315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017407639999999999,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":16059,"num_read_kb":55558,"num_write":617,"num_write_kb":11980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"138'567","reported_seq":8658,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376230+0000","last_change":"2024-09-13T09:14:48.352186+0000","last_active":"2024-09-13T09:14:54.376230+0000","last_peered":"2024-09-13T09:14:54.376230+0000","last_clean":"2024-09-13T09:14:54.376230+0000","last_became_active":"2024-09-13T09:14:48.350477+0000","last_became_peered":"2024-09-13T09:14:48.350477+0000","last_unstale":"2024-09-13T09:14:54.376230+0000","last_undegraded":"2024-09-13T09:14:54.376230+0000","last_fullsized":"2024-09-13T09:14:54.376230+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:57.759426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074163899999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8163,"num_read_kb":27760,"num_write":669,"num_write_kb":11976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"139'576","reported_seq":11071,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375944+0000","last_change":"2024-09-13T09:14:35.255587+0000","last_active":"2024-09-13T09:14:54.375944+0000","last_peered":"2024-09-13T09:14:54.375944+0000","last_clean":"2024-09-13T09:14:54.375944+0000","last_became_active":"2024-09-13T09:14:19.772799+0000","last_became_peered":"2024-09-13T09:14:19.772799+0000","last_unstale":"2024-09-13T09:14:54.375944+0000","last_undegraded":"2024-09-13T09:14:54.375944+0000","last_fullsized":"2024-09-13T09:14:54.375944+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:41:21.622976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001235604,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13056,"num_read_kb":24986,"num_write":439,"num_write_kb":7320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":104960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"147'434","reported_seq":5711,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375290+0000","last_change":"2024-09-13T09:14:44.358128+0000","last_active":"2024-09-13T09:14:54.375290+0000","last_peered":"2024-09-13T09:14:54.375290+0000","last_clean":"2024-09-13T09:14:54.375290+0000","last_became_active":"2024-09-13T09:14:44.317071+0000","last_became_peered":"2024-09-13T09:14:44.317071+0000","last_unstale":"2024-09-13T09:14:54.375290+0000","last_undegraded":"2024-09-13T09:14:54.375290+0000","last_fullsized":"2024-09-13T09:14:54.375290+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:42.567052+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015690497000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6024,"num_read_kb":31779,"num_write":503,"num_write_kb":7984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"147'972","reported_seq":6743,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689023+0000","last_change":"2024-09-13T09:14:44.415816+0000","last_active":"2024-09-13T09:14:54.689023+0000","last_peered":"2024-09-13T09:14:54.689023+0000","last_clean":"2024-09-13T09:14:54.689023+0000","last_became_active":"2024-09-13T09:14:44.309746+0000","last_became_peered":"2024-09-13T09:14:44.309746+0000","last_unstale":"2024-09-13T09:14:54.689023+0000","last_undegraded":"2024-09-13T09:14:54.689023+0000","last_fullsized":"2024-09-13T09:14:54.689023+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":972,"log_dups_size":0,"ondisk_log_size":972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:16:58.217708+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0084025269999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8520,"num_read_kb":43161,"num_write":1440,"num_write_kb":11751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":706121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"139'872","reported_seq":20338,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189647+0000","last_change":"2024-09-13T09:14:35.256574+0000","last_active":"2024-09-13T09:14:55.189647+0000","last_peered":"2024-09-13T09:14:55.189647+0000","last_clean":"2024-09-13T09:14:55.189647+0000","last_became_active":"2024-09-13T09:12:46.250696+0000","last_became_peered":"2024-09-13T09:12:46.250696+0000","last_unstale":"2024-09-13T09:14:55.189647+0000","last_undegraded":"2024-09-13T09:14:55.189647+0000","last_fullsized":"2024-09-13T09:14:55.189647+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:43.821997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019682329999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":21840,"num_read_kb":55389,"num_write":1298,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1136450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"140'665","reported_seq":8589,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189530+0000","last_change":"2024-09-13T09:14:35.255745+0000","last_active":"2024-09-13T09:14:55.189530+0000","last_peered":"2024-09-13T09:14:55.189530+0000","last_clean":"2024-09-13T09:14:55.189530+0000","last_became_active":"2024-09-13T09:12:46.254016+0000","last_became_peered":"2024-09-13T09:12:46.254016+0000","last_unstale":"2024-09-13T09:14:55.189530+0000","last_undegraded":"2024-09-13T09:14:55.189530+0000","last_fullsized":"2024-09-13T09:14:55.189530+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:50.193567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013411219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10880,"num_read_kb":53346,"num_write":603,"num_write_kb":9445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1110608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688996+0000","last_change":"2024-09-13T09:14:44.316371+0000","last_active":"2024-09-13T09:14:54.688996+0000","last_peered":"2024-09-13T09:14:54.688996+0000","last_clean":"2024-09-13T09:14:54.688996+0000","last_became_active":"2024-09-13T09:14:44.316062+0000","last_became_peered":"2024-09-13T09:14:44.316062+0000","last_unstale":"2024-09-13T09:14:54.688996+0000","last_undegraded":"2024-09-13T09:14:54.688996+0000","last_fullsized":"2024-09-13T09:14:54.688996+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:06:27.687674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"142'753","reported_seq":10733,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375022+0000","last_change":"2024-09-13T09:14:35.253996+0000","last_active":"2024-09-13T09:14:54.375022+0000","last_peered":"2024-09-13T09:14:54.375022+0000","last_clean":"2024-09-13T09:14:54.375022+0000","last_became_active":"2024-09-13T09:12:13.365496+0000","last_became_peered":"2024-09-13T09:12:13.365496+0000","last_unstale":"2024-09-13T09:14:54.375022+0000","last_undegraded":"2024-09-13T09:14:54.375022+0000","last_fullsized":"2024-09-13T09:14:54.375022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:29:49.115977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063501900000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11213,"num_read_kb":65815,"num_write":999,"num_write_kb":12170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":156,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376437+0000","last_change":"2024-09-13T09:14:48.352849+0000","last_active":"2024-09-13T09:14:54.376437+0000","last_peered":"2024-09-13T09:14:54.376437+0000","last_clean":"2024-09-13T09:14:54.376437+0000","last_became_active":"2024-09-13T09:14:48.352502+0000","last_became_peered":"2024-09-13T09:14:48.352502+0000","last_unstale":"2024-09-13T09:14:54.376437+0000","last_undegraded":"2024-09-13T09:14:54.376437+0000","last_fullsized":"2024-09-13T09:14:54.376437+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:04.532498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"141'1047","reported_seq":10558,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375958+0000","last_change":"2024-09-13T09:14:35.256434+0000","last_active":"2024-09-13T09:14:54.375958+0000","last_peered":"2024-09-13T09:14:54.375958+0000","last_clean":"2024-09-13T09:14:54.375958+0000","last_became_active":"2024-09-13T09:14:16.752774+0000","last_became_peered":"2024-09-13T09:14:16.752774+0000","last_unstale":"2024-09-13T09:14:54.375958+0000","last_undegraded":"2024-09-13T09:14:54.375958+0000","last_fullsized":"2024-09-13T09:14:54.375958+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1047,"log_dups_size":0,"ondisk_log_size":1047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:37:58.086486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001863822,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":16931,"num_read_kb":50979,"num_write":1279,"num_write_kb":12713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":413184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":121,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189554+0000","last_change":"2024-09-13T09:12:43.533829+0000","last_active":"2024-09-13T09:14:55.189554+0000","last_peered":"2024-09-13T09:14:55.189554+0000","last_clean":"2024-09-13T09:14:55.189554+0000","last_became_active":"2024-09-13T09:12:43.533557+0000","last_became_peered":"2024-09-13T09:12:43.533557+0000","last_unstale":"2024-09-13T09:14:55.189554+0000","last_undegraded":"2024-09-13T09:14:55.189554+0000","last_fullsized":"2024-09-13T09:14:55.189554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:51.939164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"141'707","reported_seq":13355,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376484+0000","last_change":"2024-09-13T09:14:48.351345+0000","last_active":"2024-09-13T09:14:54.376484+0000","last_peered":"2024-09-13T09:14:54.376484+0000","last_clean":"2024-09-13T09:14:54.376484+0000","last_became_active":"2024-09-13T09:14:48.348909+0000","last_became_peered":"2024-09-13T09:14:48.348909+0000","last_unstale":"2024-09-13T09:14:54.376484+0000","last_undegraded":"2024-09-13T09:14:54.376484+0000","last_fullsized":"2024-09-13T09:14:54.376484+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:55:37.482616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074057,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":15357,"num_read_kb":39900,"num_write":798,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":29184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376531+0000","last_change":"2024-09-13T09:14:48.351236+0000","last_active":"2024-09-13T09:14:54.376531+0000","last_peered":"2024-09-13T09:14:54.376531+0000","last_clean":"2024-09-13T09:14:54.376531+0000","last_became_active":"2024-09-13T09:14:48.348560+0000","last_became_peered":"2024-09-13T09:14:48.348560+0000","last_unstale":"2024-09-13T09:14:54.376531+0000","last_undegraded":"2024-09-13T09:14:54.376531+0000","last_fullsized":"2024-09-13T09:14:54.376531+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:21.050148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"142'748","reported_seq":7661,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688947+0000","last_change":"2024-09-13T09:14:44.367184+0000","last_active":"2024-09-13T09:14:54.688947+0000","last_peered":"2024-09-13T09:14:54.688947+0000","last_clean":"2024-09-13T09:14:54.688947+0000","last_became_active":"2024-09-13T09:14:44.309435+0000","last_became_peered":"2024-09-13T09:14:44.309435+0000","last_unstale":"2024-09-13T09:14:54.688947+0000","last_undegraded":"2024-09-13T09:14:54.688947+0000","last_fullsized":"2024-09-13T09:14:54.688947+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:57.739078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039806149999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10405,"num_read_kb":27276,"num_write":715,"num_write_kb":2474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":182784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"142'634","reported_seq":11576,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189327+0000","last_change":"2024-09-13T09:14:35.258893+0000","last_active":"2024-09-13T09:14:55.189327+0000","last_peered":"2024-09-13T09:14:55.189327+0000","last_clean":"2024-09-13T09:14:55.189327+0000","last_became_active":"2024-09-13T09:12:13.367770+0000","last_became_peered":"2024-09-13T09:12:13.367770+0000","last_unstale":"2024-09-13T09:14:55.189327+0000","last_undegraded":"2024-09-13T09:14:55.189327+0000","last_fullsized":"2024-09-13T09:14:55.189327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:21:50.690425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001191176,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10993,"num_read_kb":67376,"num_write":601,"num_write_kb":14012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":122,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688871+0000","last_change":"2024-09-13T09:14:44.312125+0000","last_active":"2024-09-13T09:14:54.688871+0000","last_peered":"2024-09-13T09:14:54.688871+0000","last_clean":"2024-09-13T09:14:54.688871+0000","last_became_active":"2024-09-13T09:14:44.311178+0000","last_became_peered":"2024-09-13T09:14:44.311178+0000","last_unstale":"2024-09-13T09:14:54.688871+0000","last_undegraded":"2024-09-13T09:14:54.688871+0000","last_fullsized":"2024-09-13T09:14:54.688871+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:46.245017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"147'620","reported_seq":9651,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688842+0000","last_change":"2024-09-13T09:14:44.440284+0000","last_active":"2024-09-13T09:14:54.688842+0000","last_peered":"2024-09-13T09:14:54.688842+0000","last_clean":"2024-09-13T09:14:54.688842+0000","last_became_active":"2024-09-13T09:14:44.310016+0000","last_became_peered":"2024-09-13T09:14:44.310016+0000","last_unstale":"2024-09-13T09:14:54.688842+0000","last_undegraded":"2024-09-13T09:14:54.688842+0000","last_fullsized":"2024-09-13T09:14:54.688842+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:55.802300+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.007574824,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":9521,"num_read_kb":48768,"num_write":764,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1695776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376024+0000","last_change":"2024-09-13T09:14:16.749189+0000","last_active":"2024-09-13T09:14:54.376024+0000","last_peered":"2024-09-13T09:14:54.376024+0000","last_clean":"2024-09-13T09:14:54.376024+0000","last_became_active":"2024-09-13T09:14:16.748865+0000","last_became_peered":"2024-09-13T09:14:16.748865+0000","last_unstale":"2024-09-13T09:14:54.376024+0000","last_undegraded":"2024-09-13T09:14:54.376024+0000","last_fullsized":"2024-09-13T09:14:54.376024+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:25:39.217684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"142'332","reported_seq":3860,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.374904+0000","last_change":"2024-09-13T09:14:35.253968+0000","last_active":"2024-09-13T09:14:54.374904+0000","last_peered":"2024-09-13T09:14:54.374904+0000","last_clean":"2024-09-13T09:14:54.374904+0000","last_became_active":"2024-09-13T09:12:13.367633+0000","last_became_peered":"2024-09-13T09:12:13.367633+0000","last_unstale":"2024-09-13T09:14:54.374904+0000","last_undegraded":"2024-09-13T09:14:54.374904+0000","last_fullsized":"2024-09-13T09:14:54.374904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:34.734632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094036499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3280,"num_read_kb":26753,"num_write":312,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189164+0000","last_change":"2024-09-13T09:12:43.532201+0000","last_active":"2024-09-13T09:14:55.189164+0000","last_peered":"2024-09-13T09:14:55.189164+0000","last_clean":"2024-09-13T09:14:55.189164+0000","last_became_active":"2024-09-13T09:12:43.531942+0000","last_became_peered":"2024-09-13T09:12:43.531942+0000","last_unstale":"2024-09-13T09:14:55.189164+0000","last_undegraded":"2024-09-13T09:14:55.189164+0000","last_fullsized":"2024-09-13T09:14:55.189164+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:26.408883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"141'653","reported_seq":3816,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376444+0000","last_change":"2024-09-13T09:14:48.351027+0000","last_active":"2024-09-13T09:14:54.376444+0000","last_peered":"2024-09-13T09:14:54.376444+0000","last_clean":"2024-09-13T09:14:54.376444+0000","last_became_active":"2024-09-13T09:14:48.349138+0000","last_became_peered":"2024-09-13T09:14:48.349138+0000","last_unstale":"2024-09-13T09:14:54.376444+0000","last_undegraded":"2024-09-13T09:14:54.376444+0000","last_fullsized":"2024-09-13T09:14:54.376444+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:46.295942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039583799999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4158,"num_read_kb":20449,"num_write":848,"num_write_kb":5187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":155,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688806+0000","last_change":"2024-09-13T09:14:44.309742+0000","last_active":"2024-09-13T09:14:54.688806+0000","last_peered":"2024-09-13T09:14:54.688806+0000","last_clean":"2024-09-13T09:14:54.688806+0000","last_became_active":"2024-09-13T09:14:44.309293+0000","last_became_peered":"2024-09-13T09:14:44.309293+0000","last_unstale":"2024-09-13T09:14:54.688806+0000","last_undegraded":"2024-09-13T09:14:54.688806+0000","last_fullsized":"2024-09-13T09:14:54.688806+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:28:57.831071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":324,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189255+0000","last_change":"2024-09-13T09:12:46.248714+0000","last_active":"2024-09-13T09:14:55.189255+0000","last_peered":"2024-09-13T09:14:55.189255+0000","last_clean":"2024-09-13T09:14:55.189255+0000","last_became_active":"2024-09-13T09:12:46.248451+0000","last_became_peered":"2024-09-13T09:12:46.248451+0000","last_unstale":"2024-09-13T09:14:55.189255+0000","last_undegraded":"2024-09-13T09:14:55.189255+0000","last_fullsized":"2024-09-13T09:14:55.189255+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:27.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"141'498","reported_seq":4698,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376040+0000","last_change":"2024-09-13T09:14:35.257020+0000","last_active":"2024-09-13T09:14:54.376040+0000","last_peered":"2024-09-13T09:14:54.376040+0000","last_clean":"2024-09-13T09:14:54.376040+0000","last_became_active":"2024-09-13T09:14:16.754685+0000","last_became_peered":"2024-09-13T09:14:16.754685+0000","last_unstale":"2024-09-13T09:14:54.376040+0000","last_undegraded":"2024-09-13T09:14:54.376040+0000","last_fullsized":"2024-09-13T09:14:54.376040+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:04.966596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022974409999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4733,"num_read_kb":22943,"num_write":604,"num_write_kb":15013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":45056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"142'1216","reported_seq":13783,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189219+0000","last_change":"2024-09-13T09:14:35.259629+0000","last_active":"2024-09-13T09:14:55.189219+0000","last_peered":"2024-09-13T09:14:55.189219+0000","last_clean":"2024-09-13T09:14:55.189219+0000","last_became_active":"2024-09-13T09:14:19.774300+0000","last_became_peered":"2024-09-13T09:14:19.774300+0000","last_unstale":"2024-09-13T09:14:55.189219+0000","last_undegraded":"2024-09-13T09:14:55.189219+0000","last_fullsized":"2024-09-13T09:14:55.189219+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1216,"log_dups_size":0,"ondisk_log_size":1216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:36:43.693732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001648159,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18678,"num_read_kb":75566,"num_write":1537,"num_write_kb":13034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1249280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"141'596","reported_seq":11902,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.381858+0000","last_change":"2024-09-13T09:14:35.256213+0000","last_active":"2024-09-13T09:14:55.381858+0000","last_peered":"2024-09-13T09:14:55.381858+0000","last_clean":"2024-09-13T09:14:55.381858+0000","last_became_active":"2024-09-13T09:12:46.252578+0000","last_became_peered":"2024-09-13T09:12:46.252578+0000","last_unstale":"2024-09-13T09:14:55.381858+0000","last_undegraded":"2024-09-13T09:14:55.381858+0000","last_fullsized":"2024-09-13T09:14:55.381858+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:03.931492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001634775,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":12679,"num_read_kb":50965,"num_write":515,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2590736,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"141'525","reported_seq":8198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376250+0000","last_change":"2024-09-13T09:14:48.351125+0000","last_active":"2024-09-13T09:14:54.376250+0000","last_peered":"2024-09-13T09:14:54.376250+0000","last_clean":"2024-09-13T09:14:54.376250+0000","last_became_active":"2024-09-13T09:14:48.349512+0000","last_became_peered":"2024-09-13T09:14:48.349512+0000","last_unstale":"2024-09-13T09:14:54.376250+0000","last_undegraded":"2024-09-13T09:14:54.376250+0000","last_fullsized":"2024-09-13T09:14:54.376250+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:17.628884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016583220000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8119,"num_read_kb":73640,"num_write":604,"num_write_kb":21508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"147'866","reported_seq":13468,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688927+0000","last_change":"2024-09-13T09:14:44.477496+0000","last_active":"2024-09-13T09:14:54.688927+0000","last_peered":"2024-09-13T09:14:54.688927+0000","last_clean":"2024-09-13T09:14:54.688927+0000","last_became_active":"2024-09-13T09:14:44.313818+0000","last_became_peered":"2024-09-13T09:14:44.313818+0000","last_unstale":"2024-09-13T09:14:54.688927+0000","last_undegraded":"2024-09-13T09:14:54.688927+0000","last_fullsized":"2024-09-13T09:14:54.688927+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:00.720161+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.022344567999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":15285,"num_read_kb":90031,"num_write":1043,"num_write_kb":32713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"135'363","reported_seq":3876,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189530+0000","last_change":"2024-09-13T09:14:35.255447+0000","last_active":"2024-09-13T09:14:55.189530+0000","last_peered":"2024-09-13T09:14:55.189530+0000","last_clean":"2024-09-13T09:14:55.189530+0000","last_became_active":"2024-09-13T09:14:16.751511+0000","last_became_peered":"2024-09-13T09:14:16.751511+0000","last_unstale":"2024-09-13T09:14:55.189530+0000","last_undegraded":"2024-09-13T09:14:55.189530+0000","last_fullsized":"2024-09-13T09:14:55.189530+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:57.453044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011111789999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3406,"num_read_kb":19855,"num_write":419,"num_write_kb":12151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":4294187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"147'381","reported_seq":5925,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688949+0000","last_change":"2024-09-13T09:14:44.516389+0000","last_active":"2024-09-13T09:14:54.688949+0000","last_peered":"2024-09-13T09:14:54.688949+0000","last_clean":"2024-09-13T09:14:54.688949+0000","last_became_active":"2024-09-13T09:14:44.315188+0000","last_became_peered":"2024-09-13T09:14:44.315188+0000","last_unstale":"2024-09-13T09:14:54.688949+0000","last_undegraded":"2024-09-13T09:14:54.688949+0000","last_fullsized":"2024-09-13T09:14:54.688949+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:10.043789+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0058820570000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6135,"num_read_kb":25216,"num_write":405,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"141'532","reported_seq":7039,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689064+0000","last_change":"2024-09-13T09:14:44.426902+0000","last_active":"2024-09-13T09:14:54.689064+0000","last_peered":"2024-09-13T09:14:54.689064+0000","last_clean":"2024-09-13T09:14:54.689064+0000","last_became_active":"2024-09-13T09:14:44.309465+0000","last_became_peered":"2024-09-13T09:14:44.309465+0000","last_unstale":"2024-09-13T09:14:54.689064+0000","last_undegraded":"2024-09-13T09:14:54.689064+0000","last_fullsized":"2024-09-13T09:14:54.689064+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:19:43.291662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039480829999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6633,"num_read_kb":20805,"num_write":596,"num_write_kb":7043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":635004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"147'533","reported_seq":7949,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688828+0000","last_change":"2024-09-13T09:14:44.375157+0000","last_active":"2024-09-13T09:14:54.688828+0000","last_peered":"2024-09-13T09:14:54.688828+0000","last_clean":"2024-09-13T09:14:54.688828+0000","last_became_active":"2024-09-13T09:14:44.308779+0000","last_became_peered":"2024-09-13T09:14:44.308779+0000","last_unstale":"2024-09-13T09:14:54.688828+0000","last_undegraded":"2024-09-13T09:14:54.688828+0000","last_fullsized":"2024-09-13T09:14:54.688828+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:00.643049+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0066958310000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7617,"num_read_kb":37352,"num_write":622,"num_write_kb":17983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"139'545","reported_seq":4837,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189301+0000","last_change":"2024-09-13T09:14:35.256957+0000","last_active":"2024-09-13T09:14:55.189301+0000","last_peered":"2024-09-13T09:14:55.189301+0000","last_clean":"2024-09-13T09:14:55.189301+0000","last_became_active":"2024-09-13T09:12:43.532504+0000","last_became_peered":"2024-09-13T09:12:43.532504+0000","last_unstale":"2024-09-13T09:14:55.189301+0000","last_undegraded":"2024-09-13T09:14:55.189301+0000","last_fullsized":"2024-09-13T09:14:55.189301+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:25:04.225563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021463989999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5509,"num_read_kb":32081,"num_write":673,"num_write_kb":7448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":694311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"141'1115","reported_seq":20789,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.374845+0000","last_change":"2024-09-13T09:14:35.253524+0000","last_active":"2024-09-13T09:14:54.374845+0000","last_peered":"2024-09-13T09:14:54.374845+0000","last_clean":"2024-09-13T09:14:54.374845+0000","last_became_active":"2024-09-13T09:12:46.248261+0000","last_became_peered":"2024-09-13T09:12:46.248261+0000","last_unstale":"2024-09-13T09:14:54.374845+0000","last_undegraded":"2024-09-13T09:14:54.374845+0000","last_fullsized":"2024-09-13T09:14:54.374845+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:44.056762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092792899999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":26946,"num_read_kb":67879,"num_write":1035,"num_write_kb":10041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":676864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"141'603","reported_seq":6565,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189413+0000","last_change":"2024-09-13T09:14:44.326177+0000","last_active":"2024-09-13T09:14:55.189413+0000","last_peered":"2024-09-13T09:14:55.189413+0000","last_clean":"2024-09-13T09:14:55.189413+0000","last_became_active":"2024-09-13T09:14:44.314564+0000","last_became_peered":"2024-09-13T09:14:44.314564+0000","last_unstale":"2024-09-13T09:14:55.189413+0000","last_undegraded":"2024-09-13T09:14:55.189413+0000","last_fullsized":"2024-09-13T09:14:55.189413+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:58:58.520651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050931029999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8526,"num_read_kb":13102,"num_write":519,"num_write_kb":3941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":28777,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":68,"num_read":356751,"num_read_kb":1364316,"num_write":26574,"num_write_kb":359719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":18167194,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28441,"internal_metadata":0},"log_size":22993,"ondisk_log_size":22993,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":146,"seq":627065225221,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31180,"kb_used_data":1848,"kb_used_omap":31,"kb_used_meta":29280,"kb_avail":94340660,"statfs":{"total":96636764160,"available":96604835840,"internally_reserved":0,"allocated":1892352,"data_stored":790048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32283,"internal_metadata":29983205},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640297,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64952,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306888,"statfs":{"total":96636764160,"available":96570253312,"internally_reserved":0,"allocated":1880064,"data_stored":785020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39700000000000002},"min":{"1min":0.28199999999999997,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.52700000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.40200000000000002,"15min":0.39100000000000001},"min":{"1min":0.26800000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.31,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41799999999999998,"15min":0.41999999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56799999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.48399999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.438},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53400000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.38200000000000001,"15min":0.38900000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.52400000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.23799999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54400000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.375},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40000000000000002,"15min":0.40799999999999997},"min":{"1min":0.25,"5min":0.249,"15min":0.249},"max":{"1min":0.55400000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46999999999999997}]}]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59784,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312056,"statfs":{"total":96636764160,"available":96575545344,"internally_reserved":0,"allocated":2473984,"data_stored":1378241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.40600000000000003},"min":{"1min":0.21099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.47499999999999998,"15min":0.44500000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.54500000000000004,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.5990000000000002,"5min":4.5990000000000002,"15min":4.5990000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.40100000000000002,"15min":0.377},"min":{"1min":0.23799999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":5.165,"5min":5.165,"15min":5.165},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38300000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.45000000000000001,"15min":0.40300000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":8.7959999999999994,"5min":8.7959999999999994,"15min":8.7959999999999994},"last":0.50600000000000001}]}]},{"osd":2,"up_from":122,"seq":523986010122,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35080,"kb_used_data":1844,"kb_used_omap":37,"kb_used_meta":33178,"kb_avail":94336760,"statfs":{"total":96636764160,"available":96600842240,"internally_reserved":0,"allocated":1888256,"data_stored":790029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38061,"internal_metadata":33975123},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":1,"up_from":11,"seq":47244640297,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55112,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316728,"statfs":{"total":96636764160,"available":96580329472,"internally_reserved":0,"allocated":1896448,"data_stored":790067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.32700000000000001,"15min":0.33700000000000002},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.54700000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.32100000000000001,"15min":0.32500000000000001},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.56000000000000005,"5min":0.753,"15min":0.753},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.379},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.433}]},{"osd":3,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.496,"15min":0.50700000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.60099999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.58199999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.46600000000000003,"15min":0.47399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.622,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.68300000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.496,"15min":0.504},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.747},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.47099999999999997,"15min":0.48399999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.66900000000000004}]}]},{"osd":0,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63880,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307960,"statfs":{"total":96636764160,"available":96571351040,"internally_reserved":0,"allocated":2482176,"data_stored":1380397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.26600000000000001,"5min":0.27900000000000003,"15min":0.28100000000000003},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.28000000000000003,"15min":0.28199999999999997},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":0.54800000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.84399999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.44800000000000001,"15min":0.45900000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.69899999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.442,"15min":0.44500000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.745}]},{"osd":4,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45000000000000001,"15min":0.45600000000000002},"min":{"1min":0.26300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.874,"5min":0.879,"15min":0.879},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.45000000000000001,"15min":0.45700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68500000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.88}]},{"osd":5,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46700000000000003,"15min":0.47599999999999998},"min":{"1min":0.29199999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88700000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.45500000000000002,"15min":0.45800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69799999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.77600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16703,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11738,"internal_metadata":0}]}} 2024-09-13T09:15:01.107 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T09:15:01.107 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:15:01.393 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:15:01.394 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:15:01.405 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-13T09:14:59.549222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":68,"num_read":356867,"num_read_kb":1364416,"num_write":26735,"num_write_kb":362137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":666,"num_bytes_recovered":19347930,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23113,"ondisk_log_size":23113,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":309988,"kb_used_data":12220,"kb_used_omap":74,"kb_used_meta":297589,"kb_avail":565921052,"statfs":{"total":579820584960,"available":579503157248,"internally_reserved":0,"allocated":12513280,"data_stored":5913802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76719,"internal_metadata":304731217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.181465"},"pg_stats":[{"pgid":"3.1f","version":"142'857","reported_seq":9653,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375199+0000","last_change":"2024-09-13T09:14:35.254388+0000","last_active":"2024-09-13T09:14:54.375199+0000","last_peered":"2024-09-13T09:14:54.375199+0000","last_clean":"2024-09-13T09:14:54.375199+0000","last_became_active":"2024-09-13T09:12:43.534990+0000","last_became_peered":"2024-09-13T09:12:43.534990+0000","last_unstale":"2024-09-13T09:14:54.375199+0000","last_undegraded":"2024-09-13T09:14:54.375199+0000","last_fullsized":"2024-09-13T09:14:54.375199+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:45.810610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093900099999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":11993,"num_read_kb":21637,"num_write":921,"num_write_kb":7095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":848912,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"140'605","reported_seq":7893,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375888+0000","last_change":"2024-09-13T09:14:35.256646+0000","last_active":"2024-09-13T09:14:54.375888+0000","last_peered":"2024-09-13T09:14:54.375888+0000","last_clean":"2024-09-13T09:14:54.375888+0000","last_became_active":"2024-09-13T09:14:16.749234+0000","last_became_peered":"2024-09-13T09:14:16.749234+0000","last_unstale":"2024-09-13T09:14:54.375888+0000","last_undegraded":"2024-09-13T09:14:54.375888+0000","last_fullsized":"2024-09-13T09:14:54.375888+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:18:42.349049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020237760000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10031,"num_read_kb":30858,"num_write":737,"num_write_kb":10943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":313856,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"147'1435","reported_seq":16444,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688945+0000","last_change":"2024-09-13T09:14:44.400159+0000","last_active":"2024-09-13T09:14:54.688945+0000","last_peered":"2024-09-13T09:14:54.688945+0000","last_clean":"2024-09-13T09:14:54.688945+0000","last_became_active":"2024-09-13T09:14:44.312734+0000","last_became_peered":"2024-09-13T09:14:44.312734+0000","last_unstale":"2024-09-13T09:14:54.688945+0000","last_undegraded":"2024-09-13T09:14:54.688945+0000","last_fullsized":"2024-09-13T09:14:54.688945+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1435,"log_dups_size":0,"ondisk_log_size":1435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.934438+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011136765999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":20060,"num_read_kb":34855,"num_write":1554,"num_write_kb":7371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"147'1480","reported_seq":8609,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689014+0000","last_change":"2024-09-13T09:14:44.511014+0000","last_active":"2024-09-13T09:14:54.689014+0000","last_peered":"2024-09-13T09:14:54.689014+0000","last_clean":"2024-09-13T09:14:54.689014+0000","last_became_active":"2024-09-13T09:14:44.311550+0000","last_became_peered":"2024-09-13T09:14:44.311550+0000","last_unstale":"2024-09-13T09:14:54.689014+0000","last_undegraded":"2024-09-13T09:14:54.689014+0000","last_fullsized":"2024-09-13T09:14:54.689014+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1480,"log_dups_size":0,"ondisk_log_size":1480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:16:38.500976+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.022244201000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13526,"num_read_kb":55870,"num_write":2225,"num_write_kb":20106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"142'635","reported_seq":9508,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375955+0000","last_change":"2024-09-13T09:14:35.256035+0000","last_active":"2024-09-13T09:14:54.375955+0000","last_peered":"2024-09-13T09:14:54.375955+0000","last_clean":"2024-09-13T09:14:54.375955+0000","last_became_active":"2024-09-13T09:14:19.772731+0000","last_became_peered":"2024-09-13T09:14:19.772731+0000","last_unstale":"2024-09-13T09:14:54.375955+0000","last_undegraded":"2024-09-13T09:14:54.375955+0000","last_fullsized":"2024-09-13T09:14:54.375955+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:24:40.563709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001509144,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10475,"num_read_kb":52366,"num_write":680,"num_write_kb":12577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1411144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"141'628","reported_seq":14444,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189254+0000","last_change":"2024-09-13T09:14:35.259843+0000","last_active":"2024-09-13T09:14:55.189254+0000","last_peered":"2024-09-13T09:14:55.189254+0000","last_clean":"2024-09-13T09:14:55.189254+0000","last_became_active":"2024-09-13T09:12:13.362533+0000","last_became_peered":"2024-09-13T09:12:13.362533+0000","last_unstale":"2024-09-13T09:14:55.189254+0000","last_undegraded":"2024-09-13T09:14:55.189254+0000","last_fullsized":"2024-09-13T09:14:55.189254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:50.399315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017407639999999999,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":16059,"num_read_kb":55558,"num_write":617,"num_write_kb":11980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"138'567","reported_seq":8658,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376230+0000","last_change":"2024-09-13T09:14:48.352186+0000","last_active":"2024-09-13T09:14:54.376230+0000","last_peered":"2024-09-13T09:14:54.376230+0000","last_clean":"2024-09-13T09:14:54.376230+0000","last_became_active":"2024-09-13T09:14:48.350477+0000","last_became_peered":"2024-09-13T09:14:48.350477+0000","last_unstale":"2024-09-13T09:14:54.376230+0000","last_undegraded":"2024-09-13T09:14:54.376230+0000","last_fullsized":"2024-09-13T09:14:54.376230+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:57.759426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074163899999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8163,"num_read_kb":27760,"num_write":669,"num_write_kb":11976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"139'576","reported_seq":11071,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375944+0000","last_change":"2024-09-13T09:14:35.255587+0000","last_active":"2024-09-13T09:14:54.375944+0000","last_peered":"2024-09-13T09:14:54.375944+0000","last_clean":"2024-09-13T09:14:54.375944+0000","last_became_active":"2024-09-13T09:14:19.772799+0000","last_became_peered":"2024-09-13T09:14:19.772799+0000","last_unstale":"2024-09-13T09:14:54.375944+0000","last_undegraded":"2024-09-13T09:14:54.375944+0000","last_fullsized":"2024-09-13T09:14:54.375944+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:41:21.622976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001235604,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13056,"num_read_kb":24986,"num_write":439,"num_write_kb":7320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":104960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"147'434","reported_seq":5711,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375290+0000","last_change":"2024-09-13T09:14:44.358128+0000","last_active":"2024-09-13T09:14:54.375290+0000","last_peered":"2024-09-13T09:14:54.375290+0000","last_clean":"2024-09-13T09:14:54.375290+0000","last_became_active":"2024-09-13T09:14:44.317071+0000","last_became_peered":"2024-09-13T09:14:44.317071+0000","last_unstale":"2024-09-13T09:14:54.375290+0000","last_undegraded":"2024-09-13T09:14:54.375290+0000","last_fullsized":"2024-09-13T09:14:54.375290+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:42.567052+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015690497000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6024,"num_read_kb":31779,"num_write":503,"num_write_kb":7984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"147'972","reported_seq":6743,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689023+0000","last_change":"2024-09-13T09:14:44.415816+0000","last_active":"2024-09-13T09:14:54.689023+0000","last_peered":"2024-09-13T09:14:54.689023+0000","last_clean":"2024-09-13T09:14:54.689023+0000","last_became_active":"2024-09-13T09:14:44.309746+0000","last_became_peered":"2024-09-13T09:14:44.309746+0000","last_unstale":"2024-09-13T09:14:54.689023+0000","last_undegraded":"2024-09-13T09:14:54.689023+0000","last_fullsized":"2024-09-13T09:14:54.689023+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":972,"log_dups_size":0,"ondisk_log_size":972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:16:58.217708+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0084025269999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8520,"num_read_kb":43161,"num_write":1440,"num_write_kb":11751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":706121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"139'872","reported_seq":20338,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189647+0000","last_change":"2024-09-13T09:14:35.256574+0000","last_active":"2024-09-13T09:14:55.189647+0000","last_peered":"2024-09-13T09:14:55.189647+0000","last_clean":"2024-09-13T09:14:55.189647+0000","last_became_active":"2024-09-13T09:12:46.250696+0000","last_became_peered":"2024-09-13T09:12:46.250696+0000","last_unstale":"2024-09-13T09:14:55.189647+0000","last_undegraded":"2024-09-13T09:14:55.189647+0000","last_fullsized":"2024-09-13T09:14:55.189647+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:43.821997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019682329999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":21840,"num_read_kb":55389,"num_write":1298,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1136450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"140'665","reported_seq":8589,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189530+0000","last_change":"2024-09-13T09:14:35.255745+0000","last_active":"2024-09-13T09:14:55.189530+0000","last_peered":"2024-09-13T09:14:55.189530+0000","last_clean":"2024-09-13T09:14:55.189530+0000","last_became_active":"2024-09-13T09:12:46.254016+0000","last_became_peered":"2024-09-13T09:12:46.254016+0000","last_unstale":"2024-09-13T09:14:55.189530+0000","last_undegraded":"2024-09-13T09:14:55.189530+0000","last_fullsized":"2024-09-13T09:14:55.189530+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:50.193567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013411219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10880,"num_read_kb":53346,"num_write":603,"num_write_kb":9445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1110608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688996+0000","last_change":"2024-09-13T09:14:44.316371+0000","last_active":"2024-09-13T09:14:54.688996+0000","last_peered":"2024-09-13T09:14:54.688996+0000","last_clean":"2024-09-13T09:14:54.688996+0000","last_became_active":"2024-09-13T09:14:44.316062+0000","last_became_peered":"2024-09-13T09:14:44.316062+0000","last_unstale":"2024-09-13T09:14:54.688996+0000","last_undegraded":"2024-09-13T09:14:54.688996+0000","last_fullsized":"2024-09-13T09:14:54.688996+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:06:27.687674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"142'753","reported_seq":10733,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375022+0000","last_change":"2024-09-13T09:14:35.253996+0000","last_active":"2024-09-13T09:14:54.375022+0000","last_peered":"2024-09-13T09:14:54.375022+0000","last_clean":"2024-09-13T09:14:54.375022+0000","last_became_active":"2024-09-13T09:12:13.365496+0000","last_became_peered":"2024-09-13T09:12:13.365496+0000","last_unstale":"2024-09-13T09:14:54.375022+0000","last_undegraded":"2024-09-13T09:14:54.375022+0000","last_fullsized":"2024-09-13T09:14:54.375022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:29:49.115977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063501900000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11213,"num_read_kb":65815,"num_write":999,"num_write_kb":12170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":156,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376437+0000","last_change":"2024-09-13T09:14:48.352849+0000","last_active":"2024-09-13T09:14:54.376437+0000","last_peered":"2024-09-13T09:14:54.376437+0000","last_clean":"2024-09-13T09:14:54.376437+0000","last_became_active":"2024-09-13T09:14:48.352502+0000","last_became_peered":"2024-09-13T09:14:48.352502+0000","last_unstale":"2024-09-13T09:14:54.376437+0000","last_undegraded":"2024-09-13T09:14:54.376437+0000","last_fullsized":"2024-09-13T09:14:54.376437+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:04.532498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"141'1047","reported_seq":10558,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375958+0000","last_change":"2024-09-13T09:14:35.256434+0000","last_active":"2024-09-13T09:14:54.375958+0000","last_peered":"2024-09-13T09:14:54.375958+0000","last_clean":"2024-09-13T09:14:54.375958+0000","last_became_active":"2024-09-13T09:14:16.752774+0000","last_became_peered":"2024-09-13T09:14:16.752774+0000","last_unstale":"2024-09-13T09:14:54.375958+0000","last_undegraded":"2024-09-13T09:14:54.375958+0000","last_fullsized":"2024-09-13T09:14:54.375958+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1047,"log_dups_size":0,"ondisk_log_size":1047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:37:58.086486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001863822,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":16931,"num_read_kb":50979,"num_write":1279,"num_write_kb":12713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":413184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":121,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189554+0000","last_change":"2024-09-13T09:12:43.533829+0000","last_active":"2024-09-13T09:14:55.189554+0000","last_peered":"2024-09-13T09:14:55.189554+0000","last_clean":"2024-09-13T09:14:55.189554+0000","last_became_active":"2024-09-13T09:12:43.533557+0000","last_became_peered":"2024-09-13T09:12:43.533557+0000","last_unstale":"2024-09-13T09:14:55.189554+0000","last_undegraded":"2024-09-13T09:14:55.189554+0000","last_fullsized":"2024-09-13T09:14:55.189554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:51.939164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"141'707","reported_seq":13355,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376484+0000","last_change":"2024-09-13T09:14:48.351345+0000","last_active":"2024-09-13T09:14:54.376484+0000","last_peered":"2024-09-13T09:14:54.376484+0000","last_clean":"2024-09-13T09:14:54.376484+0000","last_became_active":"2024-09-13T09:14:48.348909+0000","last_became_peered":"2024-09-13T09:14:48.348909+0000","last_unstale":"2024-09-13T09:14:54.376484+0000","last_undegraded":"2024-09-13T09:14:54.376484+0000","last_fullsized":"2024-09-13T09:14:54.376484+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:55:37.482616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074057,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":15357,"num_read_kb":39900,"num_write":798,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":29184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376531+0000","last_change":"2024-09-13T09:14:48.351236+0000","last_active":"2024-09-13T09:14:54.376531+0000","last_peered":"2024-09-13T09:14:54.376531+0000","last_clean":"2024-09-13T09:14:54.376531+0000","last_became_active":"2024-09-13T09:14:48.348560+0000","last_became_peered":"2024-09-13T09:14:48.348560+0000","last_unstale":"2024-09-13T09:14:54.376531+0000","last_undegraded":"2024-09-13T09:14:54.376531+0000","last_fullsized":"2024-09-13T09:14:54.376531+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:21.050148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"142'748","reported_seq":7661,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688947+0000","last_change":"2024-09-13T09:14:44.367184+0000","last_active":"2024-09-13T09:14:54.688947+0000","last_peered":"2024-09-13T09:14:54.688947+0000","last_clean":"2024-09-13T09:14:54.688947+0000","last_became_active":"2024-09-13T09:14:44.309435+0000","last_became_peered":"2024-09-13T09:14:44.309435+0000","last_unstale":"2024-09-13T09:14:54.688947+0000","last_undegraded":"2024-09-13T09:14:54.688947+0000","last_fullsized":"2024-09-13T09:14:54.688947+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:57.739078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039806149999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10405,"num_read_kb":27276,"num_write":715,"num_write_kb":2474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":182784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"142'634","reported_seq":11576,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189327+0000","last_change":"2024-09-13T09:14:35.258893+0000","last_active":"2024-09-13T09:14:55.189327+0000","last_peered":"2024-09-13T09:14:55.189327+0000","last_clean":"2024-09-13T09:14:55.189327+0000","last_became_active":"2024-09-13T09:12:13.367770+0000","last_became_peered":"2024-09-13T09:12:13.367770+0000","last_unstale":"2024-09-13T09:14:55.189327+0000","last_undegraded":"2024-09-13T09:14:55.189327+0000","last_fullsized":"2024-09-13T09:14:55.189327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:21:50.690425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001191176,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10993,"num_read_kb":67376,"num_write":601,"num_write_kb":14012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":122,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688871+0000","last_change":"2024-09-13T09:14:44.312125+0000","last_active":"2024-09-13T09:14:54.688871+0000","last_peered":"2024-09-13T09:14:54.688871+0000","last_clean":"2024-09-13T09:14:54.688871+0000","last_became_active":"2024-09-13T09:14:44.311178+0000","last_became_peered":"2024-09-13T09:14:44.311178+0000","last_unstale":"2024-09-13T09:14:54.688871+0000","last_undegraded":"2024-09-13T09:14:54.688871+0000","last_fullsized":"2024-09-13T09:14:54.688871+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:46.245017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"147'620","reported_seq":9651,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688842+0000","last_change":"2024-09-13T09:14:44.440284+0000","last_active":"2024-09-13T09:14:54.688842+0000","last_peered":"2024-09-13T09:14:54.688842+0000","last_clean":"2024-09-13T09:14:54.688842+0000","last_became_active":"2024-09-13T09:14:44.310016+0000","last_became_peered":"2024-09-13T09:14:44.310016+0000","last_unstale":"2024-09-13T09:14:54.688842+0000","last_undegraded":"2024-09-13T09:14:54.688842+0000","last_fullsized":"2024-09-13T09:14:54.688842+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:55.802300+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.007574824,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":9521,"num_read_kb":48768,"num_write":764,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1695776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376024+0000","last_change":"2024-09-13T09:14:16.749189+0000","last_active":"2024-09-13T09:14:54.376024+0000","last_peered":"2024-09-13T09:14:54.376024+0000","last_clean":"2024-09-13T09:14:54.376024+0000","last_became_active":"2024-09-13T09:14:16.748865+0000","last_became_peered":"2024-09-13T09:14:16.748865+0000","last_unstale":"2024-09-13T09:14:54.376024+0000","last_undegraded":"2024-09-13T09:14:54.376024+0000","last_fullsized":"2024-09-13T09:14:54.376024+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:25:39.217684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"142'332","reported_seq":3860,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.374904+0000","last_change":"2024-09-13T09:14:35.253968+0000","last_active":"2024-09-13T09:14:54.374904+0000","last_peered":"2024-09-13T09:14:54.374904+0000","last_clean":"2024-09-13T09:14:54.374904+0000","last_became_active":"2024-09-13T09:12:13.367633+0000","last_became_peered":"2024-09-13T09:12:13.367633+0000","last_unstale":"2024-09-13T09:14:54.374904+0000","last_undegraded":"2024-09-13T09:14:54.374904+0000","last_fullsized":"2024-09-13T09:14:54.374904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:34.734632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094036499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3280,"num_read_kb":26753,"num_write":312,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189164+0000","last_change":"2024-09-13T09:12:43.532201+0000","last_active":"2024-09-13T09:14:55.189164+0000","last_peered":"2024-09-13T09:14:55.189164+0000","last_clean":"2024-09-13T09:14:55.189164+0000","last_became_active":"2024-09-13T09:12:43.531942+0000","last_became_peered":"2024-09-13T09:12:43.531942+0000","last_unstale":"2024-09-13T09:14:55.189164+0000","last_undegraded":"2024-09-13T09:14:55.189164+0000","last_fullsized":"2024-09-13T09:14:55.189164+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:26.408883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"141'653","reported_seq":3816,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376444+0000","last_change":"2024-09-13T09:14:48.351027+0000","last_active":"2024-09-13T09:14:54.376444+0000","last_peered":"2024-09-13T09:14:54.376444+0000","last_clean":"2024-09-13T09:14:54.376444+0000","last_became_active":"2024-09-13T09:14:48.349138+0000","last_became_peered":"2024-09-13T09:14:48.349138+0000","last_unstale":"2024-09-13T09:14:54.376444+0000","last_undegraded":"2024-09-13T09:14:54.376444+0000","last_fullsized":"2024-09-13T09:14:54.376444+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:46.295942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039583799999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4158,"num_read_kb":20449,"num_write":848,"num_write_kb":5187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":155,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688806+0000","last_change":"2024-09-13T09:14:44.309742+0000","last_active":"2024-09-13T09:14:54.688806+0000","last_peered":"2024-09-13T09:14:54.688806+0000","last_clean":"2024-09-13T09:14:54.688806+0000","last_became_active":"2024-09-13T09:14:44.309293+0000","last_became_peered":"2024-09-13T09:14:44.309293+0000","last_unstale":"2024-09-13T09:14:54.688806+0000","last_undegraded":"2024-09-13T09:14:54.688806+0000","last_fullsized":"2024-09-13T09:14:54.688806+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:28:57.831071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":324,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189255+0000","last_change":"2024-09-13T09:12:46.248714+0000","last_active":"2024-09-13T09:14:55.189255+0000","last_peered":"2024-09-13T09:14:55.189255+0000","last_clean":"2024-09-13T09:14:55.189255+0000","last_became_active":"2024-09-13T09:12:46.248451+0000","last_became_peered":"2024-09-13T09:12:46.248451+0000","last_unstale":"2024-09-13T09:14:55.189255+0000","last_undegraded":"2024-09-13T09:14:55.189255+0000","last_fullsized":"2024-09-13T09:14:55.189255+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:27.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"141'498","reported_seq":4698,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376040+0000","last_change":"2024-09-13T09:14:35.257020+0000","last_active":"2024-09-13T09:14:54.376040+0000","last_peered":"2024-09-13T09:14:54.376040+0000","last_clean":"2024-09-13T09:14:54.376040+0000","last_became_active":"2024-09-13T09:14:16.754685+0000","last_became_peered":"2024-09-13T09:14:16.754685+0000","last_unstale":"2024-09-13T09:14:54.376040+0000","last_undegraded":"2024-09-13T09:14:54.376040+0000","last_fullsized":"2024-09-13T09:14:54.376040+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:04.966596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022974409999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4733,"num_read_kb":22943,"num_write":604,"num_write_kb":15013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":45056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"142'1216","reported_seq":13783,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189219+0000","last_change":"2024-09-13T09:14:35.259629+0000","last_active":"2024-09-13T09:14:55.189219+0000","last_peered":"2024-09-13T09:14:55.189219+0000","last_clean":"2024-09-13T09:14:55.189219+0000","last_became_active":"2024-09-13T09:14:19.774300+0000","last_became_peered":"2024-09-13T09:14:19.774300+0000","last_unstale":"2024-09-13T09:14:55.189219+0000","last_undegraded":"2024-09-13T09:14:55.189219+0000","last_fullsized":"2024-09-13T09:14:55.189219+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1216,"log_dups_size":0,"ondisk_log_size":1216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:36:43.693732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001648159,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18678,"num_read_kb":75566,"num_write":1537,"num_write_kb":13034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1249280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"141'596","reported_seq":11902,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.381858+0000","last_change":"2024-09-13T09:14:35.256213+0000","last_active":"2024-09-13T09:14:55.381858+0000","last_peered":"2024-09-13T09:14:55.381858+0000","last_clean":"2024-09-13T09:14:55.381858+0000","last_became_active":"2024-09-13T09:12:46.252578+0000","last_became_peered":"2024-09-13T09:12:46.252578+0000","last_unstale":"2024-09-13T09:14:55.381858+0000","last_undegraded":"2024-09-13T09:14:55.381858+0000","last_fullsized":"2024-09-13T09:14:55.381858+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:03.931492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001634775,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":12679,"num_read_kb":50965,"num_write":515,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2590736,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"141'525","reported_seq":8198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376250+0000","last_change":"2024-09-13T09:14:48.351125+0000","last_active":"2024-09-13T09:14:54.376250+0000","last_peered":"2024-09-13T09:14:54.376250+0000","last_clean":"2024-09-13T09:14:54.376250+0000","last_became_active":"2024-09-13T09:14:48.349512+0000","last_became_peered":"2024-09-13T09:14:48.349512+0000","last_unstale":"2024-09-13T09:14:54.376250+0000","last_undegraded":"2024-09-13T09:14:54.376250+0000","last_fullsized":"2024-09-13T09:14:54.376250+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:17.628884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016583220000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8119,"num_read_kb":73640,"num_write":604,"num_write_kb":21508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"147'866","reported_seq":13468,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688927+0000","last_change":"2024-09-13T09:14:44.477496+0000","last_active":"2024-09-13T09:14:54.688927+0000","last_peered":"2024-09-13T09:14:54.688927+0000","last_clean":"2024-09-13T09:14:54.688927+0000","last_became_active":"2024-09-13T09:14:44.313818+0000","last_became_peered":"2024-09-13T09:14:44.313818+0000","last_unstale":"2024-09-13T09:14:54.688927+0000","last_undegraded":"2024-09-13T09:14:54.688927+0000","last_fullsized":"2024-09-13T09:14:54.688927+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:00.720161+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.022344567999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":15285,"num_read_kb":90031,"num_write":1043,"num_write_kb":32713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"135'363","reported_seq":3876,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189530+0000","last_change":"2024-09-13T09:14:35.255447+0000","last_active":"2024-09-13T09:14:55.189530+0000","last_peered":"2024-09-13T09:14:55.189530+0000","last_clean":"2024-09-13T09:14:55.189530+0000","last_became_active":"2024-09-13T09:14:16.751511+0000","last_became_peered":"2024-09-13T09:14:16.751511+0000","last_unstale":"2024-09-13T09:14:55.189530+0000","last_undegraded":"2024-09-13T09:14:55.189530+0000","last_fullsized":"2024-09-13T09:14:55.189530+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:57.453044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011111789999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3406,"num_read_kb":19855,"num_write":419,"num_write_kb":12151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":4294187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"147'381","reported_seq":5925,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688949+0000","last_change":"2024-09-13T09:14:44.516389+0000","last_active":"2024-09-13T09:14:54.688949+0000","last_peered":"2024-09-13T09:14:54.688949+0000","last_clean":"2024-09-13T09:14:54.688949+0000","last_became_active":"2024-09-13T09:14:44.315188+0000","last_became_peered":"2024-09-13T09:14:44.315188+0000","last_unstale":"2024-09-13T09:14:54.688949+0000","last_undegraded":"2024-09-13T09:14:54.688949+0000","last_fullsized":"2024-09-13T09:14:54.688949+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:10.043789+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0058820570000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6135,"num_read_kb":25216,"num_write":405,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"141'532","reported_seq":7039,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689064+0000","last_change":"2024-09-13T09:14:44.426902+0000","last_active":"2024-09-13T09:14:54.689064+0000","last_peered":"2024-09-13T09:14:54.689064+0000","last_clean":"2024-09-13T09:14:54.689064+0000","last_became_active":"2024-09-13T09:14:44.309465+0000","last_became_peered":"2024-09-13T09:14:44.309465+0000","last_unstale":"2024-09-13T09:14:54.689064+0000","last_undegraded":"2024-09-13T09:14:54.689064+0000","last_fullsized":"2024-09-13T09:14:54.689064+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:19:43.291662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039480829999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6633,"num_read_kb":20805,"num_write":596,"num_write_kb":7043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":635004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"147'533","reported_seq":7949,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688828+0000","last_change":"2024-09-13T09:14:44.375157+0000","last_active":"2024-09-13T09:14:54.688828+0000","last_peered":"2024-09-13T09:14:54.688828+0000","last_clean":"2024-09-13T09:14:54.688828+0000","last_became_active":"2024-09-13T09:14:44.308779+0000","last_became_peered":"2024-09-13T09:14:44.308779+0000","last_unstale":"2024-09-13T09:14:54.688828+0000","last_undegraded":"2024-09-13T09:14:54.688828+0000","last_fullsized":"2024-09-13T09:14:54.688828+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:00.643049+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0066958310000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7617,"num_read_kb":37352,"num_write":622,"num_write_kb":17983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"139'545","reported_seq":4837,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189301+0000","last_change":"2024-09-13T09:14:35.256957+0000","last_active":"2024-09-13T09:14:55.189301+0000","last_peered":"2024-09-13T09:14:55.189301+0000","last_clean":"2024-09-13T09:14:55.189301+0000","last_became_active":"2024-09-13T09:12:43.532504+0000","last_became_peered":"2024-09-13T09:12:43.532504+0000","last_unstale":"2024-09-13T09:14:55.189301+0000","last_undegraded":"2024-09-13T09:14:55.189301+0000","last_fullsized":"2024-09-13T09:14:55.189301+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:25:04.225563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021463989999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5509,"num_read_kb":32081,"num_write":673,"num_write_kb":7448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":694311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"141'1115","reported_seq":20789,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.374845+0000","last_change":"2024-09-13T09:14:35.253524+0000","last_active":"2024-09-13T09:14:54.374845+0000","last_peered":"2024-09-13T09:14:54.374845+0000","last_clean":"2024-09-13T09:14:54.374845+0000","last_became_active":"2024-09-13T09:12:46.248261+0000","last_became_peered":"2024-09-13T09:12:46.248261+0000","last_unstale":"2024-09-13T09:14:54.374845+0000","last_undegraded":"2024-09-13T09:14:54.374845+0000","last_fullsized":"2024-09-13T09:14:54.374845+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:44.056762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092792899999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":26946,"num_read_kb":67879,"num_write":1035,"num_write_kb":10041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":676864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"141'603","reported_seq":6565,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189413+0000","last_change":"2024-09-13T09:14:44.326177+0000","last_active":"2024-09-13T09:14:55.189413+0000","last_peered":"2024-09-13T09:14:55.189413+0000","last_clean":"2024-09-13T09:14:55.189413+0000","last_became_active":"2024-09-13T09:14:44.314564+0000","last_became_peered":"2024-09-13T09:14:44.314564+0000","last_unstale":"2024-09-13T09:14:55.189413+0000","last_undegraded":"2024-09-13T09:14:55.189413+0000","last_fullsized":"2024-09-13T09:14:55.189413+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:58:58.520651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050931029999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8526,"num_read_kb":13102,"num_write":519,"num_write_kb":3941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":28777,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":68,"num_read":356751,"num_read_kb":1364316,"num_write":26574,"num_write_kb":359719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":18167194,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28441,"internal_metadata":0},"log_size":22993,"ondisk_log_size":22993,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":146,"seq":627065225221,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31180,"kb_used_data":1848,"kb_used_omap":31,"kb_used_meta":29280,"kb_avail":94340660,"statfs":{"total":96636764160,"available":96604835840,"internally_reserved":0,"allocated":1892352,"data_stored":790048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32283,"internal_metadata":29983205},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640297,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64952,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306888,"statfs":{"total":96636764160,"available":96570253312,"internally_reserved":0,"allocated":1880064,"data_stored":785020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39700000000000002},"min":{"1min":0.28199999999999997,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.52700000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.40200000000000002,"15min":0.39100000000000001},"min":{"1min":0.26800000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.31,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41799999999999998,"15min":0.41999999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56799999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.48399999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.438},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53400000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.38200000000000001,"15min":0.38900000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.52400000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.23799999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54400000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.375},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40000000000000002,"15min":0.40799999999999997},"min":{"1min":0.25,"5min":0.249,"15min":0.249},"max":{"1min":0.55400000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46999999999999997}]}]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59784,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312056,"statfs":{"total":96636764160,"available":96575545344,"internally_reserved":0,"allocated":2473984,"data_stored":1378241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.40600000000000003},"min":{"1min":0.21099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.47499999999999998,"15min":0.44500000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.54500000000000004,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.5990000000000002,"5min":4.5990000000000002,"15min":4.5990000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.40100000000000002,"15min":0.377},"min":{"1min":0.23799999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":5.165,"5min":5.165,"15min":5.165},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38300000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.45000000000000001,"15min":0.40300000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":8.7959999999999994,"5min":8.7959999999999994,"15min":8.7959999999999994},"last":0.50600000000000001}]}]},{"osd":2,"up_from":122,"seq":523986010122,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35080,"kb_used_data":1844,"kb_used_omap":37,"kb_used_meta":33178,"kb_avail":94336760,"statfs":{"total":96636764160,"available":96600842240,"internally_reserved":0,"allocated":1888256,"data_stored":790029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38061,"internal_metadata":33975123},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":1,"up_from":11,"seq":47244640297,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55112,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316728,"statfs":{"total":96636764160,"available":96580329472,"internally_reserved":0,"allocated":1896448,"data_stored":790067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.32700000000000001,"15min":0.33700000000000002},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.54700000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.32100000000000001,"15min":0.32500000000000001},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.56000000000000005,"5min":0.753,"15min":0.753},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.379},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.433}]},{"osd":3,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.496,"15min":0.50700000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.60099999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.58199999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.46600000000000003,"15min":0.47399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.622,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.68300000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.496,"15min":0.504},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.747},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.47099999999999997,"15min":0.48399999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.66900000000000004}]}]},{"osd":0,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63880,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307960,"statfs":{"total":96636764160,"available":96571351040,"internally_reserved":0,"allocated":2482176,"data_stored":1380397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.26600000000000001,"5min":0.27900000000000003,"15min":0.28100000000000003},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.28000000000000003,"15min":0.28199999999999997},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":0.54800000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.84399999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.44800000000000001,"15min":0.45900000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.69899999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.442,"15min":0.44500000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.745}]},{"osd":4,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45000000000000001,"15min":0.45600000000000002},"min":{"1min":0.26300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.874,"5min":0.879,"15min":0.879},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.45000000000000001,"15min":0.45700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68500000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.88}]},{"osd":5,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46700000000000003,"15min":0.47599999999999998},"min":{"1min":0.29199999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88700000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.45500000000000002,"15min":0.45800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69799999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.77600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16703,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11738,"internal_metadata":0}]}} 2024-09-13T09:15:01.407 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:15:01.693 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:15:01.693 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":158,"fsid":"06cda68b-dd73-4266-b5e6-93ae2b3f7bd5","created":"2024-09-13T09:11:45.458147+0000","modified":"2024-09-13T09:14:55.377898+0000","last_up_change":"2024-09-13T09:14:43.297335+0000","last_in_change":"2024-09-13T09:12:42.513669+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:11:57.562930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T09:12:03.214789+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T09:12:12.120436+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":79,"snap_epoch":142,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"156c6730-f804-4806-bc2d-3a7d0856c2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6810","nonce":2268459392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6815","nonce":2268459392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6823","nonce":2268459392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6817","nonce":2268459392},{"type":"v1","addr":"172.21.15.44:6819","nonce":2268459392}]},"public_addr":"172.21.15.44:6810/2268459392","cluster_addr":"172.21.15.44:6815/2268459392","heartbeat_back_addr":"172.21.15.44:6823/2268459392","heartbeat_front_addr":"172.21.15.44:6819/2268459392","state":["exists","up"]},{"osd":1,"uuid":"00735025-3bf9-4dd0-8b42-f08f1bd1c421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6801","nonce":2209612586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6803","nonce":2209612586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6807","nonce":2209612586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2209612586},{"type":"v1","addr":"172.21.15.44:6805","nonce":2209612586}]},"public_addr":"172.21.15.44:6801/2209612586","cluster_addr":"172.21.15.44:6803/2209612586","heartbeat_back_addr":"172.21.15.44:6807/2209612586","heartbeat_front_addr":"172.21.15.44:6805/2209612586","state":["exists","up"]},{"osd":2,"uuid":"395516ac-33b6-4f43-8333-dcb1f4cded32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":84,"up_from":122,"up_thru":133,"down_at":85,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":2109106906},{"type":"v1","addr":"172.21.15.44:6811","nonce":2109106906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2109106906},{"type":"v1","addr":"172.21.15.44:6814","nonce":2109106906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2109106906},{"type":"v1","addr":"172.21.15.44:6822","nonce":2109106906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2109106906},{"type":"v1","addr":"172.21.15.44:6818","nonce":2109106906}]},"public_addr":"172.21.15.44:6811/2109106906","cluster_addr":"172.21.15.44:6814/2109106906","heartbeat_back_addr":"172.21.15.44:6822/2109106906","heartbeat_front_addr":"172.21.15.44:6818/2109106906","state":["exists","up"]},{"osd":3,"uuid":"429e877f-5462-402a-80e5-92a7b17a2eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6809","nonce":2132151186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6811","nonce":2132151186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6815","nonce":2132151186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2132151186},{"type":"v1","addr":"172.21.15.129:6813","nonce":2132151186}]},"public_addr":"172.21.15.129:6809/2132151186","cluster_addr":"172.21.15.129:6811/2132151186","heartbeat_back_addr":"172.21.15.129:6815/2132151186","heartbeat_front_addr":"172.21.15.129:6813/2132151186","state":["exists","up"]},{"osd":4,"uuid":"9b3be757-15ee-4157-80a8-0537cd1bcf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393542416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393542416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393542416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393542416},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393542416}]},"public_addr":"172.21.15.129:6801/3393542416","cluster_addr":"172.21.15.129:6803/3393542416","heartbeat_back_addr":"172.21.15.129:6807/3393542416","heartbeat_front_addr":"172.21.15.129:6805/3393542416","state":["exists","up"]},{"osd":5,"uuid":"c7601d5e-2164-45e1-ab21-368d9e3af27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":131,"up_from":146,"up_thru":146,"down_at":133,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1674281758},{"type":"v1","addr":"172.21.15.129:6817","nonce":1674281758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1674281758},{"type":"v1","addr":"172.21.15.129:6819","nonce":1674281758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1674281758},{"type":"v1","addr":"172.21.15.129:6823","nonce":1674281758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1674281758},{"type":"v1","addr":"172.21.15.129:6821","nonce":1674281758}]},"public_addr":"172.21.15.129:6817/1674281758","cluster_addr":"172.21.15.129:6819/1674281758","heartbeat_back_addr":"172.21.15.129:6823/1674281758","heartbeat_front_addr":"172.21.15.129:6821/1674281758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.230474+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.195476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.248729+0000","dead_epoch":85},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.195482+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.198746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:11:53.279957+0000","dead_epoch":133}],"pg_upmap":[{"pgid":"2.7","osds":[5,4]},{"pgid":"3.0","osds":[5,1]},{"pgid":"3.1b","osds":[2,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:15:02.706 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T09:15:02.707 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:15:02.844 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:15:02.844 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:15:02.845 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:15:02.855 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T09:15:03.136 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T09:15:03.148 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T09:15:03.149 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:15:03.272 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:15:03.272 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:15:03.272 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:15:03.283 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T09:15:03.569 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T09:15:03.582 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T09:15:03.582 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:15:03.705 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:15:03.705 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:15:03.705 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:15:03.716 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T09:15:03.999 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T09:15:04.011 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T09:15:04.012 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:15:04.136 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:15:04.136 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:15:04.136 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:15:04.147 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T09:15:04.433 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T09:15:04.446 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T09:15:04.446 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:15:04.570 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:15:04.571 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:15:04.571 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:15:04.581 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T09:15:04.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-09-13T09:15:04.884+0000 7f0a552d5640 0 --2- 172.21.15.44:0/3248769350 >> v2:172.21.15.44:3300/0 conn(0x7f0a5009e9b0 0x7f0a5009a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T09:15:05.065 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T09:15:05.078 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T09:15:05.078 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:15:05.204 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-09-13T09:15:05.204 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-13T09:15:05.204 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-09-13T09:15:05.215 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T09:15:05.505 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T09:15:05.514 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:15:05.800 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:15:05.801 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:15:05.812 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-13T09:15:05.550700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":68,"num_read":356867,"num_read_kb":1364416,"num_write":26735,"num_write_kb":362137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":666,"num_bytes_recovered":19347930,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23113,"ondisk_log_size":23113,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":310004,"kb_used_data":12232,"kb_used_omap":74,"kb_used_meta":297589,"kb_avail":565921036,"statfs":{"total":579820584960,"available":579503140864,"internally_reserved":0,"allocated":12525568,"data_stored":5918830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76719,"internal_metadata":304731217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.170038"},"pg_stats":[{"pgid":"3.1f","version":"142'857","reported_seq":9653,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375199+0000","last_change":"2024-09-13T09:14:35.254388+0000","last_active":"2024-09-13T09:14:54.375199+0000","last_peered":"2024-09-13T09:14:54.375199+0000","last_clean":"2024-09-13T09:14:54.375199+0000","last_became_active":"2024-09-13T09:12:43.534990+0000","last_became_peered":"2024-09-13T09:12:43.534990+0000","last_unstale":"2024-09-13T09:14:54.375199+0000","last_undegraded":"2024-09-13T09:14:54.375199+0000","last_fullsized":"2024-09-13T09:14:54.375199+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:45.810610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093900099999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":11993,"num_read_kb":21637,"num_write":921,"num_write_kb":7095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":848912,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"140'605","reported_seq":7894,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.381847+0000","last_change":"2024-09-13T09:14:35.256646+0000","last_active":"2024-09-13T09:14:55.381847+0000","last_peered":"2024-09-13T09:14:55.381847+0000","last_clean":"2024-09-13T09:14:55.381847+0000","last_became_active":"2024-09-13T09:14:16.749234+0000","last_became_peered":"2024-09-13T09:14:16.749234+0000","last_unstale":"2024-09-13T09:14:55.381847+0000","last_undegraded":"2024-09-13T09:14:55.381847+0000","last_fullsized":"2024-09-13T09:14:55.381847+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020237760000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10031,"num_read_kb":30858,"num_write":737,"num_write_kb":10943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":313856,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"147'1435","reported_seq":16444,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688945+0000","last_change":"2024-09-13T09:14:44.400159+0000","last_active":"2024-09-13T09:14:54.688945+0000","last_peered":"2024-09-13T09:14:54.688945+0000","last_clean":"2024-09-13T09:14:54.688945+0000","last_became_active":"2024-09-13T09:14:44.312734+0000","last_became_peered":"2024-09-13T09:14:44.312734+0000","last_unstale":"2024-09-13T09:14:54.688945+0000","last_undegraded":"2024-09-13T09:14:54.688945+0000","last_fullsized":"2024-09-13T09:14:54.688945+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1435,"log_dups_size":0,"ondisk_log_size":1435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.934438+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011136765999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":20060,"num_read_kb":34855,"num_write":1554,"num_write_kb":7371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"147'1480","reported_seq":8609,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689014+0000","last_change":"2024-09-13T09:14:44.511014+0000","last_active":"2024-09-13T09:14:54.689014+0000","last_peered":"2024-09-13T09:14:54.689014+0000","last_clean":"2024-09-13T09:14:54.689014+0000","last_became_active":"2024-09-13T09:14:44.311550+0000","last_became_peered":"2024-09-13T09:14:44.311550+0000","last_unstale":"2024-09-13T09:14:54.689014+0000","last_undegraded":"2024-09-13T09:14:54.689014+0000","last_fullsized":"2024-09-13T09:14:54.689014+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1480,"log_dups_size":0,"ondisk_log_size":1480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:16:38.500976+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.022244201000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13526,"num_read_kb":55870,"num_write":2225,"num_write_kb":20106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"142'635","reported_seq":9509,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.382075+0000","last_change":"2024-09-13T09:14:35.256035+0000","last_active":"2024-09-13T09:14:55.382075+0000","last_peered":"2024-09-13T09:14:55.382075+0000","last_clean":"2024-09-13T09:14:55.382075+0000","last_became_active":"2024-09-13T09:14:19.772731+0000","last_became_peered":"2024-09-13T09:14:19.772731+0000","last_unstale":"2024-09-13T09:14:55.382075+0000","last_undegraded":"2024-09-13T09:14:55.382075+0000","last_fullsized":"2024-09-13T09:14:55.382075+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001509144,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10475,"num_read_kb":52366,"num_write":680,"num_write_kb":12577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1411144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"141'628","reported_seq":14444,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189254+0000","last_change":"2024-09-13T09:14:35.259843+0000","last_active":"2024-09-13T09:14:55.189254+0000","last_peered":"2024-09-13T09:14:55.189254+0000","last_clean":"2024-09-13T09:14:55.189254+0000","last_became_active":"2024-09-13T09:12:13.362533+0000","last_became_peered":"2024-09-13T09:12:13.362533+0000","last_unstale":"2024-09-13T09:14:55.189254+0000","last_undegraded":"2024-09-13T09:14:55.189254+0000","last_fullsized":"2024-09-13T09:14:55.189254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:50.399315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017407639999999999,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":16059,"num_read_kb":55558,"num_write":617,"num_write_kb":11980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"138'567","reported_seq":8658,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376230+0000","last_change":"2024-09-13T09:14:48.352186+0000","last_active":"2024-09-13T09:14:54.376230+0000","last_peered":"2024-09-13T09:14:54.376230+0000","last_clean":"2024-09-13T09:14:54.376230+0000","last_became_active":"2024-09-13T09:14:48.350477+0000","last_became_peered":"2024-09-13T09:14:48.350477+0000","last_unstale":"2024-09-13T09:14:54.376230+0000","last_undegraded":"2024-09-13T09:14:54.376230+0000","last_fullsized":"2024-09-13T09:14:54.376230+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:57.759426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074163899999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8163,"num_read_kb":27760,"num_write":669,"num_write_kb":11976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"139'576","reported_seq":11072,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.381370+0000","last_change":"2024-09-13T09:14:35.255587+0000","last_active":"2024-09-13T09:14:55.381370+0000","last_peered":"2024-09-13T09:14:55.381370+0000","last_clean":"2024-09-13T09:14:55.381370+0000","last_became_active":"2024-09-13T09:14:19.772799+0000","last_became_peered":"2024-09-13T09:14:19.772799+0000","last_unstale":"2024-09-13T09:14:55.381370+0000","last_undegraded":"2024-09-13T09:14:55.381370+0000","last_fullsized":"2024-09-13T09:14:55.381370+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001235604,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13056,"num_read_kb":24986,"num_write":439,"num_write_kb":7320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":104960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"147'434","reported_seq":5711,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375290+0000","last_change":"2024-09-13T09:14:44.358128+0000","last_active":"2024-09-13T09:14:54.375290+0000","last_peered":"2024-09-13T09:14:54.375290+0000","last_clean":"2024-09-13T09:14:54.375290+0000","last_became_active":"2024-09-13T09:14:44.317071+0000","last_became_peered":"2024-09-13T09:14:44.317071+0000","last_unstale":"2024-09-13T09:14:54.375290+0000","last_undegraded":"2024-09-13T09:14:54.375290+0000","last_fullsized":"2024-09-13T09:14:54.375290+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:42.567052+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015690497000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6024,"num_read_kb":31779,"num_write":503,"num_write_kb":7984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"147'972","reported_seq":6743,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689023+0000","last_change":"2024-09-13T09:14:44.415816+0000","last_active":"2024-09-13T09:14:54.689023+0000","last_peered":"2024-09-13T09:14:54.689023+0000","last_clean":"2024-09-13T09:14:54.689023+0000","last_became_active":"2024-09-13T09:14:44.309746+0000","last_became_peered":"2024-09-13T09:14:44.309746+0000","last_unstale":"2024-09-13T09:14:54.689023+0000","last_undegraded":"2024-09-13T09:14:54.689023+0000","last_fullsized":"2024-09-13T09:14:54.689023+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":972,"log_dups_size":0,"ondisk_log_size":972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:16:58.217708+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0084025269999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8520,"num_read_kb":43161,"num_write":1440,"num_write_kb":11751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":706121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"139'872","reported_seq":20338,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189647+0000","last_change":"2024-09-13T09:14:35.256574+0000","last_active":"2024-09-13T09:14:55.189647+0000","last_peered":"2024-09-13T09:14:55.189647+0000","last_clean":"2024-09-13T09:14:55.189647+0000","last_became_active":"2024-09-13T09:12:46.250696+0000","last_became_peered":"2024-09-13T09:12:46.250696+0000","last_unstale":"2024-09-13T09:14:55.189647+0000","last_undegraded":"2024-09-13T09:14:55.189647+0000","last_fullsized":"2024-09-13T09:14:55.189647+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:37:43.821997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019682329999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":21840,"num_read_kb":55389,"num_write":1298,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1136450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"140'665","reported_seq":8589,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189530+0000","last_change":"2024-09-13T09:14:35.255745+0000","last_active":"2024-09-13T09:14:55.189530+0000","last_peered":"2024-09-13T09:14:55.189530+0000","last_clean":"2024-09-13T09:14:55.189530+0000","last_became_active":"2024-09-13T09:12:46.254016+0000","last_became_peered":"2024-09-13T09:12:46.254016+0000","last_unstale":"2024-09-13T09:14:55.189530+0000","last_undegraded":"2024-09-13T09:14:55.189530+0000","last_fullsized":"2024-09-13T09:14:55.189530+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:50.193567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013411219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10880,"num_read_kb":53346,"num_write":603,"num_write_kb":9445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1110608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688996+0000","last_change":"2024-09-13T09:14:44.316371+0000","last_active":"2024-09-13T09:14:54.688996+0000","last_peered":"2024-09-13T09:14:54.688996+0000","last_clean":"2024-09-13T09:14:54.688996+0000","last_became_active":"2024-09-13T09:14:44.316062+0000","last_became_peered":"2024-09-13T09:14:44.316062+0000","last_unstale":"2024-09-13T09:14:54.688996+0000","last_undegraded":"2024-09-13T09:14:54.688996+0000","last_fullsized":"2024-09-13T09:14:54.688996+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:06:27.687674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"142'753","reported_seq":10733,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.375022+0000","last_change":"2024-09-13T09:14:35.253996+0000","last_active":"2024-09-13T09:14:54.375022+0000","last_peered":"2024-09-13T09:14:54.375022+0000","last_clean":"2024-09-13T09:14:54.375022+0000","last_became_active":"2024-09-13T09:12:13.365496+0000","last_became_peered":"2024-09-13T09:12:13.365496+0000","last_unstale":"2024-09-13T09:14:54.375022+0000","last_undegraded":"2024-09-13T09:14:54.375022+0000","last_fullsized":"2024-09-13T09:14:54.375022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:29:49.115977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063501900000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11213,"num_read_kb":65815,"num_write":999,"num_write_kb":12170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":156,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376437+0000","last_change":"2024-09-13T09:14:48.352849+0000","last_active":"2024-09-13T09:14:54.376437+0000","last_peered":"2024-09-13T09:14:54.376437+0000","last_clean":"2024-09-13T09:14:54.376437+0000","last_became_active":"2024-09-13T09:14:48.352502+0000","last_became_peered":"2024-09-13T09:14:48.352502+0000","last_unstale":"2024-09-13T09:14:54.376437+0000","last_undegraded":"2024-09-13T09:14:54.376437+0000","last_fullsized":"2024-09-13T09:14:54.376437+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:04.532498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"141'1047","reported_seq":10559,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.382164+0000","last_change":"2024-09-13T09:14:35.256434+0000","last_active":"2024-09-13T09:14:55.382164+0000","last_peered":"2024-09-13T09:14:55.382164+0000","last_clean":"2024-09-13T09:14:55.382164+0000","last_became_active":"2024-09-13T09:14:16.752774+0000","last_became_peered":"2024-09-13T09:14:16.752774+0000","last_unstale":"2024-09-13T09:14:55.382164+0000","last_undegraded":"2024-09-13T09:14:55.382164+0000","last_fullsized":"2024-09-13T09:14:55.382164+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1047,"log_dups_size":0,"ondisk_log_size":1047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001863822,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":16931,"num_read_kb":50979,"num_write":1279,"num_write_kb":12713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":413184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":121,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189554+0000","last_change":"2024-09-13T09:12:43.533829+0000","last_active":"2024-09-13T09:14:55.189554+0000","last_peered":"2024-09-13T09:14:55.189554+0000","last_clean":"2024-09-13T09:14:55.189554+0000","last_became_active":"2024-09-13T09:12:43.533557+0000","last_became_peered":"2024-09-13T09:12:43.533557+0000","last_unstale":"2024-09-13T09:14:55.189554+0000","last_undegraded":"2024-09-13T09:14:55.189554+0000","last_fullsized":"2024-09-13T09:14:55.189554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:51.939164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"141'707","reported_seq":13355,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376484+0000","last_change":"2024-09-13T09:14:48.351345+0000","last_active":"2024-09-13T09:14:54.376484+0000","last_peered":"2024-09-13T09:14:54.376484+0000","last_clean":"2024-09-13T09:14:54.376484+0000","last_became_active":"2024-09-13T09:14:48.348909+0000","last_became_peered":"2024-09-13T09:14:48.348909+0000","last_unstale":"2024-09-13T09:14:54.376484+0000","last_undegraded":"2024-09-13T09:14:54.376484+0000","last_fullsized":"2024-09-13T09:14:54.376484+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:55:37.482616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001074057,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":15357,"num_read_kb":39900,"num_write":798,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":29184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376531+0000","last_change":"2024-09-13T09:14:48.351236+0000","last_active":"2024-09-13T09:14:54.376531+0000","last_peered":"2024-09-13T09:14:54.376531+0000","last_clean":"2024-09-13T09:14:54.376531+0000","last_became_active":"2024-09-13T09:14:48.348560+0000","last_became_peered":"2024-09-13T09:14:48.348560+0000","last_unstale":"2024-09-13T09:14:54.376531+0000","last_undegraded":"2024-09-13T09:14:54.376531+0000","last_fullsized":"2024-09-13T09:14:54.376531+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:21.050148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"142'748","reported_seq":7661,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688947+0000","last_change":"2024-09-13T09:14:44.367184+0000","last_active":"2024-09-13T09:14:54.688947+0000","last_peered":"2024-09-13T09:14:54.688947+0000","last_clean":"2024-09-13T09:14:54.688947+0000","last_became_active":"2024-09-13T09:14:44.309435+0000","last_became_peered":"2024-09-13T09:14:44.309435+0000","last_unstale":"2024-09-13T09:14:54.688947+0000","last_undegraded":"2024-09-13T09:14:54.688947+0000","last_fullsized":"2024-09-13T09:14:54.688947+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:57.739078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039806149999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10405,"num_read_kb":27276,"num_write":715,"num_write_kb":2474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":182784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"142'634","reported_seq":11576,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189327+0000","last_change":"2024-09-13T09:14:35.258893+0000","last_active":"2024-09-13T09:14:55.189327+0000","last_peered":"2024-09-13T09:14:55.189327+0000","last_clean":"2024-09-13T09:14:55.189327+0000","last_became_active":"2024-09-13T09:12:13.367770+0000","last_became_peered":"2024-09-13T09:12:13.367770+0000","last_unstale":"2024-09-13T09:14:55.189327+0000","last_undegraded":"2024-09-13T09:14:55.189327+0000","last_fullsized":"2024-09-13T09:14:55.189327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:21:50.690425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001191176,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10993,"num_read_kb":67376,"num_write":601,"num_write_kb":14012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":122,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688871+0000","last_change":"2024-09-13T09:14:44.312125+0000","last_active":"2024-09-13T09:14:54.688871+0000","last_peered":"2024-09-13T09:14:54.688871+0000","last_clean":"2024-09-13T09:14:54.688871+0000","last_became_active":"2024-09-13T09:14:44.311178+0000","last_became_peered":"2024-09-13T09:14:44.311178+0000","last_unstale":"2024-09-13T09:14:54.688871+0000","last_undegraded":"2024-09-13T09:14:54.688871+0000","last_fullsized":"2024-09-13T09:14:54.688871+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:46.245017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"147'620","reported_seq":9651,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688842+0000","last_change":"2024-09-13T09:14:44.440284+0000","last_active":"2024-09-13T09:14:54.688842+0000","last_peered":"2024-09-13T09:14:54.688842+0000","last_clean":"2024-09-13T09:14:54.688842+0000","last_became_active":"2024-09-13T09:14:44.310016+0000","last_became_peered":"2024-09-13T09:14:44.310016+0000","last_unstale":"2024-09-13T09:14:54.688842+0000","last_undegraded":"2024-09-13T09:14:54.688842+0000","last_fullsized":"2024-09-13T09:14:54.688842+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:55.802300+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.007574824,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":9521,"num_read_kb":48768,"num_write":764,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1695776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.381370+0000","last_change":"2024-09-13T09:14:16.749189+0000","last_active":"2024-09-13T09:14:55.381370+0000","last_peered":"2024-09-13T09:14:55.381370+0000","last_clean":"2024-09-13T09:14:55.381370+0000","last_became_active":"2024-09-13T09:14:16.748865+0000","last_became_peered":"2024-09-13T09:14:16.748865+0000","last_unstale":"2024-09-13T09:14:55.381370+0000","last_undegraded":"2024-09-13T09:14:55.381370+0000","last_fullsized":"2024-09-13T09:14:55.381370+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"142'332","reported_seq":3860,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.374904+0000","last_change":"2024-09-13T09:14:35.253968+0000","last_active":"2024-09-13T09:14:54.374904+0000","last_peered":"2024-09-13T09:14:54.374904+0000","last_clean":"2024-09-13T09:14:54.374904+0000","last_became_active":"2024-09-13T09:12:13.367633+0000","last_became_peered":"2024-09-13T09:12:13.367633+0000","last_unstale":"2024-09-13T09:14:54.374904+0000","last_undegraded":"2024-09-13T09:14:54.374904+0000","last_fullsized":"2024-09-13T09:14:54.374904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:34.734632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094036499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3280,"num_read_kb":26753,"num_write":312,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":121,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189164+0000","last_change":"2024-09-13T09:12:43.532201+0000","last_active":"2024-09-13T09:14:55.189164+0000","last_peered":"2024-09-13T09:14:55.189164+0000","last_clean":"2024-09-13T09:14:55.189164+0000","last_became_active":"2024-09-13T09:12:43.531942+0000","last_became_peered":"2024-09-13T09:12:43.531942+0000","last_unstale":"2024-09-13T09:14:55.189164+0000","last_undegraded":"2024-09-13T09:14:55.189164+0000","last_fullsized":"2024-09-13T09:14:55.189164+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:26.408883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"141'653","reported_seq":3816,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376444+0000","last_change":"2024-09-13T09:14:48.351027+0000","last_active":"2024-09-13T09:14:54.376444+0000","last_peered":"2024-09-13T09:14:54.376444+0000","last_clean":"2024-09-13T09:14:54.376444+0000","last_became_active":"2024-09-13T09:14:48.349138+0000","last_became_peered":"2024-09-13T09:14:48.349138+0000","last_unstale":"2024-09-13T09:14:54.376444+0000","last_undegraded":"2024-09-13T09:14:54.376444+0000","last_fullsized":"2024-09-13T09:14:54.376444+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:46.295942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039583799999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4158,"num_read_kb":20449,"num_write":848,"num_write_kb":5187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":155,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688806+0000","last_change":"2024-09-13T09:14:44.309742+0000","last_active":"2024-09-13T09:14:54.688806+0000","last_peered":"2024-09-13T09:14:54.688806+0000","last_clean":"2024-09-13T09:14:54.688806+0000","last_became_active":"2024-09-13T09:14:44.309293+0000","last_became_peered":"2024-09-13T09:14:44.309293+0000","last_unstale":"2024-09-13T09:14:54.688806+0000","last_undegraded":"2024-09-13T09:14:54.688806+0000","last_fullsized":"2024-09-13T09:14:54.688806+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:28:57.831071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":324,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189255+0000","last_change":"2024-09-13T09:12:46.248714+0000","last_active":"2024-09-13T09:14:55.189255+0000","last_peered":"2024-09-13T09:14:55.189255+0000","last_clean":"2024-09-13T09:14:55.189255+0000","last_became_active":"2024-09-13T09:12:46.248451+0000","last_became_peered":"2024-09-13T09:12:46.248451+0000","last_unstale":"2024-09-13T09:14:55.189255+0000","last_undegraded":"2024-09-13T09:14:55.189255+0000","last_fullsized":"2024-09-13T09:14:55.189255+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:58.275223+0000","last_clean_scrub_stamp":"2024-09-13T09:11:58.275223+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:52:27.713989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"141'498","reported_seq":4699,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.381459+0000","last_change":"2024-09-13T09:14:35.257020+0000","last_active":"2024-09-13T09:14:55.381459+0000","last_peered":"2024-09-13T09:14:55.381459+0000","last_clean":"2024-09-13T09:14:55.381459+0000","last_became_active":"2024-09-13T09:14:16.754685+0000","last_became_peered":"2024-09-13T09:14:16.754685+0000","last_unstale":"2024-09-13T09:14:55.381459+0000","last_undegraded":"2024-09-13T09:14:55.381459+0000","last_fullsized":"2024-09-13T09:14:55.381459+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022974409999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4733,"num_read_kb":22943,"num_write":604,"num_write_kb":15013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":45056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"142'1216","reported_seq":13783,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189219+0000","last_change":"2024-09-13T09:14:35.259629+0000","last_active":"2024-09-13T09:14:55.189219+0000","last_peered":"2024-09-13T09:14:55.189219+0000","last_clean":"2024-09-13T09:14:55.189219+0000","last_became_active":"2024-09-13T09:14:19.774300+0000","last_became_peered":"2024-09-13T09:14:19.774300+0000","last_unstale":"2024-09-13T09:14:55.189219+0000","last_undegraded":"2024-09-13T09:14:55.189219+0000","last_fullsized":"2024-09-13T09:14:55.189219+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1216,"log_dups_size":0,"ondisk_log_size":1216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:36:43.693732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001648159,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18678,"num_read_kb":75566,"num_write":1537,"num_write_kb":13034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1249280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"141'596","reported_seq":11902,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.381858+0000","last_change":"2024-09-13T09:14:35.256213+0000","last_active":"2024-09-13T09:14:55.381858+0000","last_peered":"2024-09-13T09:14:55.381858+0000","last_clean":"2024-09-13T09:14:55.381858+0000","last_became_active":"2024-09-13T09:12:46.252578+0000","last_became_peered":"2024-09-13T09:12:46.252578+0000","last_unstale":"2024-09-13T09:14:55.381858+0000","last_undegraded":"2024-09-13T09:14:55.381858+0000","last_fullsized":"2024-09-13T09:14:55.381858+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:09:03.931492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001634775,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":12679,"num_read_kb":50965,"num_write":515,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2590736,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"141'525","reported_seq":8198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.376250+0000","last_change":"2024-09-13T09:14:48.351125+0000","last_active":"2024-09-13T09:14:54.376250+0000","last_peered":"2024-09-13T09:14:54.376250+0000","last_clean":"2024-09-13T09:14:54.376250+0000","last_became_active":"2024-09-13T09:14:48.349512+0000","last_became_peered":"2024-09-13T09:14:48.349512+0000","last_unstale":"2024-09-13T09:14:54.376250+0000","last_undegraded":"2024-09-13T09:14:54.376250+0000","last_fullsized":"2024-09-13T09:14:54.376250+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:17.628884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016583220000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8119,"num_read_kb":73640,"num_write":604,"num_write_kb":21508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"147'866","reported_seq":13468,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688927+0000","last_change":"2024-09-13T09:14:44.477496+0000","last_active":"2024-09-13T09:14:54.688927+0000","last_peered":"2024-09-13T09:14:54.688927+0000","last_clean":"2024-09-13T09:14:54.688927+0000","last_became_active":"2024-09-13T09:14:44.313818+0000","last_became_peered":"2024-09-13T09:14:44.313818+0000","last_unstale":"2024-09-13T09:14:54.688927+0000","last_undegraded":"2024-09-13T09:14:54.688927+0000","last_fullsized":"2024-09-13T09:14:54.688927+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:00.720161+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.022344567999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":15285,"num_read_kb":90031,"num_write":1043,"num_write_kb":32713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"135'363","reported_seq":3877,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.382025+0000","last_change":"2024-09-13T09:14:35.255447+0000","last_active":"2024-09-13T09:14:55.382025+0000","last_peered":"2024-09-13T09:14:55.382025+0000","last_clean":"2024-09-13T09:14:55.382025+0000","last_became_active":"2024-09-13T09:14:16.751511+0000","last_became_peered":"2024-09-13T09:14:16.751511+0000","last_unstale":"2024-09-13T09:14:55.382025+0000","last_undegraded":"2024-09-13T09:14:55.382025+0000","last_fullsized":"2024-09-13T09:14:55.382025+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:57.453044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011111789999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3406,"num_read_kb":19855,"num_write":419,"num_write_kb":12151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":4294187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"147'381","reported_seq":5925,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688949+0000","last_change":"2024-09-13T09:14:44.516389+0000","last_active":"2024-09-13T09:14:54.688949+0000","last_peered":"2024-09-13T09:14:54.688949+0000","last_clean":"2024-09-13T09:14:54.688949+0000","last_became_active":"2024-09-13T09:14:44.315188+0000","last_became_peered":"2024-09-13T09:14:44.315188+0000","last_unstale":"2024-09-13T09:14:54.688949+0000","last_undegraded":"2024-09-13T09:14:54.688949+0000","last_fullsized":"2024-09-13T09:14:54.688949+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:10.043789+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0058820570000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6135,"num_read_kb":25216,"num_write":405,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"141'532","reported_seq":7039,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.689064+0000","last_change":"2024-09-13T09:14:44.426902+0000","last_active":"2024-09-13T09:14:54.689064+0000","last_peered":"2024-09-13T09:14:54.689064+0000","last_clean":"2024-09-13T09:14:54.689064+0000","last_became_active":"2024-09-13T09:14:44.309465+0000","last_became_peered":"2024-09-13T09:14:44.309465+0000","last_unstale":"2024-09-13T09:14:54.689064+0000","last_undegraded":"2024-09-13T09:14:54.689064+0000","last_fullsized":"2024-09-13T09:14:54.689064+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:19:43.291662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039480829999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6633,"num_read_kb":20805,"num_write":596,"num_write_kb":7043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":635004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"147'533","reported_seq":7949,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.688828+0000","last_change":"2024-09-13T09:14:44.375157+0000","last_active":"2024-09-13T09:14:54.688828+0000","last_peered":"2024-09-13T09:14:54.688828+0000","last_clean":"2024-09-13T09:14:54.688828+0000","last_became_active":"2024-09-13T09:14:44.308779+0000","last_became_peered":"2024-09-13T09:14:44.308779+0000","last_unstale":"2024-09-13T09:14:54.688828+0000","last_undegraded":"2024-09-13T09:14:54.688828+0000","last_fullsized":"2024-09-13T09:14:54.688828+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:00.643049+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0066958310000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7617,"num_read_kb":37352,"num_write":622,"num_write_kb":17983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"139'545","reported_seq":4837,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189301+0000","last_change":"2024-09-13T09:14:35.256957+0000","last_active":"2024-09-13T09:14:55.189301+0000","last_peered":"2024-09-13T09:14:55.189301+0000","last_clean":"2024-09-13T09:14:55.189301+0000","last_became_active":"2024-09-13T09:12:43.532504+0000","last_became_peered":"2024-09-13T09:12:43.532504+0000","last_unstale":"2024-09-13T09:14:55.189301+0000","last_undegraded":"2024-09-13T09:14:55.189301+0000","last_fullsized":"2024-09-13T09:14:55.189301+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:25:04.225563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021463989999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5509,"num_read_kb":32081,"num_write":673,"num_write_kb":7448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":694311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"141'1115","reported_seq":20789,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:54.374845+0000","last_change":"2024-09-13T09:14:35.253524+0000","last_active":"2024-09-13T09:14:54.374845+0000","last_peered":"2024-09-13T09:14:54.374845+0000","last_clean":"2024-09-13T09:14:54.374845+0000","last_became_active":"2024-09-13T09:12:46.248261+0000","last_became_peered":"2024-09-13T09:12:46.248261+0000","last_unstale":"2024-09-13T09:14:54.374845+0000","last_undegraded":"2024-09-13T09:14:54.374845+0000","last_fullsized":"2024-09-13T09:14:54.374845+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:05:44.056762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092792899999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":26946,"num_read_kb":67879,"num_write":1035,"num_write_kb":10041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":676864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"141'603","reported_seq":6565,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.189413+0000","last_change":"2024-09-13T09:14:44.326177+0000","last_active":"2024-09-13T09:14:55.189413+0000","last_peered":"2024-09-13T09:14:55.189413+0000","last_clean":"2024-09-13T09:14:55.189413+0000","last_became_active":"2024-09-13T09:14:44.314564+0000","last_became_peered":"2024-09-13T09:14:44.314564+0000","last_unstale":"2024-09-13T09:14:55.189413+0000","last_undegraded":"2024-09-13T09:14:55.189413+0000","last_fullsized":"2024-09-13T09:14:55.189413+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:58:58.520651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050931029999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8526,"num_read_kb":13102,"num_write":519,"num_write_kb":3941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":28777,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":68,"num_read":356751,"num_read_kb":1364316,"num_write":26574,"num_write_kb":359719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":18167194,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28441,"internal_metadata":0},"log_size":22993,"ondisk_log_size":22993,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":146,"seq":627065225222,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31180,"kb_used_data":1848,"kb_used_omap":31,"kb_used_meta":29280,"kb_avail":94340660,"statfs":{"total":96636764160,"available":96604835840,"internally_reserved":0,"allocated":1892352,"data_stored":790048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32283,"internal_metadata":29983205},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]}]},{"osd":4,"up_from":11,"seq":47244640298,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64968,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306872,"statfs":{"total":96636764160,"available":96570236928,"internally_reserved":0,"allocated":1892352,"data_stored":790048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39700000000000002},"min":{"1min":0.28199999999999997,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.52700000000000002,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.499},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.40200000000000002,"15min":0.39100000000000001},"min":{"1min":0.26800000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.63800000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.31,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.626},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41799999999999998,"15min":0.41999999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56799999999999995,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.53000000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53400000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.38200000000000001,"15min":0.38900000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.40799999999999997}]},{"osd":5,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.23799999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54400000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.313},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40000000000000002,"15min":0.40799999999999997},"min":{"1min":0.25,"5min":0.249,"15min":0.249},"max":{"1min":0.55400000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.34999999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640299,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59784,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312056,"statfs":{"total":96636764160,"available":96575545344,"internally_reserved":0,"allocated":2473984,"data_stored":1378241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.40600000000000003},"min":{"1min":0.21099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.47499999999999998,"15min":0.44500000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":0.47499999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.54500000000000004,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.5990000000000002,"5min":4.5990000000000002,"15min":4.5990000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.42599999999999999,"15min":0.42199999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.38600000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.44},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.45300000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.40100000000000002,"15min":0.377},"min":{"1min":0.23799999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":5.165,"5min":5.165,"15min":5.165},"last":0.32700000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38300000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.45000000000000001,"15min":0.40300000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":8.7959999999999994,"5min":8.7959999999999994,"15min":8.7959999999999994},"last":0.40799999999999997}]}]},{"osd":2,"up_from":122,"seq":523986010124,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35080,"kb_used_data":1844,"kb_used_omap":37,"kb_used_meta":33178,"kb_avail":94336760,"statfs":{"total":96636764160,"available":96600842240,"internally_reserved":0,"allocated":1888256,"data_stored":790029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38061,"internal_metadata":33975123},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]}]},{"osd":1,"up_from":11,"seq":47244640298,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55112,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316728,"statfs":{"total":96636764160,"available":96580329472,"internally_reserved":0,"allocated":1896448,"data_stored":790067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:14:57 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.317,"15min":0.33300000000000002},"min":{"1min":0.219,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.41699999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.32400000000000001},"min":{"1min":0.22600000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.59999999999999998,"5min":0.753,"15min":0.753},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.312}]},{"osd":3,"last update":"Fri Sep 13 09:14:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.5,"15min":0.50800000000000001},"min":{"1min":0.34300000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.495},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49299999999999999,"15min":0.50600000000000001},"min":{"1min":0.375,"5min":0.219,"15min":0.219},"max":{"1min":0.66400000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.45600000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:14:57 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.47099999999999997,"15min":0.47499999999999998},"min":{"1min":0.39200000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68400000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.47399999999999998,"15min":0.48199999999999998},"min":{"1min":0.35699999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.71499999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.47499999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.496,"15min":0.504},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.747},"last":0.44},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.47099999999999997,"15min":0.48399999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.42999999999999999}]}]},{"osd":0,"up_from":11,"seq":47244640297,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63880,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307960,"statfs":{"total":96636764160,"available":96571351040,"internally_reserved":0,"allocated":2482176,"data_stored":1380397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.26600000000000001,"5min":0.27900000000000003,"15min":0.28100000000000003},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.31},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.28000000000000003,"15min":0.28199999999999997},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":0.54800000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.28599999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"last":0.254},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.26600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.44800000000000001,"15min":0.45900000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.69899999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.442,"15min":0.44500000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45000000000000001,"15min":0.45600000000000002},"min":{"1min":0.26300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.874,"5min":0.879,"15min":0.879},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.45000000000000001,"15min":0.45700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68500000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46700000000000003,"15min":0.47599999999999998},"min":{"1min":0.29199999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88700000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.438,"5min":0.45500000000000002,"15min":0.45800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69799999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.64400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16703,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11738,"internal_metadata":0}]}} 2024-09-13T09:15:05.818 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.818 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.818 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.818 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.818 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.818 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.819 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.819 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.819 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.819 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.819 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.819 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.819 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-13T09:12:03.304621+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.819 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.820 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-13T09:12:03.304621+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.820 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.820 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T09:12:03.304621+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.820 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.820 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-13T09:12:03.304621+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.820 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.820 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.820 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-13T09:12:03.304621+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.820 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.821 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-13T09:12:03.304621+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.821 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.821 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T09:12:03.304621+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.821 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.821 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-13T09:12:03.304621+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.821 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T09:11:58.275223+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=11, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.821 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.821 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.821 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.822 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.822 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.822 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.822 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.822 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.822 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.822 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.822 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.822 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:05.823 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T09:15:25.824 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:15:26.124 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:15:26.125 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:15:26.136 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-09-13T09:15:25.555281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":68,"num_read":356867,"num_read_kb":1364416,"num_write":26735,"num_write_kb":362137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":666,"num_bytes_recovered":19347930,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23179,"ondisk_log_size":23179,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":309944,"kb_used_data":12232,"kb_used_omap":74,"kb_used_meta":297589,"kb_avail":565921096,"statfs":{"total":579820584960,"available":579503202304,"internally_reserved":0,"allocated":12525568,"data_stored":5918830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76719,"internal_metadata":304731217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002997"},"pg_stats":[{"pgid":"3.1f","version":"158'860","reported_seq":9668,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:11.710291+0000","last_change":"2024-09-13T09:15:11.710399+0000","last_active":"2024-09-13T09:15:11.710291+0000","last_peered":"2024-09-13T09:15:11.710291+0000","last_clean":"2024-09-13T09:15:11.710291+0000","last_became_active":"2024-09-13T09:12:43.534990+0000","last_became_peered":"2024-09-13T09:12:43.534990+0000","last_unstale":"2024-09-13T09:15:11.710291+0000","last_undegraded":"2024-09-13T09:15:11.710291+0000","last_fullsized":"2024-09-13T09:15:11.710291+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'860","last_scrub_stamp":"2024-09-13T09:15:11.710267+0000","last_deep_scrub":"158'860","last_deep_scrub_stamp":"2024-09-13T09:15:11.710267+0000","last_clean_scrub_stamp":"2024-09-13T09:15:11.710267+0000","objects_scrubbed":3,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:48:19.176034+0000","scrub_duration":0.0048191789999999998,"objects_trimmed":0,"snaptrim_duration":0.00093900099999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":11993,"num_read_kb":21637,"num_write":921,"num_write_kb":7095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":848912,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'606","reported_seq":7906,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:06.837089+0000","last_change":"2024-09-13T09:15:06.837206+0000","last_active":"2024-09-13T09:15:06.837089+0000","last_peered":"2024-09-13T09:15:06.837089+0000","last_clean":"2024-09-13T09:15:06.837089+0000","last_became_active":"2024-09-13T09:14:16.749234+0000","last_became_peered":"2024-09-13T09:14:16.749234+0000","last_unstale":"2024-09-13T09:15:06.837089+0000","last_undegraded":"2024-09-13T09:15:06.837089+0000","last_fullsized":"2024-09-13T09:15:06.837089+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'606","last_scrub_stamp":"2024-09-13T09:15:06.837063+0000","last_deep_scrub":"158'606","last_deep_scrub_stamp":"2024-09-13T09:15:06.837063+0000","last_clean_scrub_stamp":"2024-09-13T09:15:06.837063+0000","objects_scrubbed":1,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:15:46.209884+0000","scrub_duration":0.003657099,"objects_trimmed":0,"snaptrim_duration":0.0020237760000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10031,"num_read_kb":30858,"num_write":737,"num_write_kb":10943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":313856,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"147'1435","reported_seq":16445,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:56.031384+0000","last_change":"2024-09-13T09:14:44.400159+0000","last_active":"2024-09-13T09:14:56.031384+0000","last_peered":"2024-09-13T09:14:56.031384+0000","last_clean":"2024-09-13T09:14:56.031384+0000","last_became_active":"2024-09-13T09:14:44.312734+0000","last_became_peered":"2024-09-13T09:14:44.312734+0000","last_unstale":"2024-09-13T09:14:56.031384+0000","last_undegraded":"2024-09-13T09:14:56.031384+0000","last_fullsized":"2024-09-13T09:14:56.031384+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":1435,"log_dups_size":0,"ondisk_log_size":1435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011136765999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":20060,"num_read_kb":34855,"num_write":1554,"num_write_kb":7371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"158'1481","reported_seq":8622,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:12.953879+0000","last_change":"2024-09-13T09:15:12.953998+0000","last_active":"2024-09-13T09:15:12.953879+0000","last_peered":"2024-09-13T09:15:12.953879+0000","last_clean":"2024-09-13T09:15:12.953879+0000","last_became_active":"2024-09-13T09:14:44.311550+0000","last_became_peered":"2024-09-13T09:14:44.311550+0000","last_unstale":"2024-09-13T09:15:12.953879+0000","last_undegraded":"2024-09-13T09:15:12.953879+0000","last_fullsized":"2024-09-13T09:15:12.953879+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1481","last_scrub_stamp":"2024-09-13T09:15:12.953853+0000","last_deep_scrub":"158'1481","last_deep_scrub_stamp":"2024-09-13T09:15:12.953853+0000","last_clean_scrub_stamp":"2024-09-13T09:15:12.953853+0000","objects_scrubbed":1,"log_size":1481,"log_dups_size":0,"ondisk_log_size":1481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:25.555906+0000","scrub_duration":0.0034116120000000001,"objects_trimmed":6,"snaptrim_duration":0.022244201000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13526,"num_read_kb":55870,"num_write":2225,"num_write_kb":20106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"142'635","reported_seq":9520,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:17.768893+0000","last_change":"2024-09-13T09:15:17.769007+0000","last_active":"2024-09-13T09:15:17.768893+0000","last_peered":"2024-09-13T09:15:17.768893+0000","last_clean":"2024-09-13T09:15:17.768893+0000","last_became_active":"2024-09-13T09:14:19.772731+0000","last_became_peered":"2024-09-13T09:14:19.772731+0000","last_unstale":"2024-09-13T09:15:17.768893+0000","last_undegraded":"2024-09-13T09:15:17.768893+0000","last_fullsized":"2024-09-13T09:15:17.768893+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'635","last_scrub_stamp":"2024-09-13T09:15:17.768869+0000","last_deep_scrub":"142'635","last_deep_scrub_stamp":"2024-09-13T09:15:17.768869+0000","last_clean_scrub_stamp":"2024-09-13T09:15:17.768869+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:32.891266+0000","scrub_duration":0.0023082340000000002,"objects_trimmed":0,"snaptrim_duration":0.001509144,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10475,"num_read_kb":52366,"num_write":680,"num_write_kb":12577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1411144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"158'631","reported_seq":14459,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:12.631957+0000","last_change":"2024-09-13T09:15:12.632069+0000","last_active":"2024-09-13T09:15:12.631957+0000","last_peered":"2024-09-13T09:15:12.631957+0000","last_clean":"2024-09-13T09:15:12.631957+0000","last_became_active":"2024-09-13T09:12:13.362533+0000","last_became_peered":"2024-09-13T09:12:13.362533+0000","last_unstale":"2024-09-13T09:15:12.631957+0000","last_undegraded":"2024-09-13T09:15:12.631957+0000","last_fullsized":"2024-09-13T09:15:12.631957+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'631","last_scrub_stamp":"2024-09-13T09:15:12.631934+0000","last_deep_scrub":"158'631","last_deep_scrub_stamp":"2024-09-13T09:15:12.631934+0000","last_clean_scrub_stamp":"2024-09-13T09:15:12.631934+0000","objects_scrubbed":3,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:22.761296+0000","scrub_duration":0.0051449820000000002,"objects_trimmed":0,"snaptrim_duration":0.0017407639999999999,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":16059,"num_read_kb":55558,"num_write":617,"num_write_kb":11980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"158'570","reported_seq":8673,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:17.351227+0000","last_change":"2024-09-13T09:15:17.351340+0000","last_active":"2024-09-13T09:15:17.351227+0000","last_peered":"2024-09-13T09:15:17.351227+0000","last_clean":"2024-09-13T09:15:17.351227+0000","last_became_active":"2024-09-13T09:14:48.350477+0000","last_became_peered":"2024-09-13T09:14:48.350477+0000","last_unstale":"2024-09-13T09:15:17.351227+0000","last_undegraded":"2024-09-13T09:15:17.351227+0000","last_fullsized":"2024-09-13T09:15:17.351227+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'570","last_scrub_stamp":"2024-09-13T09:15:17.351207+0000","last_deep_scrub":"158'570","last_deep_scrub_stamp":"2024-09-13T09:15:17.351207+0000","last_clean_scrub_stamp":"2024-09-13T09:15:17.351207+0000","objects_scrubbed":3,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:07.107808+0000","scrub_duration":0.0044970440000000004,"objects_trimmed":0,"snaptrim_duration":0.00074163899999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8163,"num_read_kb":27760,"num_write":669,"num_write_kb":11976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"158'581","reported_seq":11088,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:19.851296+0000","last_change":"2024-09-13T09:15:19.851443+0000","last_active":"2024-09-13T09:15:19.851296+0000","last_peered":"2024-09-13T09:15:19.851296+0000","last_clean":"2024-09-13T09:15:19.851296+0000","last_became_active":"2024-09-13T09:14:19.772799+0000","last_became_peered":"2024-09-13T09:14:19.772799+0000","last_unstale":"2024-09-13T09:15:19.851296+0000","last_undegraded":"2024-09-13T09:15:19.851296+0000","last_fullsized":"2024-09-13T09:15:19.851296+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'581","last_scrub_stamp":"2024-09-13T09:15:19.851268+0000","last_deep_scrub":"158'581","last_deep_scrub_stamp":"2024-09-13T09:15:19.851268+0000","last_clean_scrub_stamp":"2024-09-13T09:15:19.851268+0000","objects_scrubbed":5,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:51:55.388636+0000","scrub_duration":0.0064302480000000004,"objects_trimmed":0,"snaptrim_duration":0.001235604,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13056,"num_read_kb":24986,"num_write":439,"num_write_kb":7320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":104960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"147'434","reported_seq":5723,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:09.748910+0000","last_change":"2024-09-13T09:15:09.749041+0000","last_active":"2024-09-13T09:15:09.748910+0000","last_peered":"2024-09-13T09:15:09.748910+0000","last_clean":"2024-09-13T09:15:09.748910+0000","last_became_active":"2024-09-13T09:14:44.317071+0000","last_became_peered":"2024-09-13T09:14:44.317071+0000","last_unstale":"2024-09-13T09:15:09.748910+0000","last_undegraded":"2024-09-13T09:15:09.748910+0000","last_fullsized":"2024-09-13T09:15:09.748910+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'434","last_scrub_stamp":"2024-09-13T09:15:09.748886+0000","last_deep_scrub":"147'434","last_deep_scrub_stamp":"2024-09-13T09:15:09.748886+0000","last_clean_scrub_stamp":"2024-09-13T09:15:09.748886+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:28:03.194738+0000","scrub_duration":0.0021969480000000002,"objects_trimmed":4,"snaptrim_duration":0.015690497000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6024,"num_read_kb":31779,"num_write":503,"num_write_kb":7984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"158'975","reported_seq":6758,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:08.940009+0000","last_change":"2024-09-13T09:15:08.940139+0000","last_active":"2024-09-13T09:15:08.940009+0000","last_peered":"2024-09-13T09:15:08.940009+0000","last_clean":"2024-09-13T09:15:08.940009+0000","last_became_active":"2024-09-13T09:14:44.309746+0000","last_became_peered":"2024-09-13T09:14:44.309746+0000","last_unstale":"2024-09-13T09:15:08.940009+0000","last_undegraded":"2024-09-13T09:15:08.940009+0000","last_fullsized":"2024-09-13T09:15:08.940009+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'975","last_scrub_stamp":"2024-09-13T09:15:08.939985+0000","last_deep_scrub":"158'975","last_deep_scrub_stamp":"2024-09-13T09:15:08.939985+0000","last_clean_scrub_stamp":"2024-09-13T09:15:08.939985+0000","objects_scrubbed":3,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:14:34.542109+0000","scrub_duration":0.0084448349999999995,"objects_trimmed":2,"snaptrim_duration":0.0084025269999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8520,"num_read_kb":43161,"num_write":1440,"num_write_kb":11751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":706121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"139'872","reported_seq":20340,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:04.432930+0000","last_change":"2024-09-13T09:14:35.256574+0000","last_active":"2024-09-13T09:15:04.432930+0000","last_peered":"2024-09-13T09:15:04.432930+0000","last_clean":"2024-09-13T09:15:04.432930+0000","last_became_active":"2024-09-13T09:12:46.250696+0000","last_became_peered":"2024-09-13T09:12:46.250696+0000","last_unstale":"2024-09-13T09:15:04.432930+0000","last_undegraded":"2024-09-13T09:15:04.432930+0000","last_fullsized":"2024-09-13T09:15:04.432930+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019682329999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":21840,"num_read_kb":55389,"num_write":1298,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1136450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"158'667","reported_seq":8604,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:19.354352+0000","last_change":"2024-09-13T09:15:19.354480+0000","last_active":"2024-09-13T09:15:19.354352+0000","last_peered":"2024-09-13T09:15:19.354352+0000","last_clean":"2024-09-13T09:15:19.354352+0000","last_became_active":"2024-09-13T09:12:46.254016+0000","last_became_peered":"2024-09-13T09:12:46.254016+0000","last_unstale":"2024-09-13T09:15:19.354352+0000","last_undegraded":"2024-09-13T09:15:19.354352+0000","last_fullsized":"2024-09-13T09:15:19.354352+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'667","last_scrub_stamp":"2024-09-13T09:15:19.354327+0000","last_deep_scrub":"158'667","last_deep_scrub_stamp":"2024-09-13T09:15:19.354327+0000","last_clean_scrub_stamp":"2024-09-13T09:15:19.354327+0000","objects_scrubbed":2,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:44.282864+0000","scrub_duration":0.0039345300000000003,"objects_trimmed":0,"snaptrim_duration":0.0013411219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10880,"num_read_kb":53346,"num_write":603,"num_write_kb":9445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1110608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:56.031263+0000","last_change":"2024-09-13T09:14:44.316371+0000","last_active":"2024-09-13T09:14:56.031263+0000","last_peered":"2024-09-13T09:14:56.031263+0000","last_clean":"2024-09-13T09:14:56.031263+0000","last_became_active":"2024-09-13T09:14:44.316062+0000","last_became_peered":"2024-09-13T09:14:44.316062+0000","last_unstale":"2024-09-13T09:14:56.031263+0000","last_undegraded":"2024-09-13T09:14:56.031263+0000","last_fullsized":"2024-09-13T09:14:56.031263+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"158'755","reported_seq":10747,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:07.717807+0000","last_change":"2024-09-13T09:15:07.717952+0000","last_active":"2024-09-13T09:15:07.717807+0000","last_peered":"2024-09-13T09:15:07.717807+0000","last_clean":"2024-09-13T09:15:07.717807+0000","last_became_active":"2024-09-13T09:12:13.365496+0000","last_became_peered":"2024-09-13T09:12:13.365496+0000","last_unstale":"2024-09-13T09:15:07.717807+0000","last_undegraded":"2024-09-13T09:15:07.717807+0000","last_fullsized":"2024-09-13T09:15:07.717807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'755","last_scrub_stamp":"2024-09-13T09:15:07.717786+0000","last_deep_scrub":"158'755","last_deep_scrub_stamp":"2024-09-13T09:15:07.717786+0000","last_clean_scrub_stamp":"2024-09-13T09:15:07.717786+0000","objects_scrubbed":2,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:26:11.704168+0000","scrub_duration":0.0042564270000000001,"objects_trimmed":0,"snaptrim_duration":0.00063501900000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11213,"num_read_kb":65815,"num_write":999,"num_write_kb":12170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:09.390649+0000","last_change":"2024-09-13T09:15:09.390791+0000","last_active":"2024-09-13T09:15:09.390649+0000","last_peered":"2024-09-13T09:15:09.390649+0000","last_clean":"2024-09-13T09:15:09.390649+0000","last_became_active":"2024-09-13T09:14:48.352502+0000","last_became_peered":"2024-09-13T09:14:48.352502+0000","last_unstale":"2024-09-13T09:15:09.390649+0000","last_undegraded":"2024-09-13T09:15:09.390649+0000","last_fullsized":"2024-09-13T09:15:09.390649+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:15:09.390608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:15:09.390608+0000","last_clean_scrub_stamp":"2024-09-13T09:15:09.390608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:00.477020+0000","scrub_duration":0.001921126,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"158'1049","reported_seq":10572,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:11.788249+0000","last_change":"2024-09-13T09:15:11.788378+0000","last_active":"2024-09-13T09:15:11.788249+0000","last_peered":"2024-09-13T09:15:11.788249+0000","last_clean":"2024-09-13T09:15:11.788249+0000","last_became_active":"2024-09-13T09:14:16.752774+0000","last_became_peered":"2024-09-13T09:14:16.752774+0000","last_unstale":"2024-09-13T09:15:11.788249+0000","last_undegraded":"2024-09-13T09:15:11.788249+0000","last_fullsized":"2024-09-13T09:15:11.788249+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1049","last_scrub_stamp":"2024-09-13T09:15:11.788225+0000","last_deep_scrub":"158'1049","last_deep_scrub_stamp":"2024-09-13T09:15:11.788225+0000","last_clean_scrub_stamp":"2024-09-13T09:15:11.788225+0000","objects_scrubbed":2,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:01.052086+0000","scrub_duration":0.0039185440000000004,"objects_trimmed":0,"snaptrim_duration":0.001863822,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":16931,"num_read_kb":50979,"num_write":1279,"num_write_kb":12713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":413184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":122,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:55.382015+0000","last_change":"2024-09-13T09:12:43.533829+0000","last_active":"2024-09-13T09:14:55.382015+0000","last_peered":"2024-09-13T09:14:55.382015+0000","last_clean":"2024-09-13T09:14:55.382015+0000","last_became_active":"2024-09-13T09:12:43.533557+0000","last_became_peered":"2024-09-13T09:12:43.533557+0000","last_unstale":"2024-09-13T09:14:55.382015+0000","last_undegraded":"2024-09-13T09:14:55.382015+0000","last_fullsized":"2024-09-13T09:14:55.382015+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"158'709","reported_seq":13369,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:15.412540+0000","last_change":"2024-09-13T09:15:15.412644+0000","last_active":"2024-09-13T09:15:15.412540+0000","last_peered":"2024-09-13T09:15:15.412540+0000","last_clean":"2024-09-13T09:15:15.412540+0000","last_became_active":"2024-09-13T09:14:48.348909+0000","last_became_peered":"2024-09-13T09:14:48.348909+0000","last_unstale":"2024-09-13T09:15:15.412540+0000","last_undegraded":"2024-09-13T09:15:15.412540+0000","last_fullsized":"2024-09-13T09:15:15.412540+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'709","last_scrub_stamp":"2024-09-13T09:15:15.412515+0000","last_deep_scrub":"158'709","last_deep_scrub_stamp":"2024-09-13T09:15:15.412515+0000","last_clean_scrub_stamp":"2024-09-13T09:15:15.412515+0000","objects_scrubbed":2,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:13.817613+0000","scrub_duration":0.0039132079999999996,"objects_trimmed":0,"snaptrim_duration":0.001074057,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":15357,"num_read_kb":39900,"num_write":798,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":29184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":152,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:20.333802+0000","last_change":"2024-09-13T09:15:20.333918+0000","last_active":"2024-09-13T09:15:20.333802+0000","last_peered":"2024-09-13T09:15:20.333802+0000","last_clean":"2024-09-13T09:15:20.333802+0000","last_became_active":"2024-09-13T09:14:48.348560+0000","last_became_peered":"2024-09-13T09:14:48.348560+0000","last_unstale":"2024-09-13T09:15:20.333802+0000","last_undegraded":"2024-09-13T09:15:20.333802+0000","last_fullsized":"2024-09-13T09:15:20.333802+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:15:20.333781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:15:20.333781+0000","last_clean_scrub_stamp":"2024-09-13T09:15:20.333781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:06:50.801030+0000","scrub_duration":0.001463233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'750","reported_seq":7675,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:13.998121+0000","last_change":"2024-09-13T09:15:13.998227+0000","last_active":"2024-09-13T09:15:13.998121+0000","last_peered":"2024-09-13T09:15:13.998121+0000","last_clean":"2024-09-13T09:15:13.998121+0000","last_became_active":"2024-09-13T09:14:44.309435+0000","last_became_peered":"2024-09-13T09:14:44.309435+0000","last_unstale":"2024-09-13T09:15:13.998121+0000","last_undegraded":"2024-09-13T09:15:13.998121+0000","last_fullsized":"2024-09-13T09:15:13.998121+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'750","last_scrub_stamp":"2024-09-13T09:15:13.998097+0000","last_deep_scrub":"158'750","last_deep_scrub_stamp":"2024-09-13T09:15:13.998097+0000","last_clean_scrub_stamp":"2024-09-13T09:15:13.998097+0000","objects_scrubbed":2,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:35:18.377733+0000","scrub_duration":0.0043266329999999999,"objects_trimmed":0,"snaptrim_duration":0.0039806149999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10405,"num_read_kb":27276,"num_write":715,"num_write_kb":2474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":182784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"158'637","reported_seq":11591,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:11.616512+0000","last_change":"2024-09-13T09:15:11.616642+0000","last_active":"2024-09-13T09:15:11.616512+0000","last_peered":"2024-09-13T09:15:11.616512+0000","last_clean":"2024-09-13T09:15:11.616512+0000","last_became_active":"2024-09-13T09:12:13.367770+0000","last_became_peered":"2024-09-13T09:12:13.367770+0000","last_unstale":"2024-09-13T09:15:11.616512+0000","last_undegraded":"2024-09-13T09:15:11.616512+0000","last_fullsized":"2024-09-13T09:15:11.616512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'637","last_scrub_stamp":"2024-09-13T09:15:11.616471+0000","last_deep_scrub":"158'637","last_deep_scrub_stamp":"2024-09-13T09:15:11.616471+0000","last_clean_scrub_stamp":"2024-09-13T09:15:11.616471+0000","objects_scrubbed":3,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:55.709145+0000","scrub_duration":0.0053358210000000001,"objects_trimmed":0,"snaptrim_duration":0.001191176,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10993,"num_read_kb":67376,"num_write":601,"num_write_kb":14012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":123,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:56.031043+0000","last_change":"2024-09-13T09:14:44.312125+0000","last_active":"2024-09-13T09:14:56.031043+0000","last_peered":"2024-09-13T09:14:56.031043+0000","last_clean":"2024-09-13T09:14:56.031043+0000","last_became_active":"2024-09-13T09:14:44.311178+0000","last_became_peered":"2024-09-13T09:14:44.311178+0000","last_unstale":"2024-09-13T09:14:56.031043+0000","last_undegraded":"2024-09-13T09:14:56.031043+0000","last_fullsized":"2024-09-13T09:14:56.031043+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:03.304621+0000","last_clean_scrub_stamp":"2024-09-13T09:12:03.304621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"158'623","reported_seq":9666,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:05.908691+0000","last_change":"2024-09-13T09:15:05.908843+0000","last_active":"2024-09-13T09:15:05.908691+0000","last_peered":"2024-09-13T09:15:05.908691+0000","last_clean":"2024-09-13T09:15:05.908691+0000","last_became_active":"2024-09-13T09:14:44.310016+0000","last_became_peered":"2024-09-13T09:14:44.310016+0000","last_unstale":"2024-09-13T09:15:05.908691+0000","last_undegraded":"2024-09-13T09:15:05.908691+0000","last_fullsized":"2024-09-13T09:15:05.908691+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'623","last_scrub_stamp":"2024-09-13T09:15:05.908668+0000","last_deep_scrub":"158'623","last_deep_scrub_stamp":"2024-09-13T09:15:05.908668+0000","last_clean_scrub_stamp":"2024-09-13T09:15:05.908668+0000","objects_scrubbed":3,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:48:30.181439+0000","scrub_duration":0.0067150650000000001,"objects_trimmed":6,"snaptrim_duration":0.007574824,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":9521,"num_read_kb":48768,"num_write":764,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1695776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":118,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:16.793937+0000","last_change":"2024-09-13T09:15:16.794049+0000","last_active":"2024-09-13T09:15:16.793937+0000","last_peered":"2024-09-13T09:15:16.793937+0000","last_clean":"2024-09-13T09:15:16.793937+0000","last_became_active":"2024-09-13T09:14:16.748865+0000","last_became_peered":"2024-09-13T09:14:16.748865+0000","last_unstale":"2024-09-13T09:15:16.793937+0000","last_undegraded":"2024-09-13T09:15:16.793937+0000","last_fullsized":"2024-09-13T09:15:16.793937+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:15:16.793915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:15:16.793915+0000","last_clean_scrub_stamp":"2024-09-13T09:15:16.793915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:47.978876+0000","scrub_duration":0.001464545,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"158'333","reported_seq":3873,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:03.727755+0000","last_change":"2024-09-13T09:15:03.727892+0000","last_active":"2024-09-13T09:15:03.727755+0000","last_peered":"2024-09-13T09:15:03.727755+0000","last_clean":"2024-09-13T09:15:03.727755+0000","last_became_active":"2024-09-13T09:12:13.367633+0000","last_became_peered":"2024-09-13T09:12:13.367633+0000","last_unstale":"2024-09-13T09:15:03.727755+0000","last_undegraded":"2024-09-13T09:15:03.727755+0000","last_fullsized":"2024-09-13T09:15:03.727755+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'333","last_scrub_stamp":"2024-09-13T09:15:03.727710+0000","last_deep_scrub":"158'333","last_deep_scrub_stamp":"2024-09-13T09:15:03.727710+0000","last_clean_scrub_stamp":"2024-09-13T09:15:03.727710+0000","objects_scrubbed":1,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:54.610562+0000","scrub_duration":0.003663763,"objects_trimmed":0,"snaptrim_duration":0.00094036499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3280,"num_read_kb":26753,"num_write":312,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:11.318925+0000","last_change":"2024-09-13T09:15:11.319019+0000","last_active":"2024-09-13T09:15:11.318925+0000","last_peered":"2024-09-13T09:15:11.318925+0000","last_clean":"2024-09-13T09:15:11.318925+0000","last_became_active":"2024-09-13T09:12:43.531942+0000","last_became_peered":"2024-09-13T09:12:43.531942+0000","last_unstale":"2024-09-13T09:15:11.318925+0000","last_undegraded":"2024-09-13T09:15:11.318925+0000","last_fullsized":"2024-09-13T09:15:11.318925+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:15:11.318900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:15:11.318900+0000","last_clean_scrub_stamp":"2024-09-13T09:15:11.318900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:02.547367+0000","scrub_duration":0.0019515100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"158'656","reported_seq":3831,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:21.366714+0000","last_change":"2024-09-13T09:15:21.366849+0000","last_active":"2024-09-13T09:15:21.366714+0000","last_peered":"2024-09-13T09:15:21.366714+0000","last_clean":"2024-09-13T09:15:21.366714+0000","last_became_active":"2024-09-13T09:14:48.349138+0000","last_became_peered":"2024-09-13T09:14:48.349138+0000","last_unstale":"2024-09-13T09:15:21.366714+0000","last_undegraded":"2024-09-13T09:15:21.366714+0000","last_fullsized":"2024-09-13T09:15:21.366714+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'656","last_scrub_stamp":"2024-09-13T09:15:21.366692+0000","last_deep_scrub":"158'656","last_deep_scrub_stamp":"2024-09-13T09:15:21.366692+0000","last_clean_scrub_stamp":"2024-09-13T09:15:21.366692+0000","objects_scrubbed":3,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:39.559245+0000","scrub_duration":0.007211767,"objects_trimmed":0,"snaptrim_duration":0.00039583799999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4158,"num_read_kb":20449,"num_write":848,"num_write_kb":5187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'2","reported_seq":168,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:15.944248+0000","last_change":"2024-09-13T09:15:15.944364+0000","last_active":"2024-09-13T09:15:15.944248+0000","last_peered":"2024-09-13T09:15:15.944248+0000","last_clean":"2024-09-13T09:15:15.944248+0000","last_became_active":"2024-09-13T09:14:44.309293+0000","last_became_peered":"2024-09-13T09:14:44.309293+0000","last_unstale":"2024-09-13T09:15:15.944248+0000","last_undegraded":"2024-09-13T09:15:15.944248+0000","last_fullsized":"2024-09-13T09:15:15.944248+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'2","last_scrub_stamp":"2024-09-13T09:15:15.944225+0000","last_deep_scrub":"158'2","last_deep_scrub_stamp":"2024-09-13T09:15:15.944225+0000","last_clean_scrub_stamp":"2024-09-13T09:15:15.944225+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:59.264942+0000","scrub_duration":0.0035584509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"158'119","reported_seq":338,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:09.242237+0000","last_change":"2024-09-13T09:15:09.242356+0000","last_active":"2024-09-13T09:15:09.242237+0000","last_peered":"2024-09-13T09:15:09.242237+0000","last_clean":"2024-09-13T09:15:09.242237+0000","last_became_active":"2024-09-13T09:12:46.248451+0000","last_became_peered":"2024-09-13T09:12:46.248451+0000","last_unstale":"2024-09-13T09:15:09.242237+0000","last_undegraded":"2024-09-13T09:15:09.242237+0000","last_fullsized":"2024-09-13T09:15:09.242237+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'119","last_scrub_stamp":"2024-09-13T09:15:09.242213+0000","last_deep_scrub":"158'119","last_deep_scrub_stamp":"2024-09-13T09:15:09.242213+0000","last_clean_scrub_stamp":"2024-09-13T09:15:09.242213+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:49:57.189752+0000","scrub_duration":0.0047912179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"158'501","reported_seq":4713,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:10.817551+0000","last_change":"2024-09-13T09:15:10.817665+0000","last_active":"2024-09-13T09:15:10.817551+0000","last_peered":"2024-09-13T09:15:10.817551+0000","last_clean":"2024-09-13T09:15:10.817551+0000","last_became_active":"2024-09-13T09:14:16.754685+0000","last_became_peered":"2024-09-13T09:14:16.754685+0000","last_unstale":"2024-09-13T09:15:10.817551+0000","last_undegraded":"2024-09-13T09:15:10.817551+0000","last_fullsized":"2024-09-13T09:15:10.817551+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'501","last_scrub_stamp":"2024-09-13T09:15:10.817525+0000","last_deep_scrub":"158'501","last_deep_scrub_stamp":"2024-09-13T09:15:10.817525+0000","last_clean_scrub_stamp":"2024-09-13T09:15:10.817525+0000","objects_scrubbed":3,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:57:30.466409+0000","scrub_duration":0.0041960280000000001,"objects_trimmed":0,"snaptrim_duration":0.0022974409999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4733,"num_read_kb":22943,"num_write":604,"num_write_kb":15013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":45056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"158'1220","reported_seq":13799,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:06.576292+0000","last_change":"2024-09-13T09:15:06.576429+0000","last_active":"2024-09-13T09:15:06.576292+0000","last_peered":"2024-09-13T09:15:06.576292+0000","last_clean":"2024-09-13T09:15:06.576292+0000","last_became_active":"2024-09-13T09:14:19.774300+0000","last_became_peered":"2024-09-13T09:14:19.774300+0000","last_unstale":"2024-09-13T09:15:06.576292+0000","last_undegraded":"2024-09-13T09:15:06.576292+0000","last_fullsized":"2024-09-13T09:15:06.576292+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1220","last_scrub_stamp":"2024-09-13T09:15:06.576269+0000","last_deep_scrub":"158'1220","last_deep_scrub_stamp":"2024-09-13T09:15:06.576269+0000","last_clean_scrub_stamp":"2024-09-13T09:15:06.576269+0000","objects_scrubbed":4,"log_size":1220,"log_dups_size":0,"ondisk_log_size":1220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:28:51.559616+0000","scrub_duration":0.0065206120000000003,"objects_trimmed":0,"snaptrim_duration":0.001648159,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18678,"num_read_kb":75566,"num_write":1537,"num_write_kb":13034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1249280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"158'598","reported_seq":11916,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:13.281747+0000","last_change":"2024-09-13T09:15:13.281873+0000","last_active":"2024-09-13T09:15:13.281747+0000","last_peered":"2024-09-13T09:15:13.281747+0000","last_clean":"2024-09-13T09:15:13.281747+0000","last_became_active":"2024-09-13T09:12:46.252578+0000","last_became_peered":"2024-09-13T09:12:46.252578+0000","last_unstale":"2024-09-13T09:15:13.281747+0000","last_undegraded":"2024-09-13T09:15:13.281747+0000","last_fullsized":"2024-09-13T09:15:13.281747+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'598","last_scrub_stamp":"2024-09-13T09:15:13.281723+0000","last_deep_scrub":"158'598","last_deep_scrub_stamp":"2024-09-13T09:15:13.281723+0000","last_clean_scrub_stamp":"2024-09-13T09:15:13.281723+0000","objects_scrubbed":2,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:28.279782+0000","scrub_duration":0.0035929460000000001,"objects_trimmed":0,"snaptrim_duration":0.001634775,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":12679,"num_read_kb":50965,"num_write":515,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2590736,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'527","reported_seq":8212,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:16.380252+0000","last_change":"2024-09-13T09:15:16.380375+0000","last_active":"2024-09-13T09:15:16.380252+0000","last_peered":"2024-09-13T09:15:16.380252+0000","last_clean":"2024-09-13T09:15:16.380252+0000","last_became_active":"2024-09-13T09:14:48.349512+0000","last_became_peered":"2024-09-13T09:14:48.349512+0000","last_unstale":"2024-09-13T09:15:16.380252+0000","last_undegraded":"2024-09-13T09:15:16.380252+0000","last_fullsized":"2024-09-13T09:15:16.380252+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'527","last_scrub_stamp":"2024-09-13T09:15:16.380228+0000","last_deep_scrub":"158'527","last_deep_scrub_stamp":"2024-09-13T09:15:16.380228+0000","last_clean_scrub_stamp":"2024-09-13T09:15:16.380228+0000","objects_scrubbed":2,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:17.836080+0000","scrub_duration":0.0037333000000000002,"objects_trimmed":0,"snaptrim_duration":0.0016583220000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8119,"num_read_kb":73640,"num_write":604,"num_write_kb":21508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"158'867","reported_seq":13481,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:16.968884+0000","last_change":"2024-09-13T09:15:16.969037+0000","last_active":"2024-09-13T09:15:16.968884+0000","last_peered":"2024-09-13T09:15:16.968884+0000","last_clean":"2024-09-13T09:15:16.968884+0000","last_became_active":"2024-09-13T09:14:44.313818+0000","last_became_peered":"2024-09-13T09:14:44.313818+0000","last_unstale":"2024-09-13T09:15:16.968884+0000","last_undegraded":"2024-09-13T09:15:16.968884+0000","last_fullsized":"2024-09-13T09:15:16.968884+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'867","last_scrub_stamp":"2024-09-13T09:15:16.968828+0000","last_deep_scrub":"158'867","last_deep_scrub_stamp":"2024-09-13T09:15:16.968828+0000","last_clean_scrub_stamp":"2024-09-13T09:15:16.968828+0000","objects_scrubbed":1,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:30.960784+0000","scrub_duration":0.0040212370000000004,"objects_trimmed":8,"snaptrim_duration":0.022344567999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":15285,"num_read_kb":90031,"num_write":1043,"num_write_kb":32713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"135'363","reported_seq":3878,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:01.555460+0000","last_change":"2024-09-13T09:14:35.255447+0000","last_active":"2024-09-13T09:15:01.555460+0000","last_peered":"2024-09-13T09:15:01.555460+0000","last_clean":"2024-09-13T09:15:01.555460+0000","last_became_active":"2024-09-13T09:14:16.751511+0000","last_became_peered":"2024-09-13T09:14:16.751511+0000","last_unstale":"2024-09-13T09:15:01.555460+0000","last_undegraded":"2024-09-13T09:15:01.555460+0000","last_fullsized":"2024-09-13T09:15:01.555460+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011111789999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3406,"num_read_kb":19855,"num_write":419,"num_write_kb":12151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":4294187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"158'382","reported_seq":5938,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:11.935243+0000","last_change":"2024-09-13T09:15:11.935386+0000","last_active":"2024-09-13T09:15:11.935243+0000","last_peered":"2024-09-13T09:15:11.935243+0000","last_clean":"2024-09-13T09:15:11.935243+0000","last_became_active":"2024-09-13T09:14:44.315188+0000","last_became_peered":"2024-09-13T09:14:44.315188+0000","last_unstale":"2024-09-13T09:15:11.935243+0000","last_undegraded":"2024-09-13T09:15:11.935243+0000","last_fullsized":"2024-09-13T09:15:11.935243+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'382","last_scrub_stamp":"2024-09-13T09:15:11.935201+0000","last_deep_scrub":"158'382","last_deep_scrub_stamp":"2024-09-13T09:15:11.935201+0000","last_clean_scrub_stamp":"2024-09-13T09:15:11.935201+0000","objects_scrubbed":1,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:02:29.994265+0000","scrub_duration":0.0028760159999999999,"objects_trimmed":2,"snaptrim_duration":0.0058820570000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6135,"num_read_kb":25216,"num_write":405,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"141'532","reported_seq":7040,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:14:56.031386+0000","last_change":"2024-09-13T09:14:44.426902+0000","last_active":"2024-09-13T09:14:56.031386+0000","last_peered":"2024-09-13T09:14:56.031386+0000","last_clean":"2024-09-13T09:14:56.031386+0000","last_became_active":"2024-09-13T09:14:44.309465+0000","last_became_peered":"2024-09-13T09:14:44.309465+0000","last_unstale":"2024-09-13T09:14:56.031386+0000","last_undegraded":"2024-09-13T09:14:56.031386+0000","last_fullsized":"2024-09-13T09:14:56.031386+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039480829999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6633,"num_read_kb":20805,"num_write":596,"num_write_kb":7043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":635004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"158'535","reported_seq":7963,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:14.954879+0000","last_change":"2024-09-13T09:15:14.954979+0000","last_active":"2024-09-13T09:15:14.954879+0000","last_peered":"2024-09-13T09:15:14.954879+0000","last_clean":"2024-09-13T09:15:14.954879+0000","last_became_active":"2024-09-13T09:14:44.308779+0000","last_became_peered":"2024-09-13T09:14:44.308779+0000","last_unstale":"2024-09-13T09:15:14.954879+0000","last_undegraded":"2024-09-13T09:15:14.954879+0000","last_fullsized":"2024-09-13T09:15:14.954879+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'535","last_scrub_stamp":"2024-09-13T09:15:14.954854+0000","last_deep_scrub":"158'535","last_deep_scrub_stamp":"2024-09-13T09:15:14.954854+0000","last_clean_scrub_stamp":"2024-09-13T09:15:14.954854+0000","objects_scrubbed":2,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:09:48.360500+0000","scrub_duration":0.0044186119999999997,"objects_trimmed":4,"snaptrim_duration":0.0066958310000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7617,"num_read_kb":37352,"num_write":622,"num_write_kb":17983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"158'546","reported_seq":4851,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:15.327145+0000","last_change":"2024-09-13T09:15:15.327268+0000","last_active":"2024-09-13T09:15:15.327145+0000","last_peered":"2024-09-13T09:15:15.327145+0000","last_clean":"2024-09-13T09:15:15.327145+0000","last_became_active":"2024-09-13T09:12:43.532504+0000","last_became_peered":"2024-09-13T09:12:43.532504+0000","last_unstale":"2024-09-13T09:15:15.327145+0000","last_undegraded":"2024-09-13T09:15:15.327145+0000","last_fullsized":"2024-09-13T09:15:15.327145+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'546","last_scrub_stamp":"2024-09-13T09:15:15.327121+0000","last_deep_scrub":"158'546","last_deep_scrub_stamp":"2024-09-13T09:15:15.327121+0000","last_clean_scrub_stamp":"2024-09-13T09:15:15.327121+0000","objects_scrubbed":1,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:41.649933+0000","scrub_duration":0.0030860060000000001,"objects_trimmed":0,"snaptrim_duration":0.0021463989999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5509,"num_read_kb":32081,"num_write":673,"num_write_kb":7448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":694311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"158'1123","reported_seq":20809,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:10.744288+0000","last_change":"2024-09-13T09:15:10.744394+0000","last_active":"2024-09-13T09:15:10.744288+0000","last_peered":"2024-09-13T09:15:10.744288+0000","last_clean":"2024-09-13T09:15:10.744288+0000","last_became_active":"2024-09-13T09:12:46.248261+0000","last_became_peered":"2024-09-13T09:12:46.248261+0000","last_unstale":"2024-09-13T09:15:10.744288+0000","last_undegraded":"2024-09-13T09:15:10.744288+0000","last_fullsized":"2024-09-13T09:15:10.744288+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1123","last_scrub_stamp":"2024-09-13T09:15:10.744263+0000","last_deep_scrub":"158'1123","last_deep_scrub_stamp":"2024-09-13T09:15:10.744263+0000","last_clean_scrub_stamp":"2024-09-13T09:15:10.744263+0000","objects_scrubbed":8,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:50:35.274717+0000","scrub_duration":0.0081009050000000003,"objects_trimmed":0,"snaptrim_duration":0.00092792899999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":26946,"num_read_kb":67879,"num_write":1035,"num_write_kb":10041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":676864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"141'603","reported_seq":6567,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:04.432844+0000","last_change":"2024-09-13T09:14:44.326177+0000","last_active":"2024-09-13T09:15:04.432844+0000","last_peered":"2024-09-13T09:15:04.432844+0000","last_clean":"2024-09-13T09:15:04.432844+0000","last_became_active":"2024-09-13T09:14:44.314564+0000","last_became_peered":"2024-09-13T09:14:44.314564+0000","last_unstale":"2024-09-13T09:15:04.432844+0000","last_undegraded":"2024-09-13T09:15:04.432844+0000","last_fullsized":"2024-09-13T09:15:04.432844+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:12:12.341598+0000","last_clean_scrub_stamp":"2024-09-13T09:12:12.341598+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050931029999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8526,"num_read_kb":13102,"num_write":519,"num_write_kb":3941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":28777,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":68,"num_read":356751,"num_read_kb":1364316,"num_write":26574,"num_write_kb":359719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":18167194,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28441,"internal_metadata":0},"log_size":23056,"ondisk_log_size":23056,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":146,"seq":627065225226,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31168,"kb_used_data":1848,"kb_used_omap":31,"kb_used_meta":29280,"kb_avail":94340672,"statfs":{"total":96636764160,"available":96604848128,"internally_reserved":0,"allocated":1892352,"data_stored":790048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32283,"internal_metadata":29983205},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]}]},{"osd":4,"up_from":11,"seq":47244640302,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64960,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306880,"statfs":{"total":96636764160,"available":96570245120,"internally_reserved":0,"allocated":1892352,"data_stored":790048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:15:01 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.41899999999999998,"15min":0.40400000000000003},"min":{"1min":0.28799999999999998,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.499},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.42099999999999999,"15min":0.39700000000000002},"min":{"1min":0.26200000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.44500000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:15:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.435,"15min":0.41199999999999998},"min":{"1min":0.29899999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.435,"15min":0.42499999999999999},"min":{"1min":0.29799999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Sep 13 09:15:18 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:15:01 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.33700000000000002},"min":{"1min":0.23599999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.309},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.253,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.629,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.35199999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.23799999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54400000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.372},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40000000000000002,"15min":0.40799999999999997},"min":{"1min":0.25,"5min":0.249,"15min":0.249},"max":{"1min":0.55400000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.39800000000000002}]}]},{"osd":3,"up_from":11,"seq":47244640303,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59768,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312072,"statfs":{"total":96636764160,"available":96575561728,"internally_reserved":0,"allocated":2473984,"data_stored":1378241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.40899999999999997},"min":{"1min":0.27400000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.72799999999999998,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48299999999999998,"15min":0.44700000000000001},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.77300000000000002,"5min":4.569,"15min":4.569},"last":0.57199999999999995}]},{"osd":1,"last update":"Fri Sep 13 09:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55400000000000005,"15min":0.497},"min":{"1min":0.26100000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.4179999999999999,"5min":4.5990000000000002,"15min":4.5990000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.32700000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.66300000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:15:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.63400000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34699999999999998,"15min":0.35199999999999998},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.39900000000000002,"15min":0.377},"min":{"1min":0.23100000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.64900000000000002,"5min":5.165,"15min":5.165},"last":0.42699999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38300000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.45000000000000001,"15min":0.40300000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":8.7959999999999994,"5min":8.7959999999999994,"15min":8.7959999999999994},"last":0.41699999999999998}]}]},{"osd":2,"up_from":122,"seq":523986010128,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35068,"kb_used_data":1844,"kb_used_omap":37,"kb_used_meta":33178,"kb_avail":94336772,"statfs":{"total":96636764160,"available":96600854528,"internally_reserved":0,"allocated":1888256,"data_stored":790029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38061,"internal_metadata":33975123},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:15:17 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.34100000000000003}]},{"osd":1,"last update":"Fri Sep 13 09:15:17 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.317},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.376}]},{"osd":3,"last update":"Fri Sep 13 09:15:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:15:17 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]}]},{"osd":1,"up_from":11,"seq":47244640302,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55108,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316732,"statfs":{"total":96636764160,"available":96580333568,"internally_reserved":0,"allocated":1896448,"data_stored":790067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:14:57 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.317,"15min":0.33300000000000002},"min":{"1min":0.219,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.41699999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.32400000000000001},"min":{"1min":0.22600000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.59999999999999998,"5min":0.753,"15min":0.753},"last":0.45100000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.499}]},{"osd":3,"last update":"Fri Sep 13 09:14:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.5,"15min":0.50800000000000001},"min":{"1min":0.34300000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49299999999999999,"15min":0.50600000000000001},"min":{"1min":0.375,"5min":0.219,"15min":0.219},"max":{"1min":0.66400000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Sep 13 09:14:57 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.47099999999999997,"15min":0.47499999999999998},"min":{"1min":0.39200000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68400000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.623},{"interface":"front","average":{"1min":0.497,"5min":0.47399999999999998,"15min":0.48199999999999998},"min":{"1min":0.35699999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.71499999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.63800000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.496,"15min":0.504},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.747},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.47099999999999997,"15min":0.48399999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.58199999999999996}]}]},{"osd":0,"up_from":11,"seq":47244640302,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63872,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307968,"statfs":{"total":96636764160,"available":96571359232,"internally_reserved":0,"allocated":2482176,"data_stored":1380397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:15:05 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.28799999999999998,"15min":0.28399999999999997},"min":{"1min":0.186,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.28899999999999998,"15min":0.28499999999999998},"min":{"1min":0.214,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.315}]},{"osd":2,"last update":"Fri Sep 13 09:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.27300000000000002}]},{"osd":3,"last update":"Fri Sep 13 09:15:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.46200000000000002,"15min":0.46400000000000002},"min":{"1min":0.32900000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.84899999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.46300000000000002,"15min":0.45200000000000001},"min":{"1min":0.35299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.745,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.54300000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:15:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.46999999999999997,"15min":0.46300000000000002},"min":{"1min":0.35299999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.46300000000000002},"min":{"1min":0.35999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46700000000000003,"15min":0.47599999999999998},"min":{"1min":0.29199999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88700000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.438,"5min":0.45500000000000002,"15min":0.45800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69799999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.504}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16703,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11738,"internal_metadata":0}]}} 2024-09-13T09:15:26.139 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:26.139 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:26.139 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-13T09:12:03.304621+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:26.139 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T09:12:03.304621+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:26.139 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-13T09:12:03.304621+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:26.140 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:26.140 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:26.140 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T09:12:12.341598+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=12, tm_sec=12, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=15, tm_sec=1, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:15:26.140 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T09:15:46.141 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:15:46.440 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-09-13T09:15:46.440 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-09-13T09:15:46.451 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-13T09:15:45.559210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":68,"num_read":356867,"num_read_kb":1364416,"num_write":26735,"num_write_kb":362137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":666,"num_bytes_recovered":19347930,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23189,"ondisk_log_size":23189,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":309944,"kb_used_data":12232,"kb_used_omap":74,"kb_used_meta":297589,"kb_avail":565921096,"statfs":{"total":579820584960,"available":579503202304,"internally_reserved":0,"allocated":12525568,"data_stored":5918830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":76719,"internal_metadata":304731217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002262"},"pg_stats":[{"pgid":"3.1f","version":"158'860","reported_seq":9668,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:11.710291+0000","last_change":"2024-09-13T09:15:11.710399+0000","last_active":"2024-09-13T09:15:11.710291+0000","last_peered":"2024-09-13T09:15:11.710291+0000","last_clean":"2024-09-13T09:15:11.710291+0000","last_became_active":"2024-09-13T09:12:43.534990+0000","last_became_peered":"2024-09-13T09:12:43.534990+0000","last_unstale":"2024-09-13T09:15:11.710291+0000","last_undegraded":"2024-09-13T09:15:11.710291+0000","last_fullsized":"2024-09-13T09:15:11.710291+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'860","last_scrub_stamp":"2024-09-13T09:15:11.710267+0000","last_deep_scrub":"158'860","last_deep_scrub_stamp":"2024-09-13T09:15:11.710267+0000","last_clean_scrub_stamp":"2024-09-13T09:15:11.710267+0000","objects_scrubbed":3,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:48:19.176034+0000","scrub_duration":0.0048191789999999998,"objects_trimmed":0,"snaptrim_duration":0.00093900099999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":11993,"num_read_kb":21637,"num_write":921,"num_write_kb":7095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":848912,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'606","reported_seq":7906,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:06.837089+0000","last_change":"2024-09-13T09:15:06.837206+0000","last_active":"2024-09-13T09:15:06.837089+0000","last_peered":"2024-09-13T09:15:06.837089+0000","last_clean":"2024-09-13T09:15:06.837089+0000","last_became_active":"2024-09-13T09:14:16.749234+0000","last_became_peered":"2024-09-13T09:14:16.749234+0000","last_unstale":"2024-09-13T09:15:06.837089+0000","last_undegraded":"2024-09-13T09:15:06.837089+0000","last_fullsized":"2024-09-13T09:15:06.837089+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'606","last_scrub_stamp":"2024-09-13T09:15:06.837063+0000","last_deep_scrub":"158'606","last_deep_scrub_stamp":"2024-09-13T09:15:06.837063+0000","last_clean_scrub_stamp":"2024-09-13T09:15:06.837063+0000","objects_scrubbed":1,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:15:46.209884+0000","scrub_duration":0.003657099,"objects_trimmed":0,"snaptrim_duration":0.0020237760000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10031,"num_read_kb":30858,"num_write":737,"num_write_kb":10943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":313856,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"158'1439","reported_seq":16460,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:24.777014+0000","last_change":"2024-09-13T09:15:24.777139+0000","last_active":"2024-09-13T09:15:24.777014+0000","last_peered":"2024-09-13T09:15:24.777014+0000","last_clean":"2024-09-13T09:15:24.777014+0000","last_became_active":"2024-09-13T09:14:44.312734+0000","last_became_peered":"2024-09-13T09:14:44.312734+0000","last_unstale":"2024-09-13T09:15:24.777014+0000","last_undegraded":"2024-09-13T09:15:24.777014+0000","last_fullsized":"2024-09-13T09:15:24.777014+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1439","last_scrub_stamp":"2024-09-13T09:15:24.776990+0000","last_deep_scrub":"158'1439","last_deep_scrub_stamp":"2024-09-13T09:15:24.776990+0000","last_clean_scrub_stamp":"2024-09-13T09:15:24.776990+0000","objects_scrubbed":4,"log_size":1439,"log_dups_size":0,"ondisk_log_size":1439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:27.007843+0000","scrub_duration":0.0085349350000000004,"objects_trimmed":2,"snaptrim_duration":0.011136765999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":20060,"num_read_kb":34855,"num_write":1554,"num_write_kb":7371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"158'1481","reported_seq":8622,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:12.953879+0000","last_change":"2024-09-13T09:15:12.953998+0000","last_active":"2024-09-13T09:15:12.953879+0000","last_peered":"2024-09-13T09:15:12.953879+0000","last_clean":"2024-09-13T09:15:12.953879+0000","last_became_active":"2024-09-13T09:14:44.311550+0000","last_became_peered":"2024-09-13T09:14:44.311550+0000","last_unstale":"2024-09-13T09:15:12.953879+0000","last_undegraded":"2024-09-13T09:15:12.953879+0000","last_fullsized":"2024-09-13T09:15:12.953879+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1481","last_scrub_stamp":"2024-09-13T09:15:12.953853+0000","last_deep_scrub":"158'1481","last_deep_scrub_stamp":"2024-09-13T09:15:12.953853+0000","last_clean_scrub_stamp":"2024-09-13T09:15:12.953853+0000","objects_scrubbed":1,"log_size":1481,"log_dups_size":0,"ondisk_log_size":1481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:25.555906+0000","scrub_duration":0.0034116120000000001,"objects_trimmed":6,"snaptrim_duration":0.022244201000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13526,"num_read_kb":55870,"num_write":2225,"num_write_kb":20106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"142'635","reported_seq":9520,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:17.768893+0000","last_change":"2024-09-13T09:15:17.769007+0000","last_active":"2024-09-13T09:15:17.768893+0000","last_peered":"2024-09-13T09:15:17.768893+0000","last_clean":"2024-09-13T09:15:17.768893+0000","last_became_active":"2024-09-13T09:14:19.772731+0000","last_became_peered":"2024-09-13T09:14:19.772731+0000","last_unstale":"2024-09-13T09:15:17.768893+0000","last_undegraded":"2024-09-13T09:15:17.768893+0000","last_fullsized":"2024-09-13T09:15:17.768893+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'635","last_scrub_stamp":"2024-09-13T09:15:17.768869+0000","last_deep_scrub":"142'635","last_deep_scrub_stamp":"2024-09-13T09:15:17.768869+0000","last_clean_scrub_stamp":"2024-09-13T09:15:17.768869+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:32.891266+0000","scrub_duration":0.0023082340000000002,"objects_trimmed":0,"snaptrim_duration":0.001509144,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10475,"num_read_kb":52366,"num_write":680,"num_write_kb":12577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1411144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"158'631","reported_seq":14459,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:12.631957+0000","last_change":"2024-09-13T09:15:12.632069+0000","last_active":"2024-09-13T09:15:12.631957+0000","last_peered":"2024-09-13T09:15:12.631957+0000","last_clean":"2024-09-13T09:15:12.631957+0000","last_became_active":"2024-09-13T09:12:13.362533+0000","last_became_peered":"2024-09-13T09:12:13.362533+0000","last_unstale":"2024-09-13T09:15:12.631957+0000","last_undegraded":"2024-09-13T09:15:12.631957+0000","last_fullsized":"2024-09-13T09:15:12.631957+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'631","last_scrub_stamp":"2024-09-13T09:15:12.631934+0000","last_deep_scrub":"158'631","last_deep_scrub_stamp":"2024-09-13T09:15:12.631934+0000","last_clean_scrub_stamp":"2024-09-13T09:15:12.631934+0000","objects_scrubbed":3,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:22.761296+0000","scrub_duration":0.0051449820000000002,"objects_trimmed":0,"snaptrim_duration":0.0017407639999999999,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":16059,"num_read_kb":55558,"num_write":617,"num_write_kb":11980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"158'570","reported_seq":8673,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:17.351227+0000","last_change":"2024-09-13T09:15:17.351340+0000","last_active":"2024-09-13T09:15:17.351227+0000","last_peered":"2024-09-13T09:15:17.351227+0000","last_clean":"2024-09-13T09:15:17.351227+0000","last_became_active":"2024-09-13T09:14:48.350477+0000","last_became_peered":"2024-09-13T09:14:48.350477+0000","last_unstale":"2024-09-13T09:15:17.351227+0000","last_undegraded":"2024-09-13T09:15:17.351227+0000","last_fullsized":"2024-09-13T09:15:17.351227+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'570","last_scrub_stamp":"2024-09-13T09:15:17.351207+0000","last_deep_scrub":"158'570","last_deep_scrub_stamp":"2024-09-13T09:15:17.351207+0000","last_clean_scrub_stamp":"2024-09-13T09:15:17.351207+0000","objects_scrubbed":3,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:07.107808+0000","scrub_duration":0.0044970440000000004,"objects_trimmed":0,"snaptrim_duration":0.00074163899999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8163,"num_read_kb":27760,"num_write":669,"num_write_kb":11976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"158'581","reported_seq":11088,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:19.851296+0000","last_change":"2024-09-13T09:15:19.851443+0000","last_active":"2024-09-13T09:15:19.851296+0000","last_peered":"2024-09-13T09:15:19.851296+0000","last_clean":"2024-09-13T09:15:19.851296+0000","last_became_active":"2024-09-13T09:14:19.772799+0000","last_became_peered":"2024-09-13T09:14:19.772799+0000","last_unstale":"2024-09-13T09:15:19.851296+0000","last_undegraded":"2024-09-13T09:15:19.851296+0000","last_fullsized":"2024-09-13T09:15:19.851296+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'581","last_scrub_stamp":"2024-09-13T09:15:19.851268+0000","last_deep_scrub":"158'581","last_deep_scrub_stamp":"2024-09-13T09:15:19.851268+0000","last_clean_scrub_stamp":"2024-09-13T09:15:19.851268+0000","objects_scrubbed":5,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:51:55.388636+0000","scrub_duration":0.0064302480000000004,"objects_trimmed":0,"snaptrim_duration":0.001235604,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13056,"num_read_kb":24986,"num_write":439,"num_write_kb":7320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":104960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"147'434","reported_seq":5723,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:09.748910+0000","last_change":"2024-09-13T09:15:09.749041+0000","last_active":"2024-09-13T09:15:09.748910+0000","last_peered":"2024-09-13T09:15:09.748910+0000","last_clean":"2024-09-13T09:15:09.748910+0000","last_became_active":"2024-09-13T09:14:44.317071+0000","last_became_peered":"2024-09-13T09:14:44.317071+0000","last_unstale":"2024-09-13T09:15:09.748910+0000","last_undegraded":"2024-09-13T09:15:09.748910+0000","last_fullsized":"2024-09-13T09:15:09.748910+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'434","last_scrub_stamp":"2024-09-13T09:15:09.748886+0000","last_deep_scrub":"147'434","last_deep_scrub_stamp":"2024-09-13T09:15:09.748886+0000","last_clean_scrub_stamp":"2024-09-13T09:15:09.748886+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:28:03.194738+0000","scrub_duration":0.0021969480000000002,"objects_trimmed":4,"snaptrim_duration":0.015690497000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6024,"num_read_kb":31779,"num_write":503,"num_write_kb":7984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"158'975","reported_seq":6758,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:08.940009+0000","last_change":"2024-09-13T09:15:08.940139+0000","last_active":"2024-09-13T09:15:08.940009+0000","last_peered":"2024-09-13T09:15:08.940009+0000","last_clean":"2024-09-13T09:15:08.940009+0000","last_became_active":"2024-09-13T09:14:44.309746+0000","last_became_peered":"2024-09-13T09:14:44.309746+0000","last_unstale":"2024-09-13T09:15:08.940009+0000","last_undegraded":"2024-09-13T09:15:08.940009+0000","last_fullsized":"2024-09-13T09:15:08.940009+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'975","last_scrub_stamp":"2024-09-13T09:15:08.939985+0000","last_deep_scrub":"158'975","last_deep_scrub_stamp":"2024-09-13T09:15:08.939985+0000","last_clean_scrub_stamp":"2024-09-13T09:15:08.939985+0000","objects_scrubbed":3,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:14:34.542109+0000","scrub_duration":0.0084448349999999995,"objects_trimmed":2,"snaptrim_duration":0.0084025269999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8520,"num_read_kb":43161,"num_write":1440,"num_write_kb":11751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":706121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"158'874","reported_seq":20353,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:31.398329+0000","last_change":"2024-09-13T09:15:31.398435+0000","last_active":"2024-09-13T09:15:31.398329+0000","last_peered":"2024-09-13T09:15:31.398329+0000","last_clean":"2024-09-13T09:15:31.398329+0000","last_became_active":"2024-09-13T09:12:46.250696+0000","last_became_peered":"2024-09-13T09:12:46.250696+0000","last_unstale":"2024-09-13T09:15:31.398329+0000","last_undegraded":"2024-09-13T09:15:31.398329+0000","last_fullsized":"2024-09-13T09:15:31.398329+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'874","last_scrub_stamp":"2024-09-13T09:15:31.398306+0000","last_deep_scrub":"158'874","last_deep_scrub_stamp":"2024-09-13T09:15:31.398306+0000","last_clean_scrub_stamp":"2024-09-13T09:15:31.398306+0000","objects_scrubbed":2,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:57:46.844501+0000","scrub_duration":0.004318204,"objects_trimmed":0,"snaptrim_duration":0.0019682329999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":21840,"num_read_kb":55389,"num_write":1298,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1136450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"158'667","reported_seq":8604,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:19.354352+0000","last_change":"2024-09-13T09:15:19.354480+0000","last_active":"2024-09-13T09:15:19.354352+0000","last_peered":"2024-09-13T09:15:19.354352+0000","last_clean":"2024-09-13T09:15:19.354352+0000","last_became_active":"2024-09-13T09:12:46.254016+0000","last_became_peered":"2024-09-13T09:12:46.254016+0000","last_unstale":"2024-09-13T09:15:19.354352+0000","last_undegraded":"2024-09-13T09:15:19.354352+0000","last_fullsized":"2024-09-13T09:15:19.354352+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'667","last_scrub_stamp":"2024-09-13T09:15:19.354327+0000","last_deep_scrub":"158'667","last_deep_scrub_stamp":"2024-09-13T09:15:19.354327+0000","last_clean_scrub_stamp":"2024-09-13T09:15:19.354327+0000","objects_scrubbed":2,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:44.282864+0000","scrub_duration":0.0039345300000000003,"objects_trimmed":0,"snaptrim_duration":0.0013411219999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10880,"num_read_kb":53346,"num_write":603,"num_write_kb":9445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1110608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:26.795518+0000","last_change":"2024-09-13T09:15:26.795628+0000","last_active":"2024-09-13T09:15:26.795518+0000","last_peered":"2024-09-13T09:15:26.795518+0000","last_clean":"2024-09-13T09:15:26.795518+0000","last_became_active":"2024-09-13T09:14:44.316062+0000","last_became_peered":"2024-09-13T09:14:44.316062+0000","last_unstale":"2024-09-13T09:15:26.795518+0000","last_undegraded":"2024-09-13T09:15:26.795518+0000","last_fullsized":"2024-09-13T09:15:26.795518+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:15:26.795496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:15:26.795496+0000","last_clean_scrub_stamp":"2024-09-13T09:15:26.795496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:14.108658+0000","scrub_duration":0.001461473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"158'755","reported_seq":10747,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:07.717807+0000","last_change":"2024-09-13T09:15:07.717952+0000","last_active":"2024-09-13T09:15:07.717807+0000","last_peered":"2024-09-13T09:15:07.717807+0000","last_clean":"2024-09-13T09:15:07.717807+0000","last_became_active":"2024-09-13T09:12:13.365496+0000","last_became_peered":"2024-09-13T09:12:13.365496+0000","last_unstale":"2024-09-13T09:15:07.717807+0000","last_undegraded":"2024-09-13T09:15:07.717807+0000","last_fullsized":"2024-09-13T09:15:07.717807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'755","last_scrub_stamp":"2024-09-13T09:15:07.717786+0000","last_deep_scrub":"158'755","last_deep_scrub_stamp":"2024-09-13T09:15:07.717786+0000","last_clean_scrub_stamp":"2024-09-13T09:15:07.717786+0000","objects_scrubbed":2,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:26:11.704168+0000","scrub_duration":0.0042564270000000001,"objects_trimmed":0,"snaptrim_duration":0.00063501900000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11213,"num_read_kb":65815,"num_write":999,"num_write_kb":12170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:09.390649+0000","last_change":"2024-09-13T09:15:09.390791+0000","last_active":"2024-09-13T09:15:09.390649+0000","last_peered":"2024-09-13T09:15:09.390649+0000","last_clean":"2024-09-13T09:15:09.390649+0000","last_became_active":"2024-09-13T09:14:48.352502+0000","last_became_peered":"2024-09-13T09:14:48.352502+0000","last_unstale":"2024-09-13T09:15:09.390649+0000","last_undegraded":"2024-09-13T09:15:09.390649+0000","last_fullsized":"2024-09-13T09:15:09.390649+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:15:09.390608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:15:09.390608+0000","last_clean_scrub_stamp":"2024-09-13T09:15:09.390608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:00.477020+0000","scrub_duration":0.001921126,"objects_trimmed":0,"snaptrim_duration":0.00040313000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"158'1049","reported_seq":10572,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:11.788249+0000","last_change":"2024-09-13T09:15:11.788378+0000","last_active":"2024-09-13T09:15:11.788249+0000","last_peered":"2024-09-13T09:15:11.788249+0000","last_clean":"2024-09-13T09:15:11.788249+0000","last_became_active":"2024-09-13T09:14:16.752774+0000","last_became_peered":"2024-09-13T09:14:16.752774+0000","last_unstale":"2024-09-13T09:15:11.788249+0000","last_undegraded":"2024-09-13T09:15:11.788249+0000","last_fullsized":"2024-09-13T09:15:11.788249+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1049","last_scrub_stamp":"2024-09-13T09:15:11.788225+0000","last_deep_scrub":"158'1049","last_deep_scrub_stamp":"2024-09-13T09:15:11.788225+0000","last_clean_scrub_stamp":"2024-09-13T09:15:11.788225+0000","objects_scrubbed":2,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:01.052086+0000","scrub_duration":0.0039185440000000004,"objects_trimmed":0,"snaptrim_duration":0.001863822,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":16931,"num_read_kb":50979,"num_write":1279,"num_write_kb":12713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":413184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":133,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:26.385304+0000","last_change":"2024-09-13T09:15:26.385422+0000","last_active":"2024-09-13T09:15:26.385304+0000","last_peered":"2024-09-13T09:15:26.385304+0000","last_clean":"2024-09-13T09:15:26.385304+0000","last_became_active":"2024-09-13T09:12:43.533557+0000","last_became_peered":"2024-09-13T09:12:43.533557+0000","last_unstale":"2024-09-13T09:15:26.385304+0000","last_undegraded":"2024-09-13T09:15:26.385304+0000","last_fullsized":"2024-09-13T09:15:26.385304+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:15:26.385284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:15:26.385284+0000","last_clean_scrub_stamp":"2024-09-13T09:15:26.385284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:48:04.531304+0000","scrub_duration":0.0016319150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"158'709","reported_seq":13369,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:15.412540+0000","last_change":"2024-09-13T09:15:15.412644+0000","last_active":"2024-09-13T09:15:15.412540+0000","last_peered":"2024-09-13T09:15:15.412540+0000","last_clean":"2024-09-13T09:15:15.412540+0000","last_became_active":"2024-09-13T09:14:48.348909+0000","last_became_peered":"2024-09-13T09:14:48.348909+0000","last_unstale":"2024-09-13T09:15:15.412540+0000","last_undegraded":"2024-09-13T09:15:15.412540+0000","last_fullsized":"2024-09-13T09:15:15.412540+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'709","last_scrub_stamp":"2024-09-13T09:15:15.412515+0000","last_deep_scrub":"158'709","last_deep_scrub_stamp":"2024-09-13T09:15:15.412515+0000","last_clean_scrub_stamp":"2024-09-13T09:15:15.412515+0000","objects_scrubbed":2,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:13.817613+0000","scrub_duration":0.0039132079999999996,"objects_trimmed":0,"snaptrim_duration":0.001074057,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":15357,"num_read_kb":39900,"num_write":798,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":29184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":152,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:20.333802+0000","last_change":"2024-09-13T09:15:20.333918+0000","last_active":"2024-09-13T09:15:20.333802+0000","last_peered":"2024-09-13T09:15:20.333802+0000","last_clean":"2024-09-13T09:15:20.333802+0000","last_became_active":"2024-09-13T09:14:48.348560+0000","last_became_peered":"2024-09-13T09:14:48.348560+0000","last_unstale":"2024-09-13T09:15:20.333802+0000","last_undegraded":"2024-09-13T09:15:20.333802+0000","last_fullsized":"2024-09-13T09:15:20.333802+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:15:20.333781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:15:20.333781+0000","last_clean_scrub_stamp":"2024-09-13T09:15:20.333781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:06:50.801030+0000","scrub_duration":0.001463233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"158'750","reported_seq":7675,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:13.998121+0000","last_change":"2024-09-13T09:15:13.998227+0000","last_active":"2024-09-13T09:15:13.998121+0000","last_peered":"2024-09-13T09:15:13.998121+0000","last_clean":"2024-09-13T09:15:13.998121+0000","last_became_active":"2024-09-13T09:14:44.309435+0000","last_became_peered":"2024-09-13T09:14:44.309435+0000","last_unstale":"2024-09-13T09:15:13.998121+0000","last_undegraded":"2024-09-13T09:15:13.998121+0000","last_fullsized":"2024-09-13T09:15:13.998121+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'750","last_scrub_stamp":"2024-09-13T09:15:13.998097+0000","last_deep_scrub":"158'750","last_deep_scrub_stamp":"2024-09-13T09:15:13.998097+0000","last_clean_scrub_stamp":"2024-09-13T09:15:13.998097+0000","objects_scrubbed":2,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:35:18.377733+0000","scrub_duration":0.0043266329999999999,"objects_trimmed":0,"snaptrim_duration":0.0039806149999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10405,"num_read_kb":27276,"num_write":715,"num_write_kb":2474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":182784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"158'637","reported_seq":11591,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:11.616512+0000","last_change":"2024-09-13T09:15:11.616642+0000","last_active":"2024-09-13T09:15:11.616512+0000","last_peered":"2024-09-13T09:15:11.616512+0000","last_clean":"2024-09-13T09:15:11.616512+0000","last_became_active":"2024-09-13T09:12:13.367770+0000","last_became_peered":"2024-09-13T09:12:13.367770+0000","last_unstale":"2024-09-13T09:15:11.616512+0000","last_undegraded":"2024-09-13T09:15:11.616512+0000","last_fullsized":"2024-09-13T09:15:11.616512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'637","last_scrub_stamp":"2024-09-13T09:15:11.616471+0000","last_deep_scrub":"158'637","last_deep_scrub_stamp":"2024-09-13T09:15:11.616471+0000","last_clean_scrub_stamp":"2024-09-13T09:15:11.616471+0000","objects_scrubbed":3,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:55.709145+0000","scrub_duration":0.0053358210000000001,"objects_trimmed":0,"snaptrim_duration":0.001191176,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10993,"num_read_kb":67376,"num_write":601,"num_write_kb":14012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"158'3","reported_seq":135,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:22.850740+0000","last_change":"2024-09-13T09:15:22.850896+0000","last_active":"2024-09-13T09:15:22.850740+0000","last_peered":"2024-09-13T09:15:22.850740+0000","last_clean":"2024-09-13T09:15:22.850740+0000","last_became_active":"2024-09-13T09:14:44.311178+0000","last_became_peered":"2024-09-13T09:14:44.311178+0000","last_unstale":"2024-09-13T09:15:22.850740+0000","last_undegraded":"2024-09-13T09:15:22.850740+0000","last_fullsized":"2024-09-13T09:15:22.850740+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'3","last_scrub_stamp":"2024-09-13T09:15:22.850718+0000","last_deep_scrub":"158'3","last_deep_scrub_stamp":"2024-09-13T09:15:22.850718+0000","last_clean_scrub_stamp":"2024-09-13T09:15:22.850718+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:37:25.659537+0000","scrub_duration":0.0035561439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"158'623","reported_seq":9666,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:05.908691+0000","last_change":"2024-09-13T09:15:05.908843+0000","last_active":"2024-09-13T09:15:05.908691+0000","last_peered":"2024-09-13T09:15:05.908691+0000","last_clean":"2024-09-13T09:15:05.908691+0000","last_became_active":"2024-09-13T09:14:44.310016+0000","last_became_peered":"2024-09-13T09:14:44.310016+0000","last_unstale":"2024-09-13T09:15:05.908691+0000","last_undegraded":"2024-09-13T09:15:05.908691+0000","last_fullsized":"2024-09-13T09:15:05.908691+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'623","last_scrub_stamp":"2024-09-13T09:15:05.908668+0000","last_deep_scrub":"158'623","last_deep_scrub_stamp":"2024-09-13T09:15:05.908668+0000","last_clean_scrub_stamp":"2024-09-13T09:15:05.908668+0000","objects_scrubbed":3,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:48:30.181439+0000","scrub_duration":0.0067150650000000001,"objects_trimmed":6,"snaptrim_duration":0.007574824,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":9521,"num_read_kb":48768,"num_write":764,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1695776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":118,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:16.793937+0000","last_change":"2024-09-13T09:15:16.794049+0000","last_active":"2024-09-13T09:15:16.793937+0000","last_peered":"2024-09-13T09:15:16.793937+0000","last_clean":"2024-09-13T09:15:16.793937+0000","last_became_active":"2024-09-13T09:14:16.748865+0000","last_became_peered":"2024-09-13T09:14:16.748865+0000","last_unstale":"2024-09-13T09:15:16.793937+0000","last_undegraded":"2024-09-13T09:15:16.793937+0000","last_fullsized":"2024-09-13T09:15:16.793937+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:15:16.793915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:15:16.793915+0000","last_clean_scrub_stamp":"2024-09-13T09:15:16.793915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:47.978876+0000","scrub_duration":0.001464545,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"158'333","reported_seq":3873,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:03.727755+0000","last_change":"2024-09-13T09:15:03.727892+0000","last_active":"2024-09-13T09:15:03.727755+0000","last_peered":"2024-09-13T09:15:03.727755+0000","last_clean":"2024-09-13T09:15:03.727755+0000","last_became_active":"2024-09-13T09:12:13.367633+0000","last_became_peered":"2024-09-13T09:12:13.367633+0000","last_unstale":"2024-09-13T09:15:03.727755+0000","last_undegraded":"2024-09-13T09:15:03.727755+0000","last_fullsized":"2024-09-13T09:15:03.727755+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'333","last_scrub_stamp":"2024-09-13T09:15:03.727710+0000","last_deep_scrub":"158'333","last_deep_scrub_stamp":"2024-09-13T09:15:03.727710+0000","last_clean_scrub_stamp":"2024-09-13T09:15:03.727710+0000","objects_scrubbed":1,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:54.610562+0000","scrub_duration":0.003663763,"objects_trimmed":0,"snaptrim_duration":0.00094036499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3280,"num_read_kb":26753,"num_write":312,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:11.318925+0000","last_change":"2024-09-13T09:15:11.319019+0000","last_active":"2024-09-13T09:15:11.318925+0000","last_peered":"2024-09-13T09:15:11.318925+0000","last_clean":"2024-09-13T09:15:11.318925+0000","last_became_active":"2024-09-13T09:12:43.531942+0000","last_became_peered":"2024-09-13T09:12:43.531942+0000","last_unstale":"2024-09-13T09:15:11.318925+0000","last_undegraded":"2024-09-13T09:15:11.318925+0000","last_fullsized":"2024-09-13T09:15:11.318925+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:15:11.318900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:15:11.318900+0000","last_clean_scrub_stamp":"2024-09-13T09:15:11.318900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:02.547367+0000","scrub_duration":0.0019515100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"158'656","reported_seq":3831,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:21.366714+0000","last_change":"2024-09-13T09:15:21.366849+0000","last_active":"2024-09-13T09:15:21.366714+0000","last_peered":"2024-09-13T09:15:21.366714+0000","last_clean":"2024-09-13T09:15:21.366714+0000","last_became_active":"2024-09-13T09:14:48.349138+0000","last_became_peered":"2024-09-13T09:14:48.349138+0000","last_unstale":"2024-09-13T09:15:21.366714+0000","last_undegraded":"2024-09-13T09:15:21.366714+0000","last_fullsized":"2024-09-13T09:15:21.366714+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'656","last_scrub_stamp":"2024-09-13T09:15:21.366692+0000","last_deep_scrub":"158'656","last_deep_scrub_stamp":"2024-09-13T09:15:21.366692+0000","last_clean_scrub_stamp":"2024-09-13T09:15:21.366692+0000","objects_scrubbed":3,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:59:39.559245+0000","scrub_duration":0.007211767,"objects_trimmed":0,"snaptrim_duration":0.00039583799999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4158,"num_read_kb":20449,"num_write":848,"num_write_kb":5187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'2","reported_seq":168,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:15.944248+0000","last_change":"2024-09-13T09:15:15.944364+0000","last_active":"2024-09-13T09:15:15.944248+0000","last_peered":"2024-09-13T09:15:15.944248+0000","last_clean":"2024-09-13T09:15:15.944248+0000","last_became_active":"2024-09-13T09:14:44.309293+0000","last_became_peered":"2024-09-13T09:14:44.309293+0000","last_unstale":"2024-09-13T09:15:15.944248+0000","last_undegraded":"2024-09-13T09:15:15.944248+0000","last_fullsized":"2024-09-13T09:15:15.944248+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'2","last_scrub_stamp":"2024-09-13T09:15:15.944225+0000","last_deep_scrub":"158'2","last_deep_scrub_stamp":"2024-09-13T09:15:15.944225+0000","last_clean_scrub_stamp":"2024-09-13T09:15:15.944225+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:59.264942+0000","scrub_duration":0.0035584509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"158'119","reported_seq":338,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:09.242237+0000","last_change":"2024-09-13T09:15:09.242356+0000","last_active":"2024-09-13T09:15:09.242237+0000","last_peered":"2024-09-13T09:15:09.242237+0000","last_clean":"2024-09-13T09:15:09.242237+0000","last_became_active":"2024-09-13T09:12:46.248451+0000","last_became_peered":"2024-09-13T09:12:46.248451+0000","last_unstale":"2024-09-13T09:15:09.242237+0000","last_undegraded":"2024-09-13T09:15:09.242237+0000","last_fullsized":"2024-09-13T09:15:09.242237+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'119","last_scrub_stamp":"2024-09-13T09:15:09.242213+0000","last_deep_scrub":"158'119","last_deep_scrub_stamp":"2024-09-13T09:15:09.242213+0000","last_clean_scrub_stamp":"2024-09-13T09:15:09.242213+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:49:57.189752+0000","scrub_duration":0.0047912179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"158'501","reported_seq":4713,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:10.817551+0000","last_change":"2024-09-13T09:15:10.817665+0000","last_active":"2024-09-13T09:15:10.817551+0000","last_peered":"2024-09-13T09:15:10.817551+0000","last_clean":"2024-09-13T09:15:10.817551+0000","last_became_active":"2024-09-13T09:14:16.754685+0000","last_became_peered":"2024-09-13T09:14:16.754685+0000","last_unstale":"2024-09-13T09:15:10.817551+0000","last_undegraded":"2024-09-13T09:15:10.817551+0000","last_fullsized":"2024-09-13T09:15:10.817551+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'501","last_scrub_stamp":"2024-09-13T09:15:10.817525+0000","last_deep_scrub":"158'501","last_deep_scrub_stamp":"2024-09-13T09:15:10.817525+0000","last_clean_scrub_stamp":"2024-09-13T09:15:10.817525+0000","objects_scrubbed":3,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:57:30.466409+0000","scrub_duration":0.0041960280000000001,"objects_trimmed":0,"snaptrim_duration":0.0022974409999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4733,"num_read_kb":22943,"num_write":604,"num_write_kb":15013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":45056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"158'1220","reported_seq":13799,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:06.576292+0000","last_change":"2024-09-13T09:15:06.576429+0000","last_active":"2024-09-13T09:15:06.576292+0000","last_peered":"2024-09-13T09:15:06.576292+0000","last_clean":"2024-09-13T09:15:06.576292+0000","last_became_active":"2024-09-13T09:14:19.774300+0000","last_became_peered":"2024-09-13T09:14:19.774300+0000","last_unstale":"2024-09-13T09:15:06.576292+0000","last_undegraded":"2024-09-13T09:15:06.576292+0000","last_fullsized":"2024-09-13T09:15:06.576292+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1220","last_scrub_stamp":"2024-09-13T09:15:06.576269+0000","last_deep_scrub":"158'1220","last_deep_scrub_stamp":"2024-09-13T09:15:06.576269+0000","last_clean_scrub_stamp":"2024-09-13T09:15:06.576269+0000","objects_scrubbed":4,"log_size":1220,"log_dups_size":0,"ondisk_log_size":1220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:28:51.559616+0000","scrub_duration":0.0065206120000000003,"objects_trimmed":0,"snaptrim_duration":0.001648159,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18678,"num_read_kb":75566,"num_write":1537,"num_write_kb":13034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1249280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"158'598","reported_seq":11916,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:13.281747+0000","last_change":"2024-09-13T09:15:13.281873+0000","last_active":"2024-09-13T09:15:13.281747+0000","last_peered":"2024-09-13T09:15:13.281747+0000","last_clean":"2024-09-13T09:15:13.281747+0000","last_became_active":"2024-09-13T09:12:46.252578+0000","last_became_peered":"2024-09-13T09:12:46.252578+0000","last_unstale":"2024-09-13T09:15:13.281747+0000","last_undegraded":"2024-09-13T09:15:13.281747+0000","last_fullsized":"2024-09-13T09:15:13.281747+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'598","last_scrub_stamp":"2024-09-13T09:15:13.281723+0000","last_deep_scrub":"158'598","last_deep_scrub_stamp":"2024-09-13T09:15:13.281723+0000","last_clean_scrub_stamp":"2024-09-13T09:15:13.281723+0000","objects_scrubbed":2,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:28.279782+0000","scrub_duration":0.0035929460000000001,"objects_trimmed":0,"snaptrim_duration":0.001634775,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":12679,"num_read_kb":50965,"num_write":515,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":2590736,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'527","reported_seq":8212,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:16.380252+0000","last_change":"2024-09-13T09:15:16.380375+0000","last_active":"2024-09-13T09:15:16.380252+0000","last_peered":"2024-09-13T09:15:16.380252+0000","last_clean":"2024-09-13T09:15:16.380252+0000","last_became_active":"2024-09-13T09:14:48.349512+0000","last_became_peered":"2024-09-13T09:14:48.349512+0000","last_unstale":"2024-09-13T09:15:16.380252+0000","last_undegraded":"2024-09-13T09:15:16.380252+0000","last_fullsized":"2024-09-13T09:15:16.380252+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'527","last_scrub_stamp":"2024-09-13T09:15:16.380228+0000","last_deep_scrub":"158'527","last_deep_scrub_stamp":"2024-09-13T09:15:16.380228+0000","last_clean_scrub_stamp":"2024-09-13T09:15:16.380228+0000","objects_scrubbed":2,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:17.836080+0000","scrub_duration":0.0037333000000000002,"objects_trimmed":0,"snaptrim_duration":0.0016583220000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8119,"num_read_kb":73640,"num_write":604,"num_write_kb":21508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"158'867","reported_seq":13481,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:16.968884+0000","last_change":"2024-09-13T09:15:16.969037+0000","last_active":"2024-09-13T09:15:16.968884+0000","last_peered":"2024-09-13T09:15:16.968884+0000","last_clean":"2024-09-13T09:15:16.968884+0000","last_became_active":"2024-09-13T09:14:44.313818+0000","last_became_peered":"2024-09-13T09:14:44.313818+0000","last_unstale":"2024-09-13T09:15:16.968884+0000","last_undegraded":"2024-09-13T09:15:16.968884+0000","last_fullsized":"2024-09-13T09:15:16.968884+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'867","last_scrub_stamp":"2024-09-13T09:15:16.968828+0000","last_deep_scrub":"158'867","last_deep_scrub_stamp":"2024-09-13T09:15:16.968828+0000","last_clean_scrub_stamp":"2024-09-13T09:15:16.968828+0000","objects_scrubbed":1,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:30.960784+0000","scrub_duration":0.0040212370000000004,"objects_trimmed":8,"snaptrim_duration":0.022344567999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":15285,"num_read_kb":90031,"num_write":1043,"num_write_kb":32713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"158'365","reported_seq":3891,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:32.437400+0000","last_change":"2024-09-13T09:15:32.437510+0000","last_active":"2024-09-13T09:15:32.437400+0000","last_peered":"2024-09-13T09:15:32.437400+0000","last_clean":"2024-09-13T09:15:32.437400+0000","last_became_active":"2024-09-13T09:14:16.751511+0000","last_became_peered":"2024-09-13T09:14:16.751511+0000","last_unstale":"2024-09-13T09:15:32.437400+0000","last_undegraded":"2024-09-13T09:15:32.437400+0000","last_fullsized":"2024-09-13T09:15:32.437400+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'365","last_scrub_stamp":"2024-09-13T09:15:32.437377+0000","last_deep_scrub":"158'365","last_deep_scrub_stamp":"2024-09-13T09:15:32.437377+0000","last_clean_scrub_stamp":"2024-09-13T09:15:32.437377+0000","objects_scrubbed":2,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:31.710901+0000","scrub_duration":0.004142504,"objects_trimmed":0,"snaptrim_duration":0.0011111789999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3406,"num_read_kb":19855,"num_write":419,"num_write_kb":12151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":4294187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"158'382","reported_seq":5938,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:11.935243+0000","last_change":"2024-09-13T09:15:11.935386+0000","last_active":"2024-09-13T09:15:11.935243+0000","last_peered":"2024-09-13T09:15:11.935243+0000","last_clean":"2024-09-13T09:15:11.935243+0000","last_became_active":"2024-09-13T09:14:44.315188+0000","last_became_peered":"2024-09-13T09:14:44.315188+0000","last_unstale":"2024-09-13T09:15:11.935243+0000","last_undegraded":"2024-09-13T09:15:11.935243+0000","last_fullsized":"2024-09-13T09:15:11.935243+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'382","last_scrub_stamp":"2024-09-13T09:15:11.935201+0000","last_deep_scrub":"158'382","last_deep_scrub_stamp":"2024-09-13T09:15:11.935201+0000","last_clean_scrub_stamp":"2024-09-13T09:15:11.935201+0000","objects_scrubbed":1,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:02:29.994265+0000","scrub_duration":0.0028760159999999999,"objects_trimmed":2,"snaptrim_duration":0.0058820570000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6135,"num_read_kb":25216,"num_write":405,"num_write_kb":5645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"141'532","reported_seq":7051,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:38.720634+0000","last_change":"2024-09-13T09:15:38.720750+0000","last_active":"2024-09-13T09:15:38.720634+0000","last_peered":"2024-09-13T09:15:38.720634+0000","last_clean":"2024-09-13T09:15:38.720634+0000","last_became_active":"2024-09-13T09:14:44.309465+0000","last_became_peered":"2024-09-13T09:14:44.309465+0000","last_unstale":"2024-09-13T09:15:38.720634+0000","last_undegraded":"2024-09-13T09:15:38.720634+0000","last_fullsized":"2024-09-13T09:15:38.720634+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'532","last_scrub_stamp":"2024-09-13T09:15:38.720610+0000","last_deep_scrub":"141'532","last_deep_scrub_stamp":"2024-09-13T09:15:38.720610+0000","last_clean_scrub_stamp":"2024-09-13T09:15:38.720610+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:01.934793+0000","scrub_duration":0.0015927280000000001,"objects_trimmed":0,"snaptrim_duration":0.0039480829999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6633,"num_read_kb":20805,"num_write":596,"num_write_kb":7043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":635004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"158'535","reported_seq":7963,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:14.954879+0000","last_change":"2024-09-13T09:15:14.954979+0000","last_active":"2024-09-13T09:15:14.954879+0000","last_peered":"2024-09-13T09:15:14.954879+0000","last_clean":"2024-09-13T09:15:14.954879+0000","last_became_active":"2024-09-13T09:14:44.308779+0000","last_became_peered":"2024-09-13T09:14:44.308779+0000","last_unstale":"2024-09-13T09:15:14.954879+0000","last_undegraded":"2024-09-13T09:15:14.954879+0000","last_fullsized":"2024-09-13T09:15:14.954879+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'535","last_scrub_stamp":"2024-09-13T09:15:14.954854+0000","last_deep_scrub":"158'535","last_deep_scrub_stamp":"2024-09-13T09:15:14.954854+0000","last_clean_scrub_stamp":"2024-09-13T09:15:14.954854+0000","objects_scrubbed":2,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:09:48.360500+0000","scrub_duration":0.0044186119999999997,"objects_trimmed":4,"snaptrim_duration":0.0066958310000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7617,"num_read_kb":37352,"num_write":622,"num_write_kb":17983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"158'546","reported_seq":4851,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:15.327145+0000","last_change":"2024-09-13T09:15:15.327268+0000","last_active":"2024-09-13T09:15:15.327145+0000","last_peered":"2024-09-13T09:15:15.327145+0000","last_clean":"2024-09-13T09:15:15.327145+0000","last_became_active":"2024-09-13T09:12:43.532504+0000","last_became_peered":"2024-09-13T09:12:43.532504+0000","last_unstale":"2024-09-13T09:15:15.327145+0000","last_undegraded":"2024-09-13T09:15:15.327145+0000","last_fullsized":"2024-09-13T09:15:15.327145+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'546","last_scrub_stamp":"2024-09-13T09:15:15.327121+0000","last_deep_scrub":"158'546","last_deep_scrub_stamp":"2024-09-13T09:15:15.327121+0000","last_clean_scrub_stamp":"2024-09-13T09:15:15.327121+0000","objects_scrubbed":1,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:41.649933+0000","scrub_duration":0.0030860060000000001,"objects_trimmed":0,"snaptrim_duration":0.0021463989999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5509,"num_read_kb":32081,"num_write":673,"num_write_kb":7448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":694311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"158'1123","reported_seq":20809,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:10.744288+0000","last_change":"2024-09-13T09:15:10.744394+0000","last_active":"2024-09-13T09:15:10.744288+0000","last_peered":"2024-09-13T09:15:10.744288+0000","last_clean":"2024-09-13T09:15:10.744288+0000","last_became_active":"2024-09-13T09:12:46.248261+0000","last_became_peered":"2024-09-13T09:12:46.248261+0000","last_unstale":"2024-09-13T09:15:10.744288+0000","last_undegraded":"2024-09-13T09:15:10.744288+0000","last_fullsized":"2024-09-13T09:15:10.744288+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1123","last_scrub_stamp":"2024-09-13T09:15:10.744263+0000","last_deep_scrub":"158'1123","last_deep_scrub_stamp":"2024-09-13T09:15:10.744263+0000","last_clean_scrub_stamp":"2024-09-13T09:15:10.744263+0000","objects_scrubbed":8,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:50:35.274717+0000","scrub_duration":0.0081009050000000003,"objects_trimmed":0,"snaptrim_duration":0.00092792899999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":26946,"num_read_kb":67879,"num_write":1035,"num_write_kb":10041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":676864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"158'604","reported_seq":6579,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T09:15:33.438892+0000","last_change":"2024-09-13T09:15:33.438993+0000","last_active":"2024-09-13T09:15:33.438892+0000","last_peered":"2024-09-13T09:15:33.438892+0000","last_clean":"2024-09-13T09:15:33.438892+0000","last_became_active":"2024-09-13T09:14:44.314564+0000","last_became_peered":"2024-09-13T09:14:44.314564+0000","last_unstale":"2024-09-13T09:15:33.438892+0000","last_undegraded":"2024-09-13T09:15:33.438892+0000","last_fullsized":"2024-09-13T09:15:33.438892+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'604","last_scrub_stamp":"2024-09-13T09:15:33.438868+0000","last_deep_scrub":"158'604","last_deep_scrub_stamp":"2024-09-13T09:15:33.438868+0000","last_clean_scrub_stamp":"2024-09-13T09:15:33.438868+0000","objects_scrubbed":1,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:57.019221+0000","scrub_duration":0.003410471,"objects_trimmed":0,"snaptrim_duration":0.0050931029999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8526,"num_read_kb":13102,"num_write":519,"num_write_kb":3941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":28777,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":68,"num_read":356751,"num_read_kb":1364316,"num_write":26574,"num_write_kb":359719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":18167194,"num_keys_recovered":46,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28441,"internal_metadata":0},"log_size":23065,"ondisk_log_size":23065,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":146,"seq":627065225230,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31168,"kb_used_data":1848,"kb_used_omap":31,"kb_used_meta":29280,"kb_avail":94340672,"statfs":{"total":96636764160,"available":96604848128,"internally_reserved":0,"allocated":1892352,"data_stored":790048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32283,"internal_metadata":29983205},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640306,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64960,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306880,"statfs":{"total":96636764160,"available":96570245120,"internally_reserved":0,"allocated":1892352,"data_stored":790048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:15:01 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.41899999999999998,"15min":0.40400000000000003},"min":{"1min":0.28799999999999998,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.62},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.42099999999999999,"15min":0.39700000000000002},"min":{"1min":0.26200000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.34599999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:15:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.435,"15min":0.41199999999999998},"min":{"1min":0.29899999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.435,"15min":0.42499999999999999},"min":{"1min":0.29799999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:15:18 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.374}]},{"osd":3,"last update":"Fri Sep 13 09:15:01 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.33700000000000002},"min":{"1min":0.23599999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.316},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.253,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.629,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.35799999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.23799999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54400000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40000000000000002,"15min":0.40799999999999997},"min":{"1min":0.25,"5min":0.249,"15min":0.249},"max":{"1min":0.55400000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.39400000000000002}]}]},{"osd":3,"up_from":11,"seq":47244640307,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59768,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312072,"statfs":{"total":96636764160,"available":96575561728,"internally_reserved":0,"allocated":2473984,"data_stored":1378241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.40899999999999997},"min":{"1min":0.27400000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.72799999999999998,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48299999999999998,"15min":0.44700000000000001},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.77300000000000002,"5min":4.569,"15min":4.569},"last":0.437}]},{"osd":1,"last update":"Fri Sep 13 09:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55400000000000005,"15min":0.497},"min":{"1min":0.26100000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.4179999999999999,"5min":4.5990000000000002,"15min":4.5990000000000002},"last":0.503},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.44500000000000001,"15min":0.42899999999999999},"min":{"1min":0.32700000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.41599999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:15:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34699999999999998,"15min":0.35199999999999998},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.39900000000000002,"15min":0.377},"min":{"1min":0.23100000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.64900000000000002,"5min":5.165,"15min":5.165},"last":0.36299999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:14:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38300000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.45000000000000001,"15min":0.40300000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":8.7959999999999994,"5min":8.7959999999999994,"15min":8.7959999999999994},"last":0.45900000000000002}]}]},{"osd":2,"up_from":122,"seq":523986010132,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35068,"kb_used_data":1844,"kb_used_omap":37,"kb_used_meta":33178,"kb_avail":94336772,"statfs":{"total":96636764160,"available":96600854528,"internally_reserved":0,"allocated":1888256,"data_stored":790029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38061,"internal_metadata":33975123},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:15:17 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.317}]},{"osd":1,"last update":"Fri Sep 13 09:15:17 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.35799999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:15:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:15:17 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]},{"osd":1,"up_from":11,"seq":47244640306,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55108,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316732,"statfs":{"total":96636764160,"available":96580333568,"internally_reserved":0,"allocated":1896448,"data_stored":790067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:14:57 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.317,"15min":0.33300000000000002},"min":{"1min":0.219,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.41699999999999998,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.32400000000000001},"min":{"1min":0.22600000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.59999999999999998,"5min":0.753,"15min":0.753},"last":0.32700000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.309},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.38}]},{"osd":3,"last update":"Fri Sep 13 09:14:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.5,"15min":0.50800000000000001},"min":{"1min":0.34300000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49299999999999999,"15min":0.50600000000000001},"min":{"1min":0.375,"5min":0.219,"15min":0.219},"max":{"1min":0.66400000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:14:57 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.47099999999999997,"15min":0.47499999999999998},"min":{"1min":0.39200000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68400000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.5},{"interface":"front","average":{"1min":0.497,"5min":0.47399999999999998,"15min":0.48199999999999998},"min":{"1min":0.35699999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.71499999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.63600000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.496,"15min":0.504},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.747},"last":0.626},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.47099999999999997,"15min":0.48399999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.64700000000000002}]}]},{"osd":0,"up_from":11,"seq":47244640306,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63872,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307968,"statfs":{"total":96636764160,"available":96571359232,"internally_reserved":0,"allocated":2482176,"data_stored":1380397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:15:05 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.28799999999999998,"15min":0.28399999999999997},"min":{"1min":0.186,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.28899999999999998,"15min":0.28499999999999998},"min":{"1min":0.214,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.38900000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:15:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.46200000000000002,"15min":0.46400000000000002},"min":{"1min":0.32900000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.84899999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.46300000000000002,"15min":0.45200000000000001},"min":{"1min":0.35299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.745,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:15:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.46999999999999997,"15min":0.46300000000000002},"min":{"1min":0.35299999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.46300000000000002},"min":{"1min":0.35999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.57299999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.46700000000000003,"15min":0.47599999999999998},"min":{"1min":0.29199999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88700000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.438,"5min":0.45500000000000002,"15min":0.45800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69799999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.52200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16703,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11738,"internal_metadata":0}]}} 2024-09-13T09:15:46.455 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T09:15:46.752 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T09:15:46.752 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T09:15:46.753 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T09:15:46.753 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:15:46.789 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T09:15:46.789 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T09:15:46.789 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:15:46.860 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T09:15:46.861 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T09:15:46.861 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:15:46.935 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T09:15:46.935 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T09:15:46.935 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:15:46.987 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T09:15:46.988 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T09:15:46.988 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:15:47.040 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T09:15:47.040 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T09:15:47.040 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:15:47.084 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T09:15:47.085 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T09:15:47.085 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T09:15:47.085 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:15:47.114 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T09:15:47.114 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T09:15:47.114 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T09:15:47.114 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:15:47.126 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T09:15:47.126 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T09:15:47.126 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:15:47.178 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T09:15:47.178 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T09:15:47.178 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:15:47.190 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T09:15:47.190 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T09:15:47.191 DEBUG:teuthology.orchestra.run.smithi044:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-09-13T09:15:47.253 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T09:15:47.253 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T09:15:47.406 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T09:15:47.406 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T09:15:47.482 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T09:15:47.483 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T09:15:47.554 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T09:15:47.555 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T09:15:47.680 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T09:15:47.681 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T09:15:47.763 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T09:15:47.763 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T09:15:47.827 INFO:tasks.ceph:Archiving mon data... 2024-09-13T09:15:47.828 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020/data/mon.a.tgz 2024-09-13T09:15:47.828 DEBUG:teuthology.orchestra.run.smithi044:> mktemp 2024-09-13T09:15:47.844 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/tmp.4DfYZxDBkp 2024-09-13T09:15:47.845 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.4DfYZxDBkp 2024-09-13T09:15:47.992 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0666 /tmp/tmp.4DfYZxDBkp 2024-09-13T09:15:48.077 DEBUG:teuthology.orchestra.remote:smithi044:/tmp/tmp.4DfYZxDBkp is 323KB 2024-09-13T09:15:48.145 DEBUG:teuthology.orchestra.run.smithi044:> rm -fr /tmp/tmp.4DfYZxDBkp 2024-09-13T09:15:48.160 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020/data/mon.c.tgz 2024-09-13T09:15:48.160 DEBUG:teuthology.orchestra.run.smithi044:> mktemp 2024-09-13T09:15:48.215 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/tmp.ryPQVBMCEp 2024-09-13T09:15:48.216 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ryPQVBMCEp 2024-09-13T09:15:48.376 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0666 /tmp/tmp.ryPQVBMCEp 2024-09-13T09:15:48.461 DEBUG:teuthology.orchestra.remote:smithi044:/tmp/tmp.ryPQVBMCEp is 437KB 2024-09-13T09:15:48.558 DEBUG:teuthology.orchestra.run.smithi044:> rm -fr /tmp/tmp.ryPQVBMCEp 2024-09-13T09:15:48.574 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020/data/mon.b.tgz 2024-09-13T09:15:48.574 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-09-13T09:15:48.593 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.0nYxNhK4iL 2024-09-13T09:15:48.594 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.0nYxNhK4iL 2024-09-13T09:15:48.773 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.0nYxNhK4iL 2024-09-13T09:15:48.856 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.0nYxNhK4iL is 437KB 2024-09-13T09:15:48.953 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.0nYxNhK4iL 2024-09-13T09:15:48.968 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T09:15:48.968 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T09:15:48.971 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T09:15:49.010 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T09:15:49.042 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T09:15:49.042 DEBUG:teuthology.orchestra.run.smithi044:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T09:15:49.045 DEBUG:teuthology.orchestra.run.smithi129:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T09:15:49.072 INFO:teuthology.orchestra.run.smithi044.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T09:15:49.075 DEBUG:teuthology.orchestra.run.smithi157:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T09:15:49.099 INFO:teuthology.orchestra.run.smithi129.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T09:15:49.113 INFO:teuthology.orchestra.run.smithi157.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T09:15:49.119 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T09:15:49.120 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020/remote/smithi044/crash 2024-09-13T09:15:49.121 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T09:15:49.148 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020/remote/smithi129/crash 2024-09-13T09:15:49.149 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T09:15:49.174 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020/remote/smithi157/crash 2024-09-13T09:15:49.175 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T09:15:49.219 INFO:tasks.ceph:Compressing logs... 2024-09-13T09:15:49.220 DEBUG:teuthology.orchestra.run.smithi044:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:15:49.222 DEBUG:teuthology.orchestra.run.smithi129:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:15:49.225 DEBUG:teuthology.orchestra.run.smithi157:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:16:07.555 INFO:tasks.ceph:Archiving logs... 2024-09-13T09:16:07.556 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020/remote/smithi044/log 2024-09-13T09:16:07.557 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T09:16:09.197 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020/remote/smithi129/log 2024-09-13T09:16:09.198 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T09:16:11.059 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020/remote/smithi157/log 2024-09-13T09:16:11.060 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T09:16:11.211 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T09:16:11.226 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T09:16:11.226 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T09:16:11.229 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T09:16:11.232 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T09:16:11.328 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T09:16:11.337 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T09:16:11.337 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:16:11.340 DEBUG:teuthology.orchestra.run.smithi129:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:16:11.343 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:16:11.355 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-09-13T09:16:11.357 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-09-13T09:16:11.360 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-09-13T09:16:11.361 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T09:16:11.361 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-09-13T09:16:11.361 INFO:teuthology.orchestra.run.smithi044.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 +177us[ +173us] +/- 39ms 2024-09-13T09:16:11.361 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 -216us[ -216us] +/- 48ms 2024-09-13T09:16:11.361 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -56us[ -59us] +/- 72ms 2024-09-13T09:16:11.361 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T09:16:11.362 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T09:16:11.362 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-09-13T09:16:11.362 INFO:teuthology.orchestra.run.smithi129.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 +173us[ +172us] +/- 39ms 2024-09-13T09:16:11.362 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 -213us[ -213us] +/- 48ms 2024-09-13T09:16:11.362 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 -29us[ -30us] +/- 72ms 2024-09-13T09:16:11.362 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T09:16:11.365 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T09:16:11.365 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-09-13T09:16:11.365 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv01.front.sepia.ceph.com 3 6 377 35 +189us[ +199us] +/- 39ms 2024-09-13T09:16:11.365 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 -182us[ -182us] +/- 48ms 2024-09-13T09:16:11.366 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 -38us[ -38us] +/- 72ms 2024-09-13T09:16:11.366 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T09:16:11.366 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T09:16:11.376 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T09:16:11.376 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T09:16:11.384 DEBUG:teuthology.orchestra.run.smithi044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T09:16:11.486 DEBUG:teuthology.orchestra.run.smithi129:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T09:16:11.564 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T09:16:11.630 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T09:16:11.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:16:11.663 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T09:16:11.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:16:11.694 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T09:16:11.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:16:11.725 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T09:16:11.734 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T09:16:11.754 INFO:teuthology.task.internal:Duration was 655.100323 seconds 2024-09-13T09:16:11.755 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T09:16:11.764 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T09:16:11.764 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T09:16:11.767 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T09:16:11.770 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T09:16:11.808 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T09:16:11.809 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T09:16:11.812 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T09:16:12.143 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T09:16:12.143 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2024-09-13T09:16:12.144 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T09:16:12.197 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi129.front.sepia.ceph.com 2024-09-13T09:16:12.198 DEBUG:teuthology.orchestra.run.smithi129:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T09:16:12.222 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-09-13T09:16:12.223 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T09:16:12.250 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T09:16:12.250 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:16:12.252 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:16:12.265 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:16:12.318 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T09:16:12.318 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T09:16:12.321 DEBUG:teuthology.orchestra.run.smithi129:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T09:16:12.332 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T09:16:12.465 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T09:16:12.475 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T09:16:12.475 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T09:16:12.501 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T09:16:12.526 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T09:16:12.550 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T09:16:12.560 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:16:12.563 DEBUG:teuthology.orchestra.run.smithi129:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:16:12.568 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:16:12.585 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2024-09-13T09:16:12.588 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = core 2024-09-13T09:16:12.614 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-09-13T09:16:12.659 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:16:12.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:16:12.674 DEBUG:teuthology.orchestra.run.smithi129:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:16:12.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:16:12.688 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:16:12.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:16:12.713 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T09:16:12.724 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T09:16:12.725 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020/remote/smithi044 2024-09-13T09:16:12.725 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T09:16:12.761 DEBUG:teuthology.misc:Transferring archived files from smithi129:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020/remote/smithi129 2024-09-13T09:16:12.761 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T09:16:12.795 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901020/remote/smithi157 2024-09-13T09:16:12.796 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T09:16:12.834 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T09:16:12.834 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T09:16:12.836 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T09:16:12.839 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T09:16:12.882 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T09:16:12.896 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T09:16:12.896 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T09:16:12.906 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T09:16:12.906 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T09:16:12.908 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T09:16:12.910 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T09:16:12.922 INFO:teuthology.orchestra.run.smithi044.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 09:16 /home/ubuntu/cephtest 2024-09-13T09:16:12.923 INFO:teuthology.orchestra.run.smithi129.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 09:16 /home/ubuntu/cephtest 2024-09-13T09:16:12.937 INFO:teuthology.orchestra.run.smithi157.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 09:16 /home/ubuntu/cephtest 2024-09-13T09:16:12.938 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T09:16:12.948 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T09:16:12.970 INFO:teuthology.run:Summary data: description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} duration: 655.1003234386444 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-13T09:16:12.971 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T09:16:13.045 INFO:teuthology.run:pass